-- Logs begin at Tue 2022-08-02 09:25:21 UTC, end at Tue 2022-08-02 10:40:45 UTC. -- Aug 02 10:00:13.768766 dv-cinder-847132-4-iscsi systemd[1]: Starting Devstack devstack@keystone.service... Aug 02 10:00:13.782383 dv-cinder-847132-4-iscsi devstack@keystone.service[80363]: [uWSGI] getting INI configuration from /etc/keystone/keystone-uwsgi-public.ini Aug 02 10:00:13.785676 dv-cinder-847132-4-iscsi devstack@keystone.service[80363]: *** Starting uWSGI 2.0.18-debian (64bit) on [Tue Aug 2 10:00:13 2022] *** Aug 02 10:00:13.785676 dv-cinder-847132-4-iscsi devstack@keystone.service[80363]: compiled with version: 10.0.1 20200405 (experimental) [master revision 0be9efad938:fcb98e4978a:705510a708d3642c9c962beb663c476167e4e8a4] on 11 April 2020 11:15:55 Aug 02 10:00:13.785676 dv-cinder-847132-4-iscsi devstack@keystone.service[80363]: os: Linux-5.4.0-72-generic #80-Ubuntu SMP Mon Apr 12 17:35:00 UTC 2021 Aug 02 10:00:13.785676 dv-cinder-847132-4-iscsi devstack@keystone.service[80363]: nodename: dv-cinder-847132-4-iscsi Aug 02 10:00:13.785676 dv-cinder-847132-4-iscsi devstack@keystone.service[80363]: machine: x86_64 Aug 02 10:00:13.785676 dv-cinder-847132-4-iscsi devstack@keystone.service[80363]: clock source: unix Aug 02 10:00:13.785676 dv-cinder-847132-4-iscsi devstack@keystone.service[80363]: pcre jit disabled Aug 02 10:00:13.785676 dv-cinder-847132-4-iscsi devstack@keystone.service[80363]: detected number of CPU cores: 4 Aug 02 10:00:13.785676 dv-cinder-847132-4-iscsi devstack@keystone.service[80363]: current working directory: / Aug 02 10:00:13.785676 dv-cinder-847132-4-iscsi devstack@keystone.service[80363]: detected binary path: /usr/bin/uwsgi-core Aug 02 10:00:13.785676 dv-cinder-847132-4-iscsi devstack@keystone.service[80363]: your processes number limit is 47680 Aug 02 10:00:13.786361 dv-cinder-847132-4-iscsi devstack@keystone.service[80363]: your memory page size is 4096 bytes Aug 02 10:00:13.786361 dv-cinder-847132-4-iscsi devstack@keystone.service[80363]: detected max file descriptor number: 2048 Aug 02 10:00:13.786361 dv-cinder-847132-4-iscsi devstack@keystone.service[80363]: lock engine: pthread robust mutexes Aug 02 10:00:13.786361 dv-cinder-847132-4-iscsi devstack@keystone.service[80363]: thunder lock: enabled Aug 02 10:00:13.786361 dv-cinder-847132-4-iscsi devstack@keystone.service[80363]: uwsgi socket 0 bound to UNIX address /var/run/uwsgi/keystone-wsgi-public.socket fd 4 Aug 02 10:00:13.786687 dv-cinder-847132-4-iscsi devstack@keystone.service[80363]: Python version: 3.8.10 (default, Jun 22 2022, 20:18:18) [GCC 9.4.0] Aug 02 10:00:13.845001 dv-cinder-847132-4-iscsi devstack@keystone.service[80363]: Python main interpreter initialized at 0x55bf6677a9c0 Aug 02 10:00:13.845001 dv-cinder-847132-4-iscsi devstack@keystone.service[80363]: python threads support enabled Aug 02 10:00:13.845001 dv-cinder-847132-4-iscsi devstack@keystone.service[80363]: your server socket listen backlog is limited to 100 connections Aug 02 10:00:13.845001 dv-cinder-847132-4-iscsi devstack@keystone.service[80363]: your mercy for graceful operations on workers is 90 seconds Aug 02 10:00:13.845414 dv-cinder-847132-4-iscsi devstack@keystone.service[80363]: mapped 403077 bytes (393 KB) for 2 cores Aug 02 10:00:13.845502 dv-cinder-847132-4-iscsi devstack@keystone.service[80363]: *** Operational MODE: preforking *** Aug 02 10:00:13.845740 dv-cinder-847132-4-iscsi devstack@keystone.service[80363]: *** uWSGI is running in multiple interpreter mode *** Aug 02 10:00:13.845740 dv-cinder-847132-4-iscsi devstack@keystone.service[80363]: spawned uWSGI master process (pid: 80363) Aug 02 10:00:13.845978 dv-cinder-847132-4-iscsi systemd[1]: Started Devstack devstack@keystone.service. Aug 02 10:00:13.847969 dv-cinder-847132-4-iscsi devstack@keystone.service[80363]: spawned uWSGI worker 1 (pid: 80377, cores: 1) Aug 02 10:00:13.847969 dv-cinder-847132-4-iscsi devstack@keystone.service[80363]: spawned uWSGI worker 2 (pid: 80378, cores: 1) Aug 02 10:00:13.847969 dv-cinder-847132-4-iscsi devstack@keystone.service[80363]: running "unix_signal:15 gracefully_kill_them_all" (master-start)... Aug 02 10:00:20.818460 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: /usr/lib/python3/dist-packages/secretstorage/dhcrypto.py:15: CryptographyDeprecationWarning: int_from_bytes is deprecated, use int.from_bytes instead Aug 02 10:00:20.818460 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: from cryptography.utils import int_from_bytes Aug 02 10:00:20.819301 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: /usr/lib/python3/dist-packages/secretstorage/util.py:19: CryptographyDeprecationWarning: int_from_bytes is deprecated, use int.from_bytes instead Aug 02 10:00:20.819301 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: from cryptography.utils import int_from_bytes Aug 02 10:00:20.874255 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG uwsgi [-] ******************************************************************************** {{(pid=80377) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2589}} Aug 02 10:00:20.874520 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG uwsgi [-] Configuration options gathered from: {{(pid=80377) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2590}} Aug 02 10:00:20.874658 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG uwsgi [-] command line args: [] {{(pid=80377) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2591}} Aug 02 10:00:20.874978 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG uwsgi [-] config files: ['/etc/keystone/keystone.conf'] {{(pid=80377) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2592}} Aug 02 10:00:20.875113 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG uwsgi [-] ================================================================================ {{(pid=80377) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2594}} Aug 02 10:00:20.875355 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG uwsgi [-] admin_token = **** {{(pid=80377) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 02 10:00:20.875590 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG uwsgi [-] config_dir = [] {{(pid=80377) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 02 10:00:20.875822 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG uwsgi [-] config_file = ['/etc/keystone/keystone.conf'] {{(pid=80377) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 02 10:00:20.875963 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG uwsgi [-] config_source = [] {{(pid=80377) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 02 10:00:20.876109 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG uwsgi [-] debug = True {{(pid=80377) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 02 10:00:20.876388 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG uwsgi [-] default_log_levels = ['amqp=WARN', 'amqplib=WARN', 'boto=WARN', 'qpid=WARN', 'sqlalchemy=WARN', 'suds=INFO', 'oslo.messaging=INFO', 'oslo_messaging=INFO', 'iso8601=WARN', 'requests.packages.urllib3.connectionpool=WARN', 'urllib3.connectionpool=WARN', 'websocket=WARN', 'requests.packages.urllib3.util.retry=WARN', 'urllib3.util.retry=WARN', 'keystonemiddleware=WARN', 'routes.middleware=WARN', 'stevedore=WARN', 'taskflow=WARN', 'keystoneauth=WARN', 'oslo.cache=INFO', 'oslo_policy=INFO', 'dogpile.core.dogpile=INFO', 'dogpile=INFO', 'routes=INFO'] {{(pid=80377) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 02 10:00:20.878760 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG uwsgi [-] default_publisher_id = None {{(pid=80377) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 02 10:00:20.878760 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG uwsgi [-] insecure_debug = False {{(pid=80377) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 02 10:00:20.878760 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG uwsgi [-] instance_format = [instance: %(uuid)s] {{(pid=80377) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 02 10:00:20.878760 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG uwsgi [-] instance_uuid_format = [instance: %(uuid)s] {{(pid=80377) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 02 10:00:20.878760 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG uwsgi [-] list_limit = None {{(pid=80377) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 02 10:00:20.878760 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG uwsgi [-] log_config_append = None {{(pid=80377) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 02 10:00:20.878760 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG uwsgi [-] log_date_format = %Y-%m-%d %H:%M:%S {{(pid=80377) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 02 10:00:20.878760 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG uwsgi [-] log_dir = None {{(pid=80377) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 02 10:00:20.878760 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG uwsgi [-] log_file = None {{(pid=80377) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 02 10:00:20.879719 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG uwsgi [-] log_rotate_interval = 1 {{(pid=80377) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 02 10:00:20.879719 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG uwsgi [-] log_rotate_interval_type = days {{(pid=80377) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 02 10:00:20.879719 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG uwsgi [-] log_rotation_type = none {{(pid=80377) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 02 10:00:20.879719 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG uwsgi [-] logging_context_format_string = %(color)s%(levelname)s %(name)s [%(global_request_id)s %(request_id)s %(project_name)s %(user_name)s%(color)s] %(instance)s%(color)s%(message)s {{(pid=80377) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 02 10:00:20.879719 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG uwsgi [-] logging_debug_format_suffix = {{(pid=%(process)d) %(funcName)s %(pathname)s:%(lineno)d}} {{(pid=80377) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 02 10:00:20.879719 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG uwsgi [-] logging_default_format_string = %(color)s%(levelname)s %(name)s [-%(color)s] %(instance)s%(color)s%(message)s {{(pid=80377) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 02 10:00:20.880286 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG uwsgi [-] logging_exception_prefix = ERROR %(name)s %(instance)s {{(pid=80377) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 02 10:00:20.880286 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG uwsgi [-] logging_user_identity_format = %(user)s %(project)s %(domain)s %(system_scope)s %(user_domain)s %(project_domain)s {{(pid=80377) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 02 10:00:20.880286 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG uwsgi [-] max_logfile_count = 30 {{(pid=80377) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 02 10:00:20.880589 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG uwsgi [-] max_logfile_size_mb = 200 {{(pid=80377) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 02 10:00:20.880707 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG uwsgi [-] max_param_size = 64 {{(pid=80377) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 02 10:00:20.881475 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG uwsgi [-] max_project_tree_depth = 5 {{(pid=80377) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 02 10:00:20.881475 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG uwsgi [-] max_token_size = 16384 {{(pid=80377) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 02 10:00:20.881475 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG uwsgi [-] notification_format = cadf {{(pid=80377) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 02 10:00:20.881977 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG uwsgi [-] notification_opt_out = ['identity.authenticate.success', 'identity.authenticate.pending', 'identity.authenticate.failed'] {{(pid=80377) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 02 10:00:20.882225 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG uwsgi [-] public_endpoint = http://10.222.0.82/identity {{(pid=80377) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 02 10:00:20.882419 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG uwsgi [-] publish_errors = False {{(pid=80377) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 02 10:00:20.882639 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG uwsgi [-] pydev_debug_host = None {{(pid=80377) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 02 10:00:20.882878 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG uwsgi [-] pydev_debug_port = None {{(pid=80377) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 02 10:00:20.883079 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG uwsgi [-] rate_limit_burst = 0 {{(pid=80377) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 02 10:00:20.883298 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG uwsgi [-] rate_limit_except_level = CRITICAL {{(pid=80377) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 02 10:00:20.883535 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG uwsgi [-] rate_limit_interval = 0 {{(pid=80377) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 02 10:00:20.883773 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG uwsgi [-] standard_threads = False {{(pid=80377) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 02 10:00:20.884006 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG uwsgi [-] strict_password_check = False {{(pid=80377) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 02 10:00:20.884435 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG uwsgi [-] syslog_log_facility = LOG_USER {{(pid=80377) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 02 10:00:20.884630 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG uwsgi [-] use_eventlog = False {{(pid=80377) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 02 10:00:20.884828 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG uwsgi [-] use_journal = False {{(pid=80377) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 02 10:00:20.885020 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG uwsgi [-] use_json = False {{(pid=80377) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 02 10:00:20.885213 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG uwsgi [-] use_stderr = False {{(pid=80377) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 02 10:00:20.885399 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG uwsgi [-] use_syslog = False {{(pid=80377) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 02 10:00:20.885529 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG uwsgi [-] watch_log_file = False {{(pid=80377) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 02 10:00:20.885820 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG uwsgi [-] oslo_concurrency.disable_process_locking = False {{(pid=80377) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 02 10:00:20.886073 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG uwsgi [-] oslo_concurrency.lock_path = None {{(pid=80377) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 02 10:00:20.886512 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG uwsgi [-] profiler.connection_string = messaging:// {{(pid=80377) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 02 10:00:20.886956 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG uwsgi [-] profiler.enabled = False {{(pid=80377) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 02 10:00:20.887352 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG uwsgi [-] profiler.es_doc_type = notification {{(pid=80377) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 02 10:00:20.887765 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG uwsgi [-] profiler.es_scroll_size = 10000 {{(pid=80377) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 02 10:00:20.888106 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG uwsgi [-] profiler.es_scroll_time = 2m {{(pid=80377) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 02 10:00:20.888405 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG uwsgi [-] profiler.filter_error_trace = False {{(pid=80377) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 02 10:00:20.888694 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG uwsgi [-] profiler.hmac_keys = SECRET_KEY {{(pid=80377) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 02 10:00:20.889116 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG uwsgi [-] profiler.sentinel_service_name = mymaster {{(pid=80377) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 02 10:00:20.889591 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG uwsgi [-] profiler.socket_timeout = 0.1 {{(pid=80377) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 02 10:00:20.890007 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG uwsgi [-] profiler.trace_sqlalchemy = False {{(pid=80377) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 02 10:00:20.890481 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_buffer_size = 1000 {{(pid=80377) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 02 10:00:20.891308 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_enabled = False {{(pid=80377) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 02 10:00:20.891888 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_process_name = {{(pid=80377) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 02 10:00:20.892138 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_socket_file = /var/tmp/metrics_collector.sock {{(pid=80377) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 02 10:00:20.893419 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_thread_stop_timeout = 10 {{(pid=80377) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 02 10:00:20.893419 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG uwsgi [-] oslo_middleware.http_basic_auth_user_file = /etc/htpasswd {{(pid=80377) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 02 10:00:20.893419 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG uwsgi [-] audit.namespace = openstack {{(pid=80377) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 02 10:00:20.893419 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG uwsgi [-] oslo_policy.enforce_new_defaults = False {{(pid=80377) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 02 10:00:20.893941 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG uwsgi [-] oslo_policy.enforce_scope = False {{(pid=80377) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 02 10:00:20.894049 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG uwsgi [-] oslo_policy.policy_default_rule = default {{(pid=80377) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 02 10:00:20.894342 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG uwsgi [-] oslo_policy.policy_dirs = ['policy.d'] {{(pid=80377) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 02 10:00:20.894662 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG uwsgi [-] oslo_policy.policy_file = policy.yaml {{(pid=80377) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 02 10:00:20.894965 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG uwsgi [-] oslo_policy.remote_content_type = application/x-www-form-urlencoded {{(pid=80377) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 02 10:00:20.895235 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG uwsgi [-] oslo_policy.remote_ssl_ca_crt_file = None {{(pid=80377) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 02 10:00:20.895508 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG uwsgi [-] oslo_policy.remote_ssl_client_crt_file = None {{(pid=80377) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 02 10:00:20.895755 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG uwsgi [-] oslo_policy.remote_ssl_client_key_file = None {{(pid=80377) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 02 10:00:20.896181 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG uwsgi [-] oslo_policy.remote_ssl_verify_server_crt = False {{(pid=80377) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 02 10:00:20.896646 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG uwsgi [-] keystone_authtoken.admin_password = **** {{(pid=80377) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 02 10:00:20.897296 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG uwsgi [-] keystone_authtoken.admin_tenant_name = admin {{(pid=80377) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 02 10:00:20.897561 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG uwsgi [-] keystone_authtoken.admin_token = **** {{(pid=80377) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 02 10:00:20.898014 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG uwsgi [-] keystone_authtoken.admin_user = None {{(pid=80377) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 02 10:00:20.898644 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG uwsgi [-] keystone_authtoken.auth_admin_prefix = {{(pid=80377) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 02 10:00:20.898920 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG uwsgi [-] keystone_authtoken.auth_host = 127.0.0.1 {{(pid=80377) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 02 10:00:20.899380 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG uwsgi [-] keystone_authtoken.auth_port = 35357 {{(pid=80377) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 02 10:00:20.899857 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG uwsgi [-] keystone_authtoken.auth_protocol = https {{(pid=80377) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 02 10:00:20.900396 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG uwsgi [-] keystone_authtoken.auth_section = None {{(pid=80377) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 02 10:00:20.900576 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG uwsgi [-] keystone_authtoken.auth_type = None {{(pid=80377) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 02 10:00:20.900862 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG uwsgi [-] keystone_authtoken.auth_uri = None {{(pid=80377) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 02 10:00:20.901304 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG uwsgi [-] keystone_authtoken.auth_version = None {{(pid=80377) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 02 10:00:20.901746 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG uwsgi [-] keystone_authtoken.cache = None {{(pid=80377) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 02 10:00:20.902236 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG uwsgi [-] keystone_authtoken.cafile = None {{(pid=80377) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 02 10:00:20.902632 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG uwsgi [-] keystone_authtoken.certfile = None {{(pid=80377) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 02 10:00:20.902898 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG uwsgi [-] keystone_authtoken.delay_auth_decision = False {{(pid=80377) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 02 10:00:20.903164 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG uwsgi [-] keystone_authtoken.enforce_token_bind = permissive {{(pid=80377) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 02 10:00:20.903441 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG uwsgi [-] keystone_authtoken.http_connect_timeout = None {{(pid=80377) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 02 10:00:20.903907 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG uwsgi [-] keystone_authtoken.http_request_max_retries = 3 {{(pid=80377) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 02 10:00:20.904298 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG uwsgi [-] keystone_authtoken.identity_uri = None {{(pid=80377) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 02 10:00:20.904754 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG uwsgi [-] keystone_authtoken.include_service_catalog = True {{(pid=80377) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 02 10:00:20.905511 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG uwsgi [-] keystone_authtoken.insecure = False {{(pid=80377) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 02 10:00:20.905993 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG uwsgi [-] keystone_authtoken.interface = internal {{(pid=80377) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 02 10:00:20.906436 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG uwsgi [-] keystone_authtoken.keyfile = None {{(pid=80377) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 02 10:00:20.906899 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_conn_get_timeout = 10 {{(pid=80377) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 02 10:00:20.907350 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_dead_retry = 300 {{(pid=80377) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 02 10:00:20.907810 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_maxsize = 10 {{(pid=80377) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 02 10:00:20.908262 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_socket_timeout = 3 {{(pid=80377) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 02 10:00:20.909294 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_unused_timeout = 60 {{(pid=80377) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 02 10:00:20.909710 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG uwsgi [-] keystone_authtoken.memcache_secret_key = **** {{(pid=80377) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 02 10:00:20.910224 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG uwsgi [-] keystone_authtoken.memcache_security_strategy = None {{(pid=80377) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 02 10:00:20.910686 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG uwsgi [-] keystone_authtoken.memcache_use_advanced_pool = True {{(pid=80377) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 02 10:00:20.911146 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG uwsgi [-] keystone_authtoken.memcached_servers = None {{(pid=80377) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 02 10:00:20.911592 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG uwsgi [-] keystone_authtoken.region_name = None {{(pid=80377) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 02 10:00:20.912097 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG uwsgi [-] keystone_authtoken.service_token_roles = ['service'] {{(pid=80377) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 02 10:00:20.912558 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG uwsgi [-] keystone_authtoken.service_token_roles_required = False {{(pid=80377) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 02 10:00:20.912999 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG uwsgi [-] keystone_authtoken.service_type = None {{(pid=80377) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 02 10:00:20.913460 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG uwsgi [-] keystone_authtoken.token_cache_time = 300 {{(pid=80377) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 02 10:00:20.913946 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG uwsgi [-] keystone_authtoken.www_authenticate_uri = None {{(pid=80377) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 02 10:00:20.914465 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG uwsgi [-] cache.backend = dogpile.cache.memcached {{(pid=80377) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 02 10:00:20.914928 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG uwsgi [-] cache.backend_argument = **** {{(pid=80377) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 02 10:00:20.915404 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG uwsgi [-] cache.config_prefix = cache.oslo {{(pid=80377) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 02 10:00:20.915875 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG uwsgi [-] cache.dead_timeout = 60.0 {{(pid=80377) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 02 10:00:20.916278 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG uwsgi [-] cache.debug_cache_backend = False {{(pid=80377) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 02 10:00:20.916691 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG uwsgi [-] cache.enable_retry_client = False {{(pid=80377) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 02 10:00:20.917017 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG uwsgi [-] cache.enable_socket_keepalive = False {{(pid=80377) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 02 10:00:20.917456 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG uwsgi [-] cache.enabled = True {{(pid=80377) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 02 10:00:20.917718 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG uwsgi [-] cache.expiration_time = 600 {{(pid=80377) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 02 10:00:20.918020 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG uwsgi [-] cache.hashclient_retry_attempts = 2 {{(pid=80377) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 02 10:00:20.918284 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG uwsgi [-] cache.hashclient_retry_delay = 1.0 {{(pid=80377) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 02 10:00:20.918638 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG uwsgi [-] cache.memcache_dead_retry = 300 {{(pid=80377) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 02 10:00:20.918987 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG uwsgi [-] cache.memcache_pool_connection_get_timeout = 10 {{(pid=80377) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 02 10:00:20.919246 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG uwsgi [-] cache.memcache_pool_flush_on_reconnect = False {{(pid=80377) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 02 10:00:20.919586 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG uwsgi [-] cache.memcache_pool_maxsize = 10 {{(pid=80377) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 02 10:00:20.919987 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG uwsgi [-] cache.memcache_pool_unused_timeout = 60 {{(pid=80377) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 02 10:00:20.920504 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG uwsgi [-] cache.memcache_servers = ['localhost:11211'] {{(pid=80377) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 02 10:00:20.920960 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG uwsgi [-] cache.memcache_socket_timeout = 1.0 {{(pid=80377) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 02 10:00:20.921415 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG uwsgi [-] cache.proxies = [] {{(pid=80377) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 02 10:00:20.921883 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG uwsgi [-] cache.retry_attempts = 2 {{(pid=80377) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 02 10:00:20.922348 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG uwsgi [-] cache.retry_delay = 0.0 {{(pid=80377) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 02 10:00:20.922709 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG uwsgi [-] cache.socket_keepalive_count = 1 {{(pid=80377) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 02 10:00:20.923102 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG uwsgi [-] cache.socket_keepalive_idle = 1 {{(pid=80377) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 02 10:00:20.923558 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG uwsgi [-] cache.socket_keepalive_interval = 1 {{(pid=80377) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 02 10:00:20.924011 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG uwsgi [-] cache.tls_allowed_ciphers = None {{(pid=80377) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 02 10:00:20.924453 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG uwsgi [-] cache.tls_cafile = None {{(pid=80377) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 02 10:00:20.924895 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG uwsgi [-] cache.tls_certfile = None {{(pid=80377) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 02 10:00:20.925348 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG uwsgi [-] cache.tls_enabled = False {{(pid=80377) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 02 10:00:20.925808 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG uwsgi [-] cache.tls_keyfile = None {{(pid=80377) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 02 10:00:20.926267 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG uwsgi [-] application_credential.cache_time = None {{(pid=80377) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 02 10:00:20.926698 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG uwsgi [-] application_credential.caching = True {{(pid=80377) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 02 10:00:20.927187 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG uwsgi [-] application_credential.driver = sql {{(pid=80377) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 02 10:00:20.927648 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG uwsgi [-] application_credential.user_limit = -1 {{(pid=80377) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 02 10:00:20.928145 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG uwsgi [-] assignment.driver = sql {{(pid=80377) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 02 10:00:20.928653 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG uwsgi [-] assignment.prohibited_implied_role = ['admin'] {{(pid=80377) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 02 10:00:20.929122 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG uwsgi [-] auth.application_credential = None {{(pid=80377) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 02 10:00:20.929573 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG uwsgi [-] auth.external = None {{(pid=80377) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 02 10:00:20.930057 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG uwsgi [-] auth.mapped = None {{(pid=80377) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 02 10:00:20.930664 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG uwsgi [-] auth.methods = ['external', 'password', 'token', 'oauth1', 'mapped', 'application_credential'] {{(pid=80377) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 02 10:00:20.931109 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG uwsgi [-] auth.oauth1 = None {{(pid=80377) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 02 10:00:20.931560 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG uwsgi [-] auth.password = None {{(pid=80377) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 02 10:00:20.932582 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG uwsgi [-] auth.token = None {{(pid=80377) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 02 10:00:20.933020 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG uwsgi [-] catalog.cache_time = None {{(pid=80377) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 02 10:00:20.933479 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG uwsgi [-] catalog.caching = True {{(pid=80377) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 02 10:00:20.933984 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG uwsgi [-] catalog.driver = sql {{(pid=80377) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 02 10:00:20.934426 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG uwsgi [-] catalog.list_limit = None {{(pid=80377) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 02 10:00:20.934902 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG uwsgi [-] catalog.template_file = default_catalog.templates {{(pid=80377) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 02 10:00:20.935385 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG uwsgi [-] credential.auth_ttl = 15 {{(pid=80377) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 02 10:00:20.935833 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG uwsgi [-] credential.cache_time = None {{(pid=80377) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 02 10:00:20.936280 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG uwsgi [-] credential.caching = True {{(pid=80377) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 02 10:00:20.936753 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG uwsgi [-] credential.driver = sql {{(pid=80377) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 02 10:00:20.937186 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG uwsgi [-] credential.key_repository = /etc/keystone/credential-keys/ {{(pid=80377) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 02 10:00:20.937498 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG uwsgi [-] credential.provider = fernet {{(pid=80377) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 02 10:00:20.937773 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG uwsgi [-] credential.user_limit = -1 {{(pid=80377) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 02 10:00:20.938059 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG uwsgi [-] domain_config.cache_time = 300 {{(pid=80377) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 02 10:00:20.938312 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG uwsgi [-] domain_config.caching = True {{(pid=80377) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 02 10:00:20.938625 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG uwsgi [-] domain_config.driver = sql {{(pid=80377) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 02 10:00:20.939112 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG uwsgi [-] endpoint_filter.driver = sql {{(pid=80377) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 02 10:00:20.939540 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG uwsgi [-] endpoint_filter.return_all_endpoints_if_no_filter = True {{(pid=80377) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 02 10:00:20.939882 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG uwsgi [-] endpoint_policy.driver = sql {{(pid=80377) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 02 10:00:20.940316 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG uwsgi [-] eventlet_server.admin_bind_host = 0.0.0.0 {{(pid=80377) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 02 10:00:20.940672 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG uwsgi [-] eventlet_server.admin_port = 35357 {{(pid=80377) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 02 10:00:20.941062 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG uwsgi [-] eventlet_server.public_bind_host = 0.0.0.0 {{(pid=80377) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 02 10:00:20.941466 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG uwsgi [-] eventlet_server.public_port = 5000 {{(pid=80377) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 02 10:00:20.941911 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG uwsgi [-] federation.assertion_prefix = {{(pid=80377) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 02 10:00:20.942322 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG uwsgi [-] federation.caching = True {{(pid=80377) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 02 10:00:20.942731 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG uwsgi [-] federation.default_authorization_ttl = 0 {{(pid=80377) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 02 10:00:20.943068 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG uwsgi [-] federation.driver = sql {{(pid=80377) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 02 10:00:20.943503 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG uwsgi [-] federation.federated_domain_name = Federated {{(pid=80377) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 02 10:00:20.943900 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG uwsgi [-] federation.remote_id_attribute = None {{(pid=80377) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 02 10:00:20.944310 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG uwsgi [-] federation.sso_callback_template = /etc/keystone/sso_callback_template.html {{(pid=80377) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 02 10:00:20.944545 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG uwsgi [-] federation.trusted_dashboard = [] {{(pid=80377) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 02 10:00:20.944817 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG uwsgi [-] fernet_receipts.key_repository = /etc/keystone/fernet-keys/ {{(pid=80377) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 02 10:00:20.945069 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG uwsgi [-] fernet_receipts.max_active_keys = 3 {{(pid=80377) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 02 10:00:20.945338 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG uwsgi [-] fernet_tokens.key_repository = /etc/keystone/fernet-keys/ {{(pid=80377) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 02 10:00:20.945584 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG uwsgi [-] fernet_tokens.max_active_keys = 3 {{(pid=80377) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 02 10:00:20.945867 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG uwsgi [-] identity.cache_time = 600 {{(pid=80377) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 02 10:00:20.946115 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG uwsgi [-] identity.caching = True {{(pid=80377) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 02 10:00:20.946376 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG uwsgi [-] identity.default_domain_id = default {{(pid=80377) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 02 10:00:20.946627 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG uwsgi [-] identity.domain_config_dir = /etc/keystone/domains {{(pid=80377) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 02 10:00:20.946872 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG uwsgi [-] identity.domain_configurations_from_database = False {{(pid=80377) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 02 10:00:20.947114 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG uwsgi [-] identity.domain_specific_drivers_enabled = False {{(pid=80377) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 02 10:00:20.947373 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG uwsgi [-] identity.driver = sql {{(pid=80377) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 02 10:00:20.947623 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG uwsgi [-] identity.list_limit = None {{(pid=80377) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 02 10:00:20.947861 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG uwsgi [-] identity.max_password_length = 4096 {{(pid=80377) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 02 10:00:20.948115 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG uwsgi [-] identity.password_hash_algorithm = bcrypt {{(pid=80377) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 02 10:00:20.948372 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG uwsgi [-] identity.password_hash_rounds = 4 {{(pid=80377) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 02 10:00:20.948610 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG uwsgi [-] identity.salt_bytesize = None {{(pid=80377) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 02 10:00:20.948847 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG uwsgi [-] identity.scrypt_block_size = None {{(pid=80377) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 02 10:00:20.949081 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG uwsgi [-] identity.scrypt_parallelism = None {{(pid=80377) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 02 10:00:20.949331 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG uwsgi [-] identity_mapping.backward_compatible_ids = True {{(pid=80377) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 02 10:00:20.949583 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG uwsgi [-] identity_mapping.driver = sql {{(pid=80377) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 02 10:00:20.949962 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG uwsgi [-] identity_mapping.generator = sha256 {{(pid=80377) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 02 10:00:20.950196 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG uwsgi [-] jwt_tokens.jws_private_key_repository = /etc/keystone/jws-keys/private {{(pid=80377) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 02 10:00:20.950452 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG uwsgi [-] jwt_tokens.jws_public_key_repository = /etc/keystone/jws-keys/public {{(pid=80377) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 02 10:00:20.950937 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG uwsgi [-] ldap.alias_dereferencing = default {{(pid=80377) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 02 10:00:20.950937 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG uwsgi [-] ldap.auth_pool_connection_lifetime = 60 {{(pid=80377) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 02 10:00:20.951305 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG uwsgi [-] ldap.auth_pool_size = 100 {{(pid=80377) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 02 10:00:20.951499 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG uwsgi [-] ldap.chase_referrals = None {{(pid=80377) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 02 10:00:20.951763 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG uwsgi [-] ldap.connection_timeout = -1 {{(pid=80377) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 02 10:00:20.952003 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG uwsgi [-] ldap.debug_level = None {{(pid=80377) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 02 10:00:20.952259 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG uwsgi [-] ldap.group_ad_nesting = False {{(pid=80377) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 02 10:00:20.952552 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG uwsgi [-] ldap.group_additional_attribute_mapping = [] {{(pid=80377) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 02 10:00:20.952739 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG uwsgi [-] ldap.group_attribute_ignore = [] {{(pid=80377) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 02 10:00:20.952920 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG uwsgi [-] ldap.group_desc_attribute = description {{(pid=80377) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 02 10:00:20.953477 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG uwsgi [-] ldap.group_filter = None {{(pid=80377) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 02 10:00:20.953477 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG uwsgi [-] ldap.group_id_attribute = cn {{(pid=80377) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 02 10:00:20.953685 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG uwsgi [-] ldap.group_member_attribute = member {{(pid=80377) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 02 10:00:20.953921 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG uwsgi [-] ldap.group_members_are_ids = False {{(pid=80377) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 02 10:00:20.954132 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG uwsgi [-] ldap.group_name_attribute = ou {{(pid=80377) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 02 10:00:20.954391 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG uwsgi [-] ldap.group_objectclass = groupOfNames {{(pid=80377) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 02 10:00:20.954619 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG uwsgi [-] ldap.group_tree_dn = None {{(pid=80377) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 02 10:00:20.954862 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG uwsgi [-] ldap.page_size = 0 {{(pid=80377) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 02 10:00:20.955096 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG uwsgi [-] ldap.password = **** {{(pid=80377) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 02 10:00:20.955341 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG uwsgi [-] ldap.pool_connection_lifetime = 600 {{(pid=80377) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 02 10:00:20.955607 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG uwsgi [-] ldap.pool_connection_timeout = -1 {{(pid=80377) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 02 10:00:20.955859 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG uwsgi [-] ldap.pool_retry_delay = 0.1 {{(pid=80377) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 02 10:00:20.956097 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG uwsgi [-] ldap.pool_retry_max = 3 {{(pid=80377) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 02 10:00:20.956336 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG uwsgi [-] ldap.pool_size = 10 {{(pid=80377) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 02 10:00:20.956590 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG uwsgi [-] ldap.query_scope = one {{(pid=80377) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 02 10:00:20.956844 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG uwsgi [-] ldap.suffix = cn=example,cn=com {{(pid=80377) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 02 10:00:20.957079 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG uwsgi [-] ldap.tls_cacertdir = None {{(pid=80377) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 02 10:00:20.957314 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG uwsgi [-] ldap.tls_cacertfile = None {{(pid=80377) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 02 10:00:20.957569 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG uwsgi [-] ldap.tls_req_cert = demand {{(pid=80377) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 02 10:00:20.957840 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG uwsgi [-] ldap.url = ldap://localhost {{(pid=80377) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 02 10:00:20.958084 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG uwsgi [-] ldap.use_auth_pool = True {{(pid=80377) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 02 10:00:20.958323 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG uwsgi [-] ldap.use_pool = True {{(pid=80377) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 02 10:00:20.958892 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG uwsgi [-] ldap.use_tls = False {{(pid=80377) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 02 10:00:20.958892 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG uwsgi [-] ldap.user = None {{(pid=80377) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 02 10:00:20.959316 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG uwsgi [-] ldap.user_additional_attribute_mapping = [] {{(pid=80377) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 02 10:00:20.959316 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG uwsgi [-] ldap.user_attribute_ignore = ['default_project_id'] {{(pid=80377) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 02 10:00:20.959513 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG uwsgi [-] ldap.user_default_project_id_attribute = None {{(pid=80377) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 02 10:00:20.959746 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG uwsgi [-] ldap.user_description_attribute = description {{(pid=80377) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 02 10:00:20.959999 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG uwsgi [-] ldap.user_enabled_attribute = enabled {{(pid=80377) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 02 10:00:20.960445 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG uwsgi [-] ldap.user_enabled_default = True {{(pid=80377) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 02 10:00:20.960445 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG uwsgi [-] ldap.user_enabled_emulation = False {{(pid=80377) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 02 10:00:20.960924 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG uwsgi [-] ldap.user_enabled_emulation_dn = None {{(pid=80377) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 02 10:00:20.960924 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG uwsgi [-] ldap.user_enabled_emulation_use_group_config = False {{(pid=80377) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 02 10:00:20.961173 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG uwsgi [-] ldap.user_enabled_invert = False {{(pid=80377) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 02 10:00:20.961410 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG uwsgi [-] ldap.user_enabled_mask = 0 {{(pid=80377) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 02 10:00:20.961646 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG uwsgi [-] ldap.user_filter = None {{(pid=80377) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 02 10:00:20.961923 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG uwsgi [-] ldap.user_id_attribute = cn {{(pid=80377) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 02 10:00:20.962176 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG uwsgi [-] ldap.user_mail_attribute = mail {{(pid=80377) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 02 10:00:20.962427 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG uwsgi [-] ldap.user_name_attribute = sn {{(pid=80377) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 02 10:00:20.962836 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG uwsgi [-] ldap.user_objectclass = inetOrgPerson {{(pid=80377) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 02 10:00:20.963092 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG uwsgi [-] ldap.user_pass_attribute = userPassword {{(pid=80377) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 02 10:00:20.963334 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG uwsgi [-] ldap.user_tree_dn = None {{(pid=80377) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 02 10:00:20.963595 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG uwsgi [-] memcache.dead_retry = 300 {{(pid=80377) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 02 10:00:20.963838 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG uwsgi [-] memcache.pool_connection_get_timeout = 10 {{(pid=80377) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 02 10:00:20.964077 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG uwsgi [-] memcache.pool_maxsize = 10 {{(pid=80377) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 02 10:00:20.964325 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG uwsgi [-] memcache.pool_unused_timeout = 60 {{(pid=80377) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 02 10:00:20.964568 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG uwsgi [-] memcache.socket_timeout = 3 {{(pid=80377) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 02 10:00:20.964819 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG uwsgi [-] oauth1.access_token_duration = 86400 {{(pid=80377) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 02 10:00:20.965072 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG uwsgi [-] oauth1.driver = sql {{(pid=80377) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 02 10:00:20.965309 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG uwsgi [-] oauth1.request_token_duration = 28800 {{(pid=80377) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 02 10:00:20.965599 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG uwsgi [-] policy.driver = sql {{(pid=80377) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 02 10:00:20.965819 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG uwsgi [-] policy.list_limit = None {{(pid=80377) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 02 10:00:20.966074 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG uwsgi [-] receipt.cache_on_issue = True {{(pid=80377) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 02 10:00:20.966602 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG uwsgi [-] receipt.cache_time = 300 {{(pid=80377) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 02 10:00:20.966602 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG uwsgi [-] receipt.caching = True {{(pid=80377) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 02 10:00:20.966882 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG uwsgi [-] receipt.expiration = 300 {{(pid=80377) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 02 10:00:20.967014 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG uwsgi [-] receipt.provider = fernet {{(pid=80377) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 02 10:00:20.967261 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG uwsgi [-] resource.admin_project_domain_name = None {{(pid=80377) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 02 10:00:20.967515 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG uwsgi [-] resource.admin_project_name = None {{(pid=80377) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 02 10:00:20.967805 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG uwsgi [-] resource.cache_time = None {{(pid=80377) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 02 10:00:20.967976 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG uwsgi [-] resource.caching = True {{(pid=80377) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 02 10:00:20.968196 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG uwsgi [-] resource.domain_name_url_safe = off {{(pid=80377) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 02 10:00:20.968534 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG uwsgi [-] resource.driver = sql {{(pid=80377) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 02 10:00:20.968707 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG uwsgi [-] resource.list_limit = None {{(pid=80377) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 02 10:00:20.969006 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG uwsgi [-] resource.project_name_url_safe = off {{(pid=80377) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 02 10:00:20.969304 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG uwsgi [-] revoke.cache_time = 3600 {{(pid=80377) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 02 10:00:20.969599 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG uwsgi [-] revoke.caching = True {{(pid=80377) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 02 10:00:20.970102 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG uwsgi [-] revoke.driver = sql {{(pid=80377) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 02 10:00:20.970243 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG uwsgi [-] revoke.expiration_buffer = 1800 {{(pid=80377) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 02 10:00:20.970516 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG uwsgi [-] role.cache_time = None {{(pid=80377) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 02 10:00:20.970895 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG uwsgi [-] role.caching = True {{(pid=80377) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 02 10:00:20.971229 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG uwsgi [-] role.driver = sql {{(pid=80377) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 02 10:00:20.971605 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG uwsgi [-] role.list_limit = None {{(pid=80377) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 02 10:00:20.972058 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG uwsgi [-] saml.assertion_expiration_time = 3600 {{(pid=80377) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 02 10:00:20.972448 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG uwsgi [-] saml.certfile = /etc/keystone/ssl/certs/signing_cert.pem {{(pid=80377) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 02 10:00:20.972801 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG uwsgi [-] saml.idp_contact_company = Example, Inc. {{(pid=80377) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 02 10:00:20.973153 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG uwsgi [-] saml.idp_contact_email = support@example.com {{(pid=80377) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 02 10:00:20.973500 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG uwsgi [-] saml.idp_contact_name = SAML Identity Provider Support {{(pid=80377) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 02 10:00:20.973895 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG uwsgi [-] saml.idp_contact_surname = Support {{(pid=80377) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 02 10:00:20.974255 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG uwsgi [-] saml.idp_contact_telephone = +1 800 555 0100 {{(pid=80377) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 02 10:00:20.974608 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG uwsgi [-] saml.idp_contact_type = other {{(pid=80377) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 02 10:00:20.974940 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG uwsgi [-] saml.idp_entity_id = None {{(pid=80377) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 02 10:00:20.975289 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG uwsgi [-] saml.idp_lang = en {{(pid=80377) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 02 10:00:20.975637 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG uwsgi [-] saml.idp_metadata_path = /etc/keystone/saml2_idp_metadata.xml {{(pid=80377) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 02 10:00:20.975994 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG uwsgi [-] saml.idp_organization_display_name = OpenStack SAML Identity Provider {{(pid=80377) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 02 10:00:20.976342 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG uwsgi [-] saml.idp_organization_name = SAML Identity Provider {{(pid=80377) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 02 10:00:20.976886 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG uwsgi [-] saml.idp_organization_url = https://example.com/ {{(pid=80377) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 02 10:00:20.977304 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG uwsgi [-] saml.idp_sso_endpoint = None {{(pid=80377) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 02 10:00:20.977616 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG uwsgi [-] saml.keyfile = /etc/keystone/ssl/private/signing_key.pem {{(pid=80377) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 02 10:00:20.977949 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG uwsgi [-] saml.relay_state_prefix = ss:mem: {{(pid=80377) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 02 10:00:20.978252 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG uwsgi [-] saml.xmlsec1_binary = xmlsec1 {{(pid=80377) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 02 10:00:20.978559 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG uwsgi [-] security_compliance.change_password_upon_first_use = False {{(pid=80377) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 02 10:00:20.978850 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG uwsgi [-] security_compliance.disable_user_account_days_inactive = None {{(pid=80377) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 02 10:00:20.979207 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG uwsgi [-] security_compliance.lockout_duration = 10 {{(pid=80377) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 02 10:00:20.979556 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG uwsgi [-] security_compliance.lockout_failure_attempts = 2 {{(pid=80377) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 02 10:00:20.979918 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG uwsgi [-] security_compliance.minimum_password_age = 0 {{(pid=80377) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 02 10:00:20.980262 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG uwsgi [-] security_compliance.password_expires_days = None {{(pid=80377) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 02 10:00:20.980597 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG uwsgi [-] security_compliance.password_regex = None {{(pid=80377) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 02 10:00:20.980940 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG uwsgi [-] security_compliance.password_regex_description = None {{(pid=80377) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 02 10:00:20.981343 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG uwsgi [-] security_compliance.unique_last_password_count = 2 {{(pid=80377) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 02 10:00:20.981743 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG uwsgi [-] shadow_users.driver = sql {{(pid=80377) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 02 10:00:20.982148 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG uwsgi [-] token.allow_expired_window = 172800 {{(pid=80377) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 02 10:00:20.982492 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG uwsgi [-] token.allow_rescope_scoped_token = True {{(pid=80377) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 02 10:00:20.982840 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG uwsgi [-] token.cache_on_issue = True {{(pid=80377) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 02 10:00:20.983178 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG uwsgi [-] token.cache_time = None {{(pid=80377) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 02 10:00:20.983520 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG uwsgi [-] token.caching = True {{(pid=80377) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 02 10:00:20.983860 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG uwsgi [-] token.expiration = 3600 {{(pid=80377) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 02 10:00:20.984218 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG uwsgi [-] token.provider = fernet {{(pid=80377) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 02 10:00:20.984560 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG uwsgi [-] token.revoke_by_id = True {{(pid=80377) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 02 10:00:20.984919 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG uwsgi [-] tokenless_auth.issuer_attribute = SSL_CLIENT_I_DN {{(pid=80377) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 02 10:00:20.985276 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG uwsgi [-] tokenless_auth.protocol = x509 {{(pid=80377) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 02 10:00:20.985617 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG uwsgi [-] tokenless_auth.trusted_issuer = [] {{(pid=80377) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 02 10:00:20.986009 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG uwsgi [-] totp.included_previous_windows = 1 {{(pid=80377) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 02 10:00:20.986353 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG uwsgi [-] trust.allow_redelegation = False {{(pid=80377) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 02 10:00:20.987229 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG uwsgi [-] trust.driver = sql {{(pid=80377) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 02 10:00:20.987605 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG uwsgi [-] trust.max_redelegation_count = 3 {{(pid=80377) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 02 10:00:20.987989 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG uwsgi [-] unified_limit.cache_time = None {{(pid=80377) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 02 10:00:20.988333 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG uwsgi [-] unified_limit.caching = True {{(pid=80377) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 02 10:00:20.988684 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG uwsgi [-] unified_limit.driver = sql {{(pid=80377) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 02 10:00:20.989193 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG uwsgi [-] unified_limit.enforcement_model = flat {{(pid=80377) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 02 10:00:20.989701 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG uwsgi [-] unified_limit.list_limit = None {{(pid=80377) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 02 10:00:20.990073 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG uwsgi [-] wsgi.debug_middleware = False {{(pid=80377) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 02 10:00:20.990484 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG uwsgi [-] database.backend = sqlalchemy {{(pid=80377) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 02 10:00:20.990870 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG uwsgi [-] database.connection = **** {{(pid=80377) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 02 10:00:20.991255 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG uwsgi [-] database.connection_debug = 0 {{(pid=80377) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 02 10:00:20.992064 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG uwsgi [-] database.connection_parameters = {{(pid=80377) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 02 10:00:20.992476 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG uwsgi [-] database.connection_recycle_time = 3600 {{(pid=80377) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 02 10:00:20.992822 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG uwsgi [-] database.connection_trace = False {{(pid=80377) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 02 10:00:20.993669 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG uwsgi [-] database.db_inc_retry_interval = True {{(pid=80377) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 02 10:00:20.994101 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG uwsgi [-] database.db_max_retries = 20 {{(pid=80377) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 02 10:00:20.994479 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG uwsgi [-] database.db_max_retry_interval = 10 {{(pid=80377) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 02 10:00:20.994898 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG uwsgi [-] database.db_retry_interval = 1 {{(pid=80377) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 02 10:00:20.995287 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG uwsgi [-] database.max_overflow = 50 {{(pid=80377) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 02 10:00:20.995653 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG uwsgi [-] database.max_pool_size = 5 {{(pid=80377) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 02 10:00:20.996031 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG uwsgi [-] database.max_retries = 10 {{(pid=80377) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 02 10:00:20.996407 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG uwsgi [-] database.mysql_enable_ndb = False {{(pid=80377) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 02 10:00:20.996796 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG uwsgi [-] database.mysql_sql_mode = TRADITIONAL {{(pid=80377) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 02 10:00:20.997167 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG uwsgi [-] database.pool_timeout = None {{(pid=80377) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 02 10:00:20.997578 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG uwsgi [-] database.retry_interval = 10 {{(pid=80377) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 02 10:00:20.997979 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG uwsgi [-] database.slave_connection = **** {{(pid=80377) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 02 10:00:20.998374 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG uwsgi [-] database.sqlite_synchronous = True {{(pid=80377) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 02 10:00:20.998751 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG uwsgi [-] database.use_db_reconnect = False {{(pid=80377) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 02 10:00:20.999042 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG uwsgi [-] ******************************************************************************** {{(pid=80377) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2613}} Aug 02 10:00:21.031773 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.notifications [-] Callback: `keystone.application_credential.core.Manager._delete_app_creds_on_user_delete_callback` subscribed to event `identity.user.deleted`. {{(pid=80377) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 02 10:00:21.032211 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.notifications [-] Callback: `keystone.application_credential.core.Manager._delete_app_creds_on_user_delete_callback` subscribed to event `identity.user.disabled`. {{(pid=80377) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 02 10:00:21.032211 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.notifications [-] Callback: `keystone.application_credential.core.Manager._delete_app_creds_on_assignment_removal` subscribed to event `identity.remove_application_credentials_for_user.internal`. {{(pid=80377) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 02 10:00:21.042192 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.notifications [-] Callback: `keystone.assignment.core.Manager._delete_domain_assignments` subscribed to event `identity.domain.deleted`. {{(pid=80377) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 02 10:00:21.068177 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: /usr/lib/python3/dist-packages/secretstorage/dhcrypto.py:15: CryptographyDeprecationWarning: int_from_bytes is deprecated, use int.from_bytes instead Aug 02 10:00:21.068177 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: from cryptography.utils import int_from_bytes Aug 02 10:00:21.069006 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: /usr/lib/python3/dist-packages/secretstorage/util.py:19: CryptographyDeprecationWarning: int_from_bytes is deprecated, use int.from_bytes instead Aug 02 10:00:21.069006 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: from cryptography.utils import int_from_bytes Aug 02 10:00:21.075109 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.notifications [-] Callback: `keystone.catalog.core.Manager._on_project_or_endpoint_delete` subscribed to event `identity.project.deleted`. {{(pid=80377) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 02 10:00:21.075341 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.notifications [-] Callback: `keystone.catalog.core.Manager._on_project_or_endpoint_delete` subscribed to event `identity.endpoint.deleted`. {{(pid=80377) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 02 10:00:21.125769 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG uwsgi [-] ******************************************************************************** {{(pid=80378) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2589}} Aug 02 10:00:21.126206 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG uwsgi [-] Configuration options gathered from: {{(pid=80378) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2590}} Aug 02 10:00:21.126206 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG uwsgi [-] command line args: [] {{(pid=80378) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2591}} Aug 02 10:00:21.126646 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG uwsgi [-] config files: ['/etc/keystone/keystone.conf'] {{(pid=80378) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2592}} Aug 02 10:00:21.126856 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG uwsgi [-] ================================================================================ {{(pid=80378) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2594}} Aug 02 10:00:21.127099 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG uwsgi [-] admin_token = **** {{(pid=80378) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 02 10:00:21.128036 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG uwsgi [-] config_dir = [] {{(pid=80378) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 02 10:00:21.128328 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG uwsgi [-] config_file = ['/etc/keystone/keystone.conf'] {{(pid=80378) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 02 10:00:21.128654 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG uwsgi [-] config_source = [] {{(pid=80378) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 02 10:00:21.128992 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG uwsgi [-] debug = True {{(pid=80378) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 02 10:00:21.129361 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG uwsgi [-] default_log_levels = ['amqp=WARN', 'amqplib=WARN', 'boto=WARN', 'qpid=WARN', 'sqlalchemy=WARN', 'suds=INFO', 'oslo.messaging=INFO', 'oslo_messaging=INFO', 'iso8601=WARN', 'requests.packages.urllib3.connectionpool=WARN', 'urllib3.connectionpool=WARN', 'websocket=WARN', 'requests.packages.urllib3.util.retry=WARN', 'urllib3.util.retry=WARN', 'keystonemiddleware=WARN', 'routes.middleware=WARN', 'stevedore=WARN', 'taskflow=WARN', 'keystoneauth=WARN', 'oslo.cache=INFO', 'oslo_policy=INFO', 'dogpile.core.dogpile=INFO', 'dogpile=INFO', 'routes=INFO'] {{(pid=80378) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 02 10:00:21.129745 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG uwsgi [-] default_publisher_id = None {{(pid=80378) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 02 10:00:21.130072 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG uwsgi [-] insecure_debug = False {{(pid=80378) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 02 10:00:21.130486 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG uwsgi [-] instance_format = [instance: %(uuid)s] {{(pid=80378) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 02 10:00:21.130929 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG uwsgi [-] instance_uuid_format = [instance: %(uuid)s] {{(pid=80378) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 02 10:00:21.131218 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG uwsgi [-] list_limit = None {{(pid=80378) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 02 10:00:21.131391 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG uwsgi [-] log_config_append = None {{(pid=80378) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 02 10:00:21.131581 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG uwsgi [-] log_date_format = %Y-%m-%d %H:%M:%S {{(pid=80378) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 02 10:00:21.131766 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG uwsgi [-] log_dir = None {{(pid=80378) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 02 10:00:21.131947 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG uwsgi [-] log_file = None {{(pid=80378) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 02 10:00:21.132198 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG uwsgi [-] log_rotate_interval = 1 {{(pid=80378) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 02 10:00:21.132455 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG uwsgi [-] log_rotate_interval_type = days {{(pid=80378) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 02 10:00:21.132711 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG uwsgi [-] log_rotation_type = none {{(pid=80378) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 02 10:00:21.132991 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG uwsgi [-] logging_context_format_string = %(color)s%(levelname)s %(name)s [%(global_request_id)s %(request_id)s %(project_name)s %(user_name)s%(color)s] %(instance)s%(color)s%(message)s {{(pid=80378) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 02 10:00:21.133214 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG uwsgi [-] logging_debug_format_suffix = {{(pid=%(process)d) %(funcName)s %(pathname)s:%(lineno)d}} {{(pid=80378) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 02 10:00:21.133443 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG uwsgi [-] logging_default_format_string = %(color)s%(levelname)s %(name)s [-%(color)s] %(instance)s%(color)s%(message)s {{(pid=80378) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 02 10:00:21.133651 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG uwsgi [-] logging_exception_prefix = ERROR %(name)s %(instance)s {{(pid=80378) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 02 10:00:21.133885 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG uwsgi [-] logging_user_identity_format = %(user)s %(project)s %(domain)s %(system_scope)s %(user_domain)s %(project_domain)s {{(pid=80378) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 02 10:00:21.134125 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG uwsgi [-] max_logfile_count = 30 {{(pid=80378) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 02 10:00:21.134408 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG uwsgi [-] max_logfile_size_mb = 200 {{(pid=80378) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 02 10:00:21.134607 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG uwsgi [-] max_param_size = 64 {{(pid=80378) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 02 10:00:21.134875 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG uwsgi [-] max_project_tree_depth = 5 {{(pid=80378) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 02 10:00:21.135061 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG uwsgi [-] max_token_size = 16384 {{(pid=80378) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 02 10:00:21.135313 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG uwsgi [-] notification_format = cadf {{(pid=80378) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 02 10:00:21.135609 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG uwsgi [-] notification_opt_out = ['identity.authenticate.success', 'identity.authenticate.pending', 'identity.authenticate.failed'] {{(pid=80378) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 02 10:00:21.136137 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG uwsgi [-] public_endpoint = http://10.222.0.82/identity {{(pid=80378) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 02 10:00:21.136317 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG uwsgi [-] publish_errors = False {{(pid=80378) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 02 10:00:21.136602 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG uwsgi [-] pydev_debug_host = None {{(pid=80378) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 02 10:00:21.137002 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG uwsgi [-] pydev_debug_port = None {{(pid=80378) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 02 10:00:21.137230 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG uwsgi [-] rate_limit_burst = 0 {{(pid=80378) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 02 10:00:21.137498 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG uwsgi [-] rate_limit_except_level = CRITICAL {{(pid=80378) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 02 10:00:21.137742 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG uwsgi [-] rate_limit_interval = 0 {{(pid=80378) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 02 10:00:21.138004 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG uwsgi [-] standard_threads = False {{(pid=80378) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 02 10:00:21.138287 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG uwsgi [-] strict_password_check = False {{(pid=80378) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 02 10:00:21.138537 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG uwsgi [-] syslog_log_facility = LOG_USER {{(pid=80378) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 02 10:00:21.138716 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG uwsgi [-] use_eventlog = False {{(pid=80378) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 02 10:00:21.138895 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG uwsgi [-] use_journal = False {{(pid=80378) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 02 10:00:21.139100 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG uwsgi [-] use_json = False {{(pid=80378) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 02 10:00:21.139236 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG uwsgi [-] use_stderr = False {{(pid=80378) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 02 10:00:21.139401 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG uwsgi [-] use_syslog = False {{(pid=80378) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 02 10:00:21.139654 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG uwsgi [-] watch_log_file = False {{(pid=80378) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 02 10:00:21.140079 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG uwsgi [-] oslo_concurrency.disable_process_locking = False {{(pid=80378) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 02 10:00:21.140342 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG uwsgi [-] oslo_concurrency.lock_path = None {{(pid=80378) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 02 10:00:21.140621 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG uwsgi [-] profiler.connection_string = messaging:// {{(pid=80378) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 02 10:00:21.141060 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG uwsgi [-] profiler.enabled = False {{(pid=80378) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 02 10:00:21.141530 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG uwsgi [-] profiler.es_doc_type = notification {{(pid=80378) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 02 10:00:21.141941 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG uwsgi [-] profiler.es_scroll_size = 10000 {{(pid=80378) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 02 10:00:21.142214 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG uwsgi [-] profiler.es_scroll_time = 2m {{(pid=80378) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 02 10:00:21.142606 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG uwsgi [-] profiler.filter_error_trace = False {{(pid=80378) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 02 10:00:21.143539 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG uwsgi [-] profiler.hmac_keys = SECRET_KEY {{(pid=80378) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 02 10:00:21.143974 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG uwsgi [-] profiler.sentinel_service_name = mymaster {{(pid=80378) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 02 10:00:21.144433 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG uwsgi [-] profiler.socket_timeout = 0.1 {{(pid=80378) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 02 10:00:21.144801 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG uwsgi [-] profiler.trace_sqlalchemy = False {{(pid=80378) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 02 10:00:21.145080 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_buffer_size = 1000 {{(pid=80378) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 02 10:00:21.145332 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_enabled = False {{(pid=80378) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 02 10:00:21.145589 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_process_name = {{(pid=80378) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 02 10:00:21.145913 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_socket_file = /var/tmp/metrics_collector.sock {{(pid=80378) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 02 10:00:21.146303 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_thread_stop_timeout = 10 {{(pid=80378) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 02 10:00:21.146760 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG uwsgi [-] oslo_middleware.http_basic_auth_user_file = /etc/htpasswd {{(pid=80378) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 02 10:00:21.147157 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG uwsgi [-] audit.namespace = openstack {{(pid=80378) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 02 10:00:21.147611 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG uwsgi [-] oslo_policy.enforce_new_defaults = False {{(pid=80378) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 02 10:00:21.148065 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG uwsgi [-] oslo_policy.enforce_scope = False {{(pid=80378) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 02 10:00:21.148353 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.notifications [-] Callback: `keystone.federation.core.Manager._cleanup_identity_provider` subscribed to event `identity.domain_deleted.internal`. {{(pid=80377) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 02 10:00:21.148525 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG uwsgi [-] oslo_policy.policy_default_rule = default {{(pid=80378) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 02 10:00:21.148972 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG uwsgi [-] oslo_policy.policy_dirs = ['policy.d'] {{(pid=80378) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 02 10:00:21.149332 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG uwsgi [-] oslo_policy.policy_file = policy.yaml {{(pid=80378) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 02 10:00:21.149703 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG uwsgi [-] oslo_policy.remote_content_type = application/x-www-form-urlencoded {{(pid=80378) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 02 10:00:21.150040 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG uwsgi [-] oslo_policy.remote_ssl_ca_crt_file = None {{(pid=80378) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 02 10:00:21.150293 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG uwsgi [-] oslo_policy.remote_ssl_client_crt_file = None {{(pid=80378) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 02 10:00:21.150569 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG uwsgi [-] oslo_policy.remote_ssl_client_key_file = None {{(pid=80378) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 02 10:00:21.150955 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG uwsgi [-] oslo_policy.remote_ssl_verify_server_crt = False {{(pid=80378) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 02 10:00:21.151566 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG uwsgi [-] keystone_authtoken.admin_password = **** {{(pid=80378) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 02 10:00:21.152216 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG uwsgi [-] keystone_authtoken.admin_tenant_name = admin {{(pid=80378) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 02 10:00:21.152486 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG uwsgi [-] keystone_authtoken.admin_token = **** {{(pid=80378) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 02 10:00:21.152809 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG uwsgi [-] keystone_authtoken.admin_user = None {{(pid=80378) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 02 10:00:21.153126 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG uwsgi [-] keystone_authtoken.auth_admin_prefix = {{(pid=80378) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 02 10:00:21.153488 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG uwsgi [-] keystone_authtoken.auth_host = 127.0.0.1 {{(pid=80378) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 02 10:00:21.153960 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG uwsgi [-] keystone_authtoken.auth_port = 35357 {{(pid=80378) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 02 10:00:21.154397 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG uwsgi [-] keystone_authtoken.auth_protocol = https {{(pid=80378) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 02 10:00:21.154739 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG uwsgi [-] keystone_authtoken.auth_section = None {{(pid=80378) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 02 10:00:21.155137 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG uwsgi [-] keystone_authtoken.auth_type = None {{(pid=80378) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 02 10:00:21.155572 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG uwsgi [-] keystone_authtoken.auth_uri = None {{(pid=80378) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 02 10:00:21.155883 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG uwsgi [-] keystone_authtoken.auth_version = None {{(pid=80378) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 02 10:00:21.155883 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG uwsgi [-] keystone_authtoken.cache = None {{(pid=80378) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 02 10:00:21.156288 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG uwsgi [-] keystone_authtoken.cafile = None {{(pid=80378) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 02 10:00:21.156470 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG uwsgi [-] keystone_authtoken.certfile = None {{(pid=80378) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 02 10:00:21.156685 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG uwsgi [-] keystone_authtoken.delay_auth_decision = False {{(pid=80378) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 02 10:00:21.157140 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG uwsgi [-] keystone_authtoken.enforce_token_bind = permissive {{(pid=80378) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 02 10:00:21.157566 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG uwsgi [-] keystone_authtoken.http_connect_timeout = None {{(pid=80378) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 02 10:00:21.158051 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG uwsgi [-] keystone_authtoken.http_request_max_retries = 3 {{(pid=80378) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 02 10:00:21.158527 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG uwsgi [-] keystone_authtoken.identity_uri = None {{(pid=80378) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 02 10:00:21.159418 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG uwsgi [-] keystone_authtoken.include_service_catalog = True {{(pid=80378) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 02 10:00:21.159785 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG uwsgi [-] keystone_authtoken.insecure = False {{(pid=80378) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 02 10:00:21.160267 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG uwsgi [-] keystone_authtoken.interface = internal {{(pid=80378) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 02 10:00:21.160730 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG uwsgi [-] keystone_authtoken.keyfile = None {{(pid=80378) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 02 10:00:21.161190 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_conn_get_timeout = 10 {{(pid=80378) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 02 10:00:21.161598 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_dead_retry = 300 {{(pid=80378) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 02 10:00:21.162097 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_maxsize = 10 {{(pid=80378) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 02 10:00:21.162588 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_socket_timeout = 3 {{(pid=80378) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 02 10:00:21.162999 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_unused_timeout = 60 {{(pid=80378) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 02 10:00:21.163426 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG uwsgi [-] keystone_authtoken.memcache_secret_key = **** {{(pid=80378) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 02 10:00:21.163910 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG uwsgi [-] keystone_authtoken.memcache_security_strategy = None {{(pid=80378) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 02 10:00:21.164359 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG uwsgi [-] keystone_authtoken.memcache_use_advanced_pool = True {{(pid=80378) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 02 10:00:21.164809 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG uwsgi [-] keystone_authtoken.memcached_servers = None {{(pid=80378) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 02 10:00:21.165148 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG uwsgi [-] keystone_authtoken.region_name = None {{(pid=80378) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 02 10:00:21.165513 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG uwsgi [-] keystone_authtoken.service_token_roles = ['service'] {{(pid=80378) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 02 10:00:21.165910 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG uwsgi [-] keystone_authtoken.service_token_roles_required = False {{(pid=80378) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 02 10:00:21.166225 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG uwsgi [-] keystone_authtoken.service_type = None {{(pid=80378) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 02 10:00:21.166564 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG uwsgi [-] keystone_authtoken.token_cache_time = 300 {{(pid=80378) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 02 10:00:21.166854 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG uwsgi [-] keystone_authtoken.www_authenticate_uri = None {{(pid=80378) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 02 10:00:21.167219 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG uwsgi [-] cache.backend = dogpile.cache.memcached {{(pid=80378) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 02 10:00:21.167663 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG uwsgi [-] cache.backend_argument = **** {{(pid=80378) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 02 10:00:21.168131 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG uwsgi [-] cache.config_prefix = cache.oslo {{(pid=80378) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 02 10:00:21.168593 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG uwsgi [-] cache.dead_timeout = 60.0 {{(pid=80378) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 02 10:00:21.169054 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG uwsgi [-] cache.debug_cache_backend = False {{(pid=80378) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 02 10:00:21.169380 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG uwsgi [-] cache.enable_retry_client = False {{(pid=80378) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 02 10:00:21.169708 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG uwsgi [-] cache.enable_socket_keepalive = False {{(pid=80378) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 02 10:00:21.170037 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG uwsgi [-] cache.enabled = True {{(pid=80378) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 02 10:00:21.170401 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG uwsgi [-] cache.expiration_time = 600 {{(pid=80378) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 02 10:00:21.170733 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG uwsgi [-] cache.hashclient_retry_attempts = 2 {{(pid=80378) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 02 10:00:21.171121 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG uwsgi [-] cache.hashclient_retry_delay = 1.0 {{(pid=80378) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 02 10:00:21.171474 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG uwsgi [-] cache.memcache_dead_retry = 300 {{(pid=80378) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 02 10:00:21.171883 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG uwsgi [-] cache.memcache_pool_connection_get_timeout = 10 {{(pid=80378) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 02 10:00:21.172226 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG uwsgi [-] cache.memcache_pool_flush_on_reconnect = False {{(pid=80378) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 02 10:00:21.172627 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG uwsgi [-] cache.memcache_pool_maxsize = 10 {{(pid=80378) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 02 10:00:21.173007 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG uwsgi [-] cache.memcache_pool_unused_timeout = 60 {{(pid=80378) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 02 10:00:21.173661 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG uwsgi [-] cache.memcache_servers = ['localhost:11211'] {{(pid=80378) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 02 10:00:21.174056 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG uwsgi [-] cache.memcache_socket_timeout = 1.0 {{(pid=80378) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 02 10:00:21.174309 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG uwsgi [-] cache.proxies = [] {{(pid=80378) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 02 10:00:21.174740 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG uwsgi [-] cache.retry_attempts = 2 {{(pid=80378) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 02 10:00:21.175113 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG uwsgi [-] cache.retry_delay = 0.0 {{(pid=80378) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 02 10:00:21.175544 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG uwsgi [-] cache.socket_keepalive_count = 1 {{(pid=80378) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 02 10:00:21.175917 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG uwsgi [-] cache.socket_keepalive_idle = 1 {{(pid=80378) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 02 10:00:21.176296 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG uwsgi [-] cache.socket_keepalive_interval = 1 {{(pid=80378) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 02 10:00:21.176773 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG uwsgi [-] cache.tls_allowed_ciphers = None {{(pid=80378) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 02 10:00:21.177233 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG uwsgi [-] cache.tls_cafile = None {{(pid=80378) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 02 10:00:21.177670 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG uwsgi [-] cache.tls_certfile = None {{(pid=80378) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 02 10:00:21.178171 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG uwsgi [-] cache.tls_enabled = False {{(pid=80378) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 02 10:00:21.178611 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG uwsgi [-] cache.tls_keyfile = None {{(pid=80378) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 02 10:00:21.179085 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG uwsgi [-] application_credential.cache_time = None {{(pid=80378) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 02 10:00:21.179544 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG uwsgi [-] application_credential.caching = True {{(pid=80378) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 02 10:00:21.180017 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG uwsgi [-] application_credential.driver = sql {{(pid=80378) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 02 10:00:21.180408 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG uwsgi [-] application_credential.user_limit = -1 {{(pid=80378) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 02 10:00:21.180828 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG uwsgi [-] assignment.driver = sql {{(pid=80378) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 02 10:00:21.181280 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG uwsgi [-] assignment.prohibited_implied_role = ['admin'] {{(pid=80378) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 02 10:00:21.181670 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG uwsgi [-] auth.application_credential = None {{(pid=80378) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 02 10:00:21.182114 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG uwsgi [-] auth.external = None {{(pid=80378) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 02 10:00:21.182558 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG uwsgi [-] auth.mapped = None {{(pid=80378) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 02 10:00:21.183107 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG uwsgi [-] auth.methods = ['external', 'password', 'token', 'oauth1', 'mapped', 'application_credential'] {{(pid=80378) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 02 10:00:21.183541 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG uwsgi [-] auth.oauth1 = None {{(pid=80378) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 02 10:00:21.183999 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG uwsgi [-] auth.password = None {{(pid=80378) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 02 10:00:21.184439 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG uwsgi [-] auth.token = None {{(pid=80378) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 02 10:00:21.184878 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG uwsgi [-] catalog.cache_time = None {{(pid=80378) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 02 10:00:21.185323 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG uwsgi [-] catalog.caching = True {{(pid=80378) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 02 10:00:21.185777 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG uwsgi [-] catalog.driver = sql {{(pid=80378) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 02 10:00:21.186214 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG uwsgi [-] catalog.list_limit = None {{(pid=80378) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 02 10:00:21.186698 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG uwsgi [-] catalog.template_file = default_catalog.templates {{(pid=80378) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 02 10:00:21.187151 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG uwsgi [-] credential.auth_ttl = 15 {{(pid=80378) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 02 10:00:21.187407 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG uwsgi [-] credential.cache_time = None {{(pid=80378) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 02 10:00:21.187659 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG uwsgi [-] credential.caching = True {{(pid=80378) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 02 10:00:21.188162 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG uwsgi [-] credential.driver = sql {{(pid=80378) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 02 10:00:21.188558 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG uwsgi [-] credential.key_repository = /etc/keystone/credential-keys/ {{(pid=80378) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 02 10:00:21.188944 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG uwsgi [-] credential.provider = fernet {{(pid=80378) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 02 10:00:21.189383 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG uwsgi [-] credential.user_limit = -1 {{(pid=80378) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 02 10:00:21.189842 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG uwsgi [-] domain_config.cache_time = 300 {{(pid=80378) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 02 10:00:21.190182 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG uwsgi [-] domain_config.caching = True {{(pid=80378) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 02 10:00:21.190551 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG uwsgi [-] domain_config.driver = sql {{(pid=80378) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 02 10:00:21.191034 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG uwsgi [-] endpoint_filter.driver = sql {{(pid=80378) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 02 10:00:21.191418 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG uwsgi [-] endpoint_filter.return_all_endpoints_if_no_filter = True {{(pid=80378) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 02 10:00:21.191846 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG uwsgi [-] endpoint_policy.driver = sql {{(pid=80378) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 02 10:00:21.192308 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG uwsgi [-] eventlet_server.admin_bind_host = 0.0.0.0 {{(pid=80378) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 02 10:00:21.192733 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG uwsgi [-] eventlet_server.admin_port = 35357 {{(pid=80378) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 02 10:00:21.193253 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG uwsgi [-] eventlet_server.public_bind_host = 0.0.0.0 {{(pid=80378) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 02 10:00:21.193702 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG uwsgi [-] eventlet_server.public_port = 5000 {{(pid=80378) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 02 10:00:21.194215 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG uwsgi [-] federation.assertion_prefix = {{(pid=80378) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 02 10:00:21.194656 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG uwsgi [-] federation.caching = True {{(pid=80378) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 02 10:00:21.195027 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG uwsgi [-] federation.default_authorization_ttl = 0 {{(pid=80378) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 02 10:00:21.195382 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG uwsgi [-] federation.driver = sql {{(pid=80378) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 02 10:00:21.195845 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG uwsgi [-] federation.federated_domain_name = Federated {{(pid=80378) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 02 10:00:21.196234 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG uwsgi [-] federation.remote_id_attribute = None {{(pid=80378) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 02 10:00:21.196534 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG uwsgi [-] federation.sso_callback_template = /etc/keystone/sso_callback_template.html {{(pid=80378) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 02 10:00:21.196927 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG uwsgi [-] federation.trusted_dashboard = [] {{(pid=80378) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 02 10:00:21.197335 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG uwsgi [-] fernet_receipts.key_repository = /etc/keystone/fernet-keys/ {{(pid=80378) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 02 10:00:21.197732 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG uwsgi [-] fernet_receipts.max_active_keys = 3 {{(pid=80378) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 02 10:00:21.200425 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG uwsgi [-] fernet_tokens.key_repository = /etc/keystone/fernet-keys/ {{(pid=80378) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 02 10:00:21.200922 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG uwsgi [-] fernet_tokens.max_active_keys = 3 {{(pid=80378) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 02 10:00:21.201355 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG uwsgi [-] identity.cache_time = 600 {{(pid=80378) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 02 10:00:21.201732 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG uwsgi [-] identity.caching = True {{(pid=80378) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 02 10:00:21.202144 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG uwsgi [-] identity.default_domain_id = default {{(pid=80378) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 02 10:00:21.202571 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG uwsgi [-] identity.domain_config_dir = /etc/keystone/domains {{(pid=80378) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 02 10:00:21.202971 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG uwsgi [-] identity.domain_configurations_from_database = False {{(pid=80378) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 02 10:00:21.203390 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG uwsgi [-] identity.domain_specific_drivers_enabled = False {{(pid=80378) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 02 10:00:21.203821 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG uwsgi [-] identity.driver = sql {{(pid=80378) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 02 10:00:21.204210 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG uwsgi [-] identity.list_limit = None {{(pid=80378) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 02 10:00:21.204503 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG uwsgi [-] identity.max_password_length = 4096 {{(pid=80378) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 02 10:00:21.204928 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG uwsgi [-] identity.password_hash_algorithm = bcrypt {{(pid=80378) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 02 10:00:21.206139 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG uwsgi [-] identity.password_hash_rounds = 4 {{(pid=80378) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 02 10:00:21.206342 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG uwsgi [-] identity.salt_bytesize = None {{(pid=80378) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 02 10:00:21.206624 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG uwsgi [-] identity.scrypt_block_size = None {{(pid=80378) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 02 10:00:21.207045 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG uwsgi [-] identity.scrypt_parallelism = None {{(pid=80378) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 02 10:00:21.207444 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG uwsgi [-] identity_mapping.backward_compatible_ids = True {{(pid=80378) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 02 10:00:21.207860 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG uwsgi [-] identity_mapping.driver = sql {{(pid=80378) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 02 10:00:21.208264 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG uwsgi [-] identity_mapping.generator = sha256 {{(pid=80378) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 02 10:00:21.208739 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG uwsgi [-] jwt_tokens.jws_private_key_repository = /etc/keystone/jws-keys/private {{(pid=80378) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 02 10:00:21.209135 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG uwsgi [-] jwt_tokens.jws_public_key_repository = /etc/keystone/jws-keys/public {{(pid=80378) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 02 10:00:21.209427 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG uwsgi [-] ldap.alias_dereferencing = default {{(pid=80378) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 02 10:00:21.209860 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG uwsgi [-] ldap.auth_pool_connection_lifetime = 60 {{(pid=80378) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 02 10:00:21.210244 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG uwsgi [-] ldap.auth_pool_size = 100 {{(pid=80378) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 02 10:00:21.210626 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG uwsgi [-] ldap.chase_referrals = None {{(pid=80378) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 02 10:00:21.211114 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG uwsgi [-] ldap.connection_timeout = -1 {{(pid=80378) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 02 10:00:21.211527 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG uwsgi [-] ldap.debug_level = None {{(pid=80378) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 02 10:00:21.211905 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG uwsgi [-] ldap.group_ad_nesting = False {{(pid=80378) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 02 10:00:21.212172 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG uwsgi [-] ldap.group_additional_attribute_mapping = [] {{(pid=80378) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 02 10:00:21.212507 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG uwsgi [-] ldap.group_attribute_ignore = [] {{(pid=80378) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 02 10:00:21.212778 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG uwsgi [-] ldap.group_desc_attribute = description {{(pid=80378) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 02 10:00:21.213163 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG uwsgi [-] ldap.group_filter = None {{(pid=80378) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 02 10:00:21.213560 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG uwsgi [-] ldap.group_id_attribute = cn {{(pid=80378) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 02 10:00:21.213882 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG uwsgi [-] ldap.group_member_attribute = member {{(pid=80378) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 02 10:00:21.214220 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG uwsgi [-] ldap.group_members_are_ids = False {{(pid=80378) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 02 10:00:21.214619 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG uwsgi [-] ldap.group_name_attribute = ou {{(pid=80378) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 02 10:00:21.214987 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG uwsgi [-] ldap.group_objectclass = groupOfNames {{(pid=80378) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 02 10:00:21.215394 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG uwsgi [-] ldap.group_tree_dn = None {{(pid=80378) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 02 10:00:21.215808 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG uwsgi [-] ldap.page_size = 0 {{(pid=80378) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 02 10:00:21.216198 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG uwsgi [-] ldap.password = **** {{(pid=80378) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 02 10:00:21.216581 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG uwsgi [-] ldap.pool_connection_lifetime = 600 {{(pid=80378) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 02 10:00:21.217040 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG uwsgi [-] ldap.pool_connection_timeout = -1 {{(pid=80378) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 02 10:00:21.217507 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG uwsgi [-] ldap.pool_retry_delay = 0.1 {{(pid=80378) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 02 10:00:21.218008 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG uwsgi [-] ldap.pool_retry_max = 3 {{(pid=80378) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 02 10:00:21.218475 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG uwsgi [-] ldap.pool_size = 10 {{(pid=80378) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 02 10:00:21.218919 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG uwsgi [-] ldap.query_scope = one {{(pid=80378) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 02 10:00:21.219414 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG uwsgi [-] ldap.suffix = cn=example,cn=com {{(pid=80378) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 02 10:00:21.219866 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG uwsgi [-] ldap.tls_cacertdir = None {{(pid=80378) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 02 10:00:21.220323 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG uwsgi [-] ldap.tls_cacertfile = None {{(pid=80378) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 02 10:00:21.220817 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG uwsgi [-] ldap.tls_req_cert = demand {{(pid=80378) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 02 10:00:21.221901 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG uwsgi [-] ldap.url = ldap://localhost {{(pid=80378) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 02 10:00:21.222375 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG uwsgi [-] ldap.use_auth_pool = True {{(pid=80378) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 02 10:00:21.222834 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG uwsgi [-] ldap.use_pool = True {{(pid=80378) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 02 10:00:21.223291 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG uwsgi [-] ldap.use_tls = False {{(pid=80378) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 02 10:00:21.223746 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG uwsgi [-] ldap.user = None {{(pid=80378) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 02 10:00:21.224224 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG uwsgi [-] ldap.user_additional_attribute_mapping = [] {{(pid=80378) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 02 10:00:21.224370 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.notifications [-] Callback: `keystone.identity.core.Manager._domain_deleted` subscribed to event `identity.domain_deleted.internal`. {{(pid=80377) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 02 10:00:21.224627 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.notifications [-] Callback: `keystone.identity.core.Manager._unset_default_project` subscribed to event `identity.project.deleted`. {{(pid=80377) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 02 10:00:21.224808 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG uwsgi [-] ldap.user_attribute_ignore = ['default_project_id'] {{(pid=80378) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 02 10:00:21.225170 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG uwsgi [-] ldap.user_default_project_id_attribute = None {{(pid=80378) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 02 10:00:21.225643 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG uwsgi [-] ldap.user_description_attribute = description {{(pid=80378) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 02 10:00:21.226178 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG uwsgi [-] ldap.user_enabled_attribute = enabled {{(pid=80378) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 02 10:00:21.226664 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG uwsgi [-] ldap.user_enabled_default = True {{(pid=80378) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 02 10:00:21.227119 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG uwsgi [-] ldap.user_enabled_emulation = False {{(pid=80378) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 02 10:00:21.228144 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG uwsgi [-] ldap.user_enabled_emulation_dn = None {{(pid=80378) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 02 10:00:21.228607 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG uwsgi [-] ldap.user_enabled_emulation_use_group_config = False {{(pid=80378) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 02 10:00:21.229076 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG uwsgi [-] ldap.user_enabled_invert = False {{(pid=80378) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 02 10:00:21.229541 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG uwsgi [-] ldap.user_enabled_mask = 0 {{(pid=80378) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 02 10:00:21.230030 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG uwsgi [-] ldap.user_filter = None {{(pid=80378) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 02 10:00:21.230522 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG uwsgi [-] ldap.user_id_attribute = cn {{(pid=80378) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 02 10:00:21.231008 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG uwsgi [-] ldap.user_mail_attribute = mail {{(pid=80378) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 02 10:00:21.231485 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG uwsgi [-] ldap.user_name_attribute = sn {{(pid=80378) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 02 10:00:21.231965 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG uwsgi [-] ldap.user_objectclass = inetOrgPerson {{(pid=80378) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 02 10:00:21.232399 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG uwsgi [-] ldap.user_pass_attribute = userPassword {{(pid=80378) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 02 10:00:21.232861 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG uwsgi [-] ldap.user_tree_dn = None {{(pid=80378) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 02 10:00:21.233339 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG uwsgi [-] memcache.dead_retry = 300 {{(pid=80378) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 02 10:00:21.233812 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG uwsgi [-] memcache.pool_connection_get_timeout = 10 {{(pid=80378) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 02 10:00:21.234310 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG uwsgi [-] memcache.pool_maxsize = 10 {{(pid=80378) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 02 10:00:21.234771 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG uwsgi [-] memcache.pool_unused_timeout = 60 {{(pid=80378) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 02 10:00:21.235235 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG uwsgi [-] memcache.socket_timeout = 3 {{(pid=80378) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 02 10:00:21.235699 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG uwsgi [-] oauth1.access_token_duration = 86400 {{(pid=80378) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 02 10:00:21.236185 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG uwsgi [-] oauth1.driver = sql {{(pid=80378) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 02 10:00:21.237184 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG uwsgi [-] oauth1.request_token_duration = 28800 {{(pid=80378) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 02 10:00:21.237682 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG uwsgi [-] policy.driver = sql {{(pid=80378) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 02 10:00:21.238165 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG uwsgi [-] policy.list_limit = None {{(pid=80378) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 02 10:00:21.238646 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG uwsgi [-] receipt.cache_on_issue = True {{(pid=80378) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 02 10:00:21.239100 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG uwsgi [-] receipt.cache_time = 300 {{(pid=80378) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 02 10:00:21.239561 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG uwsgi [-] receipt.caching = True {{(pid=80378) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 02 10:00:21.240030 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG uwsgi [-] receipt.expiration = 300 {{(pid=80378) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 02 10:00:21.240518 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG uwsgi [-] receipt.provider = fernet {{(pid=80378) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 02 10:00:21.240991 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG uwsgi [-] resource.admin_project_domain_name = None {{(pid=80378) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 02 10:00:21.241473 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG uwsgi [-] resource.admin_project_name = None {{(pid=80378) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 02 10:00:21.241945 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG uwsgi [-] resource.cache_time = None {{(pid=80378) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 02 10:00:21.242420 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG uwsgi [-] resource.caching = True {{(pid=80378) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 02 10:00:21.242919 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG uwsgi [-] resource.domain_name_url_safe = off {{(pid=80378) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 02 10:00:21.243401 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG uwsgi [-] resource.driver = sql {{(pid=80378) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 02 10:00:21.243854 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG uwsgi [-] resource.list_limit = None {{(pid=80378) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 02 10:00:21.244351 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG uwsgi [-] resource.project_name_url_safe = off {{(pid=80378) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 02 10:00:21.244825 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG uwsgi [-] revoke.cache_time = 3600 {{(pid=80378) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 02 10:00:21.245080 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG uwsgi [-] revoke.caching = True {{(pid=80378) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 02 10:00:21.245348 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG uwsgi [-] revoke.driver = sql {{(pid=80378) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 02 10:00:21.245591 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG uwsgi [-] revoke.expiration_buffer = 1800 {{(pid=80378) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 02 10:00:21.245869 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG uwsgi [-] role.cache_time = None {{(pid=80378) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 02 10:00:21.246120 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG uwsgi [-] role.caching = True {{(pid=80378) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 02 10:00:21.246386 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG uwsgi [-] role.driver = sql {{(pid=80378) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 02 10:00:21.246612 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG uwsgi [-] role.list_limit = None {{(pid=80378) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 02 10:00:21.246876 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG uwsgi [-] saml.assertion_expiration_time = 3600 {{(pid=80378) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 02 10:00:21.247133 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG uwsgi [-] saml.certfile = /etc/keystone/ssl/certs/signing_cert.pem {{(pid=80378) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 02 10:00:21.247392 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG uwsgi [-] saml.idp_contact_company = Example, Inc. {{(pid=80378) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 02 10:00:21.247648 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG uwsgi [-] saml.idp_contact_email = support@example.com {{(pid=80378) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 02 10:00:21.247907 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG uwsgi [-] saml.idp_contact_name = SAML Identity Provider Support {{(pid=80378) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 02 10:00:21.248161 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG uwsgi [-] saml.idp_contact_surname = Support {{(pid=80378) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 02 10:00:21.248416 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG uwsgi [-] saml.idp_contact_telephone = +1 800 555 0100 {{(pid=80378) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 02 10:00:21.248674 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG uwsgi [-] saml.idp_contact_type = other {{(pid=80378) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 02 10:00:21.248916 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG uwsgi [-] saml.idp_entity_id = None {{(pid=80378) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 02 10:00:21.249171 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG uwsgi [-] saml.idp_lang = en {{(pid=80378) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 02 10:00:21.249423 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG uwsgi [-] saml.idp_metadata_path = /etc/keystone/saml2_idp_metadata.xml {{(pid=80378) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 02 10:00:21.249867 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG uwsgi [-] saml.idp_organization_display_name = OpenStack SAML Identity Provider {{(pid=80378) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 02 10:00:21.250333 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG uwsgi [-] saml.idp_organization_name = SAML Identity Provider {{(pid=80378) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 02 10:00:21.251118 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG uwsgi [-] saml.idp_organization_url = https://example.com/ {{(pid=80378) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 02 10:00:21.251555 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG uwsgi [-] saml.idp_sso_endpoint = None {{(pid=80378) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 02 10:00:21.252036 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG uwsgi [-] saml.keyfile = /etc/keystone/ssl/private/signing_key.pem {{(pid=80378) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 02 10:00:21.253032 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG uwsgi [-] saml.relay_state_prefix = ss:mem: {{(pid=80378) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 02 10:00:21.253504 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG uwsgi [-] saml.xmlsec1_binary = xmlsec1 {{(pid=80378) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 02 10:00:21.253937 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG uwsgi [-] security_compliance.change_password_upon_first_use = False {{(pid=80378) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 02 10:00:21.254318 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG uwsgi [-] security_compliance.disable_user_account_days_inactive = None {{(pid=80378) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 02 10:00:21.254519 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG uwsgi [-] security_compliance.lockout_duration = 10 {{(pid=80378) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 02 10:00:21.254919 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG uwsgi [-] security_compliance.lockout_failure_attempts = 2 {{(pid=80378) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 02 10:00:21.255450 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG uwsgi [-] security_compliance.minimum_password_age = 0 {{(pid=80378) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 02 10:00:21.255450 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG uwsgi [-] security_compliance.password_expires_days = None {{(pid=80378) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 02 10:00:21.255450 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG uwsgi [-] security_compliance.password_regex = None {{(pid=80378) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 02 10:00:21.256274 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG uwsgi [-] security_compliance.password_regex_description = None {{(pid=80378) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 02 10:00:21.256274 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG uwsgi [-] security_compliance.unique_last_password_count = 2 {{(pid=80378) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 02 10:00:21.256764 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG uwsgi [-] shadow_users.driver = sql {{(pid=80378) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 02 10:00:21.256764 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG uwsgi [-] token.allow_expired_window = 172800 {{(pid=80378) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 02 10:00:21.257103 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG uwsgi [-] token.allow_rescope_scoped_token = True {{(pid=80378) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 02 10:00:21.257296 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG uwsgi [-] token.cache_on_issue = True {{(pid=80378) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 02 10:00:21.257483 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG uwsgi [-] token.cache_time = None {{(pid=80378) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 02 10:00:21.257722 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG uwsgi [-] token.caching = True {{(pid=80378) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 02 10:00:21.258005 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG uwsgi [-] token.expiration = 3600 {{(pid=80378) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 02 10:00:21.258272 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG uwsgi [-] token.provider = fernet {{(pid=80378) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 02 10:00:21.258517 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG uwsgi [-] token.revoke_by_id = True {{(pid=80378) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 02 10:00:21.258785 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG uwsgi [-] tokenless_auth.issuer_attribute = SSL_CLIENT_I_DN {{(pid=80378) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 02 10:00:21.259040 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG uwsgi [-] tokenless_auth.protocol = x509 {{(pid=80378) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 02 10:00:21.259284 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG uwsgi [-] tokenless_auth.trusted_issuer = [] {{(pid=80378) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 02 10:00:21.259536 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG uwsgi [-] totp.included_previous_windows = 1 {{(pid=80378) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 02 10:00:21.259785 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG uwsgi [-] trust.allow_redelegation = False {{(pid=80378) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 02 10:00:21.260039 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG uwsgi [-] trust.driver = sql {{(pid=80378) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 02 10:00:21.260282 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG uwsgi [-] trust.max_redelegation_count = 3 {{(pid=80378) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 02 10:00:21.260954 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG uwsgi [-] unified_limit.cache_time = None {{(pid=80378) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 02 10:00:21.260954 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG uwsgi [-] unified_limit.caching = True {{(pid=80378) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 02 10:00:21.261353 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG uwsgi [-] unified_limit.driver = sql {{(pid=80378) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 02 10:00:21.261353 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG uwsgi [-] unified_limit.enforcement_model = flat {{(pid=80378) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 02 10:00:21.261741 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG uwsgi [-] unified_limit.list_limit = None {{(pid=80378) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 02 10:00:21.261741 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG uwsgi [-] wsgi.debug_middleware = False {{(pid=80378) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 02 10:00:21.262139 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG uwsgi [-] database.backend = sqlalchemy {{(pid=80378) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 02 10:00:21.262273 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG uwsgi [-] database.connection = **** {{(pid=80378) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 02 10:00:21.262524 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG uwsgi [-] database.connection_debug = 0 {{(pid=80378) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 02 10:00:21.262781 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG uwsgi [-] database.connection_parameters = {{(pid=80378) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 02 10:00:21.263026 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG uwsgi [-] database.connection_recycle_time = 3600 {{(pid=80378) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 02 10:00:21.263277 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG uwsgi [-] database.connection_trace = False {{(pid=80378) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 02 10:00:21.263535 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG uwsgi [-] database.db_inc_retry_interval = True {{(pid=80378) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 02 10:00:21.263770 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG uwsgi [-] database.db_max_retries = 20 {{(pid=80378) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 02 10:00:21.264012 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG uwsgi [-] database.db_max_retry_interval = 10 {{(pid=80378) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 02 10:00:21.264258 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG uwsgi [-] database.db_retry_interval = 1 {{(pid=80378) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 02 10:00:21.264510 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG uwsgi [-] database.max_overflow = 50 {{(pid=80378) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 02 10:00:21.264754 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG uwsgi [-] database.max_pool_size = 5 {{(pid=80378) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 02 10:00:21.265044 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG uwsgi [-] database.max_retries = 10 {{(pid=80378) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 02 10:00:21.265455 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG uwsgi [-] database.mysql_enable_ndb = False {{(pid=80378) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 02 10:00:21.265940 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG uwsgi [-] database.mysql_sql_mode = TRADITIONAL {{(pid=80378) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 02 10:00:21.266379 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG uwsgi [-] database.pool_timeout = None {{(pid=80378) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 02 10:00:21.266834 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG uwsgi [-] database.retry_interval = 10 {{(pid=80378) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 02 10:00:21.267255 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG uwsgi [-] database.slave_connection = **** {{(pid=80378) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 02 10:00:21.267651 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG uwsgi [-] database.sqlite_synchronous = True {{(pid=80378) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 02 10:00:21.268437 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG uwsgi [-] database.use_db_reconnect = False {{(pid=80378) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 02 10:00:21.268633 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG uwsgi [-] ******************************************************************************** {{(pid=80378) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2613}} Aug 02 10:00:21.317357 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._trust_callback` subscribed to event `identity.OS-TRUST:trust.deleted`. {{(pid=80377) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 02 10:00:21.317704 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._consumer_callback` subscribed to event `identity.OS-OAUTH1:consumer.deleted`. {{(pid=80377) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 02 10:00:21.318087 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._user_callback` subscribed to event `identity.user.deleted`. {{(pid=80377) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 02 10:00:21.318440 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._project_callback` subscribed to event `identity.project.deleted`. {{(pid=80377) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 02 10:00:21.318782 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._user_callback` subscribed to event `identity.user.disabled`. {{(pid=80377) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 02 10:00:21.319159 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._user_callback` subscribed to event `identity.persist_revocation_event_for_user.internal`. {{(pid=80377) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 02 10:00:21.347820 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.OS-TRUST:trust.deleted`. {{(pid=80377) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 02 10:00:21.348164 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.user.deleted`. {{(pid=80377) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 02 10:00:21.348492 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.domain.deleted`. {{(pid=80377) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 02 10:00:21.348808 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.user.disabled`. {{(pid=80377) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 02 10:00:21.349121 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.domain.disabled`. {{(pid=80377) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 02 10:00:21.349437 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.project.disabled`. {{(pid=80377) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 02 10:00:21.349753 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.invalidate_token_cache.internal`. {{(pid=80377) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 02 10:00:21.363098 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.notifications [-] Callback: `keystone.trust.core.Manager._on_user_delete` subscribed to event `identity.user.deleted`. {{(pid=80377) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 02 10:00:21.383303 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.OS-TRUST:trust.deleted`. {{(pid=80377) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 02 10:00:21.383661 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.user.deleted`. {{(pid=80377) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 02 10:00:21.383997 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.domain.deleted`. {{(pid=80377) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 02 10:00:21.384336 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.user.disabled`. {{(pid=80377) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 02 10:00:21.384676 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.domain.disabled`. {{(pid=80377) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 02 10:00:21.385006 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.project.disabled`. {{(pid=80377) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 02 10:00:21.385337 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.invalidate_token_cache.internal`. {{(pid=80377) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 02 10:00:21.407980 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/projects' {}] {{(pid=80377) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 02 10:00:21.408291 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.common [-] Adding additional resource route (alternate) to API auth: ['/OS-FEDERATION/projects' {}] {{(pid=80377) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:433}} Aug 02 10:00:21.409635 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/domains' {}] {{(pid=80377) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 02 10:00:21.409967 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.common [-] Adding additional resource route (alternate) to API auth: ['/OS-FEDERATION/domains' {}] {{(pid=80377) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:433}} Aug 02 10:00:21.410340 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/system' {}] {{(pid=80377) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 02 10:00:21.410695 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/catalog' {}] {{(pid=80377) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 02 10:00:21.411049 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/tokens/OS-PKI/revoked' {}] {{(pid=80377) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 02 10:00:21.411411 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/tokens' {}] {{(pid=80377) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 02 10:00:21.415782 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.notifications [-] Callback: `keystone.application_credential.core.Manager._delete_app_creds_on_user_delete_callback` subscribed to event `identity.user.deleted`. {{(pid=80378) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 02 10:00:21.416182 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.notifications [-] Callback: `keystone.application_credential.core.Manager._delete_app_creds_on_user_delete_callback` subscribed to event `identity.user.disabled`. {{(pid=80378) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 02 10:00:21.416594 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.notifications [-] Callback: `keystone.application_credential.core.Manager._delete_app_creds_on_assignment_removal` subscribed to event `identity.remove_application_credentials_for_user.internal`. {{(pid=80378) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 02 10:00:21.421873 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/saml2' {}] {{(pid=80377) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 02 10:00:21.422190 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/saml2/ecp' {}] {{(pid=80377) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 02 10:00:21.422466 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/websso/' {}] {{(pid=80377) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 02 10:00:21.422757 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/identity_providers//protocols//websso' {}] {{(pid=80377) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 02 10:00:21.430723 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.common [-] Adding standard routes to API credentials for `MethodViewType` (API Prefix: ) [/credentials, /credentials/] {{(pid=80377) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 02 10:00:21.435061 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.notifications [-] Callback: `keystone.assignment.core.Manager._delete_domain_assignments` subscribed to event `identity.domain.deleted`. {{(pid=80378) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 02 10:00:21.436240 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.common [-] Adding standard routes to API domains for `MethodViewType` (API Prefix: ) [/domains, /domains/] {{(pid=80377) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 02 10:00:21.436637 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//config' {}] {{(pid=80377) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 02 10:00:21.437042 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//config/' {}] {{(pid=80377) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 02 10:00:21.437421 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//config//' {}] {{(pid=80377) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 02 10:00:21.437842 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains/config/default' {}] {{(pid=80377) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 02 10:00:21.438184 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains/config//default' {}] {{(pid=80377) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 02 10:00:21.438553 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains/config///default' {}] {{(pid=80377) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 02 10:00:21.438926 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//users//roles' {}] {{(pid=80377) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 02 10:00:21.439291 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//users//roles/' {}] {{(pid=80377) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 02 10:00:21.439658 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//groups//roles' {}] {{(pid=80377) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 02 10:00:21.440041 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//groups//roles/' {}] {{(pid=80377) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 02 10:00:21.465750 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.common [-] Adding resource routes to API ec2tokens: ['/ec2tokens' {}] {{(pid=80377) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 02 10:00:21.466063 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.notifications [-] Callback: `keystone.catalog.core.Manager._on_project_or_endpoint_delete` subscribed to event `identity.project.deleted`. {{(pid=80378) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 02 10:00:21.466254 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.notifications [-] Callback: `keystone.catalog.core.Manager._on_project_or_endpoint_delete` subscribed to event `identity.endpoint.deleted`. {{(pid=80378) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 02 10:00:21.467758 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.common [-] Adding standard routes to API endpoints for `MethodViewType` (API Prefix: ) [/endpoints, /endpoints/] {{(pid=80377) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 02 10:00:21.468162 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.common [-] Adding resource routes to API endpoints: ['/endpoints//OS-ENDPOINT-POLICY/policy' {}] {{(pid=80377) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 02 10:00:21.475182 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.common [-] Adding standard routes to API groups for `MethodViewType` (API Prefix: ) [/groups, /groups/] {{(pid=80377) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 02 10:00:21.475506 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.common [-] Adding resource routes to API groups: ['/groups//users' {}] {{(pid=80377) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 02 10:00:21.475808 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.common [-] Adding resource routes to API groups: ['/groups//users/' {}] {{(pid=80377) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 02 10:00:21.487953 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.common [-] Adding standard routes to API limits for `MethodViewType` (API Prefix: ) [/limits, /limits/] {{(pid=80377) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 02 10:00:21.488262 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.common [-] Adding resource routes to API limits: ['/limits/model' {}] {{(pid=80377) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 02 10:00:21.495584 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.common [-] Adding standard routes to API OS-EP-FILTER for `MethodViewType` (API Prefix: /OS-EP-FILTER) [/endpoint_groups, /endpoint_groups/] {{(pid=80377) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 02 10:00:21.496127 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoints//projects' {}] {{(pid=80377) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 02 10:00:21.496996 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/projects//endpoints/' {}] {{(pid=80377) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 02 10:00:21.497499 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/projects//endpoints' {}] {{(pid=80377) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 02 10:00:21.497990 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/projects//endpoint_groups' {}] {{(pid=80377) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 02 10:00:21.498449 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoint_groups//endpoints' {}] {{(pid=80377) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 02 10:00:21.498890 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoint_groups//projects' {}] {{(pid=80377) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 02 10:00:21.499323 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoint_groups//projects/' {}] {{(pid=80377) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 02 10:00:21.518235 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.notifications [-] Callback: `keystone.federation.core.Manager._cleanup_identity_provider` subscribed to event `identity.domain_deleted.internal`. {{(pid=80378) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 02 10:00:21.524694 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-FEDERATION: ['/saml2/metadata' {}] {{(pid=80377) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 02 10:00:21.524999 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-FEDERATION: ['/identity_providers//protocols//auth' {}] {{(pid=80377) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 02 10:00:21.529526 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.common [-] Adding standard routes to API identity_providers for `MethodViewType` (API Prefix: /OS-FEDERATION) [/identity_providers, /identity_providers/] {{(pid=80377) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 02 10:00:21.536022 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.common [-] Adding resource routes to API protocols: ['/OS-FEDERATION/identity_providers//protocols/' {}] {{(pid=80377) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 02 10:00:21.536512 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.common [-] Adding resource routes to API protocols: ['/OS-FEDERATION/identity_providers//protocols' {}] {{(pid=80377) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 02 10:00:21.542018 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.common [-] Adding standard routes to API mappings for `MethodViewType` (API Prefix: /OS-FEDERATION) [/mappings, /mappings/] {{(pid=80377) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 02 10:00:21.545437 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.common [-] Adding standard routes to API service_providers for `MethodViewType` (API Prefix: /OS-FEDERATION) [/service_providers, /service_providers/] {{(pid=80377) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 02 10:00:21.548742 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//groups//roles//inherited_to_projects' {}] {{(pid=80377) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 02 10:00:21.549896 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//groups//roles/inherited_to_projects' {}] {{(pid=80377) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 02 10:00:21.550288 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//users//roles//inherited_to_projects' {}] {{(pid=80377) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 02 10:00:21.550667 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//users//roles/inherited_to_projects' {}] {{(pid=80377) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 02 10:00:21.551042 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/projects//users//roles//inherited_to_projects' {}] {{(pid=80377) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 02 10:00:21.551427 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/projects//groups//roles//inherited_to_projects' {}] {{(pid=80377) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 02 10:00:21.568691 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.common [-] Adding standard routes to API OS-OAUTH1 for `MethodViewType` (API Prefix: /OS-OAUTH1) [/consumers, /consumers/] {{(pid=80377) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 02 10:00:21.569109 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH1: ['/request_token' {}] {{(pid=80377) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 02 10:00:21.569544 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH1: ['/access_token' {}] {{(pid=80377) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 02 10:00:21.569905 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH1: ['/authorize/' {}] {{(pid=80377) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 02 10:00:21.575094 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.notifications [-] Callback: `keystone.identity.core.Manager._domain_deleted` subscribed to event `identity.domain_deleted.internal`. {{(pid=80378) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 02 10:00:21.575491 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.notifications [-] Callback: `keystone.identity.core.Manager._unset_default_project` subscribed to event `identity.project.deleted`. {{(pid=80378) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 02 10:00:21.577934 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.common [-] Adding resource routes to API events: ['/events' {}] {{(pid=80377) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 02 10:00:21.579756 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-SIMPLE-CERT: ['/OS-SIMPLE-CERT/ca' {}] {{(pid=80377) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 02 10:00:21.580048 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-SIMPLE-CERT: ['/OS-SIMPLE-CERT/certificates' {}] {{(pid=80377) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 02 10:00:21.585504 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.common [-] Adding standard routes to API policy for `MethodViewType` (API Prefix: ) [/policies, /policies/] {{(pid=80377) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 02 10:00:21.586042 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/endpoints' {}] {{(pid=80377) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 02 10:00:21.586414 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/endpoints/' {}] {{(pid=80377) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 02 10:00:21.586723 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/services/' {}] {{(pid=80377) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 02 10:00:21.587025 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/services//regions/' {}] {{(pid=80377) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 02 10:00:21.603568 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.common [-] Adding standard routes to API projects for `MethodViewType` (API Prefix: ) [/projects, /projects/] {{(pid=80377) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 02 10:00:21.603866 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//tags' {}] {{(pid=80377) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 02 10:00:21.604165 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//tags/' {}] {{(pid=80377) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 02 10:00:21.604454 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//users//roles/' {}] {{(pid=80377) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 02 10:00:21.604749 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//users//roles' {}] {{(pid=80377) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 02 10:00:21.605032 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//groups//roles/' {}] {{(pid=80377) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 02 10:00:21.605340 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//groups//roles' {}] {{(pid=80377) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 02 10:00:21.627098 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.common [-] Adding standard routes to API regions for `MethodViewType` (API Prefix: ) [/regions, /regions/] {{(pid=80377) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 02 10:00:21.631992 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.common [-] Adding standard routes to API registered_limit for `MethodViewType` (API Prefix: ) [/registered_limits, /registered_limits/] {{(pid=80377) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 02 10:00:21.637480 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.common [-] Adding resource routes to API role_assignments: ['/role_assignments' {}] {{(pid=80377) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 02 10:00:21.640523 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.common [-] Adding resource routes to API role_inferences: ['/role_inferences' {}] {{(pid=80377) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 02 10:00:21.642254 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.common [-] Adding standard routes to API roles for `MethodViewType` (API Prefix: ) [/roles, /roles/] {{(pid=80377) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 02 10:00:21.642555 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.common [-] Adding resource routes to API roles: ['/roles//implies' {}] {{(pid=80377) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 02 10:00:21.642876 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.common [-] Adding resource routes to API roles: ['/roles//implies/' {}] {{(pid=80377) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 02 10:00:21.653593 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.common [-] Adding resource routes to API s3tokens: ['/s3tokens' {}] {{(pid=80377) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 02 10:00:21.654051 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._trust_callback` subscribed to event `identity.OS-TRUST:trust.deleted`. {{(pid=80378) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 02 10:00:21.654479 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._consumer_callback` subscribed to event `identity.OS-OAUTH1:consumer.deleted`. {{(pid=80378) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 02 10:00:21.655150 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._user_callback` subscribed to event `identity.user.deleted`. {{(pid=80378) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 02 10:00:21.655546 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._project_callback` subscribed to event `identity.project.deleted`. {{(pid=80378) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 02 10:00:21.655666 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.common [-] Adding standard routes to API services for `MethodViewType` (API Prefix: ) [/services, /services/] {{(pid=80377) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 02 10:00:21.655949 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._user_callback` subscribed to event `identity.user.disabled`. {{(pid=80378) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 02 10:00:21.656338 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._user_callback` subscribed to event `identity.persist_revocation_event_for_user.internal`. {{(pid=80378) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 02 10:00:21.659491 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/users//roles' {}] {{(pid=80377) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 02 10:00:21.659819 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/users//roles/' {}] {{(pid=80377) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 02 10:00:21.660114 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/groups//roles' {}] {{(pid=80377) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 02 10:00:21.660399 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/groups//roles/' {}] {{(pid=80377) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 02 10:00:21.671146 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.common [-] Adding standard routes to API trusts for `MethodViewType` (API Prefix: /OS-TRUST) [/trusts, /trusts/] {{(pid=80377) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 02 10:00:21.671571 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.common [-] Adding resource routes to API trusts: ['/trusts//roles' {}] {{(pid=80377) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 02 10:00:21.671969 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.common [-] Adding resource routes to API trusts: ['/trusts//roles/' {}] {{(pid=80377) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 02 10:00:21.681993 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.OS-TRUST:trust.deleted`. {{(pid=80378) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 02 10:00:21.682249 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.common [-] Adding standard routes to API users for `MethodViewType` (API Prefix: ) [/users, /users/] {{(pid=80377) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 02 10:00:21.682457 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.user.deleted`. {{(pid=80378) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 02 10:00:21.682609 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//password' {}] {{(pid=80377) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 02 10:00:21.682852 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.domain.deleted`. {{(pid=80378) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 02 10:00:21.683095 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//groups' {}] {{(pid=80377) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 02 10:00:21.683244 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.user.disabled`. {{(pid=80378) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 02 10:00:21.683592 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//projects' {}] {{(pid=80377) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 02 10:00:21.683742 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.domain.disabled`. {{(pid=80378) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 02 10:00:21.683960 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.project.disabled`. {{(pid=80378) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 02 10:00:21.684117 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//credentials/OS-EC2' {}] {{(pid=80377) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 02 10:00:21.684329 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.invalidate_token_cache.internal`. {{(pid=80378) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 02 10:00:21.684543 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//credentials/OS-EC2/' {}] {{(pid=80377) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 02 10:00:21.684909 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens' {}] {{(pid=80377) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 02 10:00:21.685254 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens/' {}] {{(pid=80377) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 02 10:00:21.685727 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens//roles' {}] {{(pid=80377) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 02 10:00:21.686204 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens//roles/' {}] {{(pid=80377) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 02 10:00:21.686585 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//application_credentials' {}] {{(pid=80377) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 02 10:00:21.686927 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//application_credentials/' {}] {{(pid=80377) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 02 10:00:21.687286 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//access_rules' {}] {{(pid=80377) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 02 10:00:21.687622 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//access_rules/' {}] {{(pid=80377) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 02 10:00:21.700791 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.notifications [-] Callback: `keystone.trust.core.Manager._on_user_delete` subscribed to event `identity.user.deleted`. {{(pid=80378) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 02 10:00:21.707195 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.OS-TRUST:trust.deleted`. {{(pid=80378) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 02 10:00:21.707543 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.user.deleted`. {{(pid=80378) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 02 10:00:21.707872 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.domain.deleted`. {{(pid=80378) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 02 10:00:21.708227 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.user.disabled`. {{(pid=80378) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 02 10:00:21.708584 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.domain.disabled`. {{(pid=80378) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 02 10:00:21.708945 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.project.disabled`. {{(pid=80378) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 02 10:00:21.709295 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.invalidate_token_cache.internal`. {{(pid=80378) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 02 10:00:21.728648 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/projects' {}] {{(pid=80378) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 02 10:00:21.728984 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.common [-] Adding additional resource route (alternate) to API auth: ['/OS-FEDERATION/projects' {}] {{(pid=80378) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:433}} Aug 02 10:00:21.729564 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/domains' {}] {{(pid=80378) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 02 10:00:21.729951 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.common [-] Adding additional resource route (alternate) to API auth: ['/OS-FEDERATION/domains' {}] {{(pid=80378) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:433}} Aug 02 10:00:21.730401 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/system' {}] {{(pid=80378) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 02 10:00:21.730813 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/catalog' {}] {{(pid=80378) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 02 10:00:21.730963 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: WSGI app 0 (mountpoint='') ready in 8 seconds on interpreter 0x55bf6677a9c0 pid: 80377 (default app) Aug 02 10:00:21.731233 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/tokens/OS-PKI/revoked' {}] {{(pid=80378) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 02 10:00:21.731650 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/tokens' {}] {{(pid=80378) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 02 10:00:21.735228 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a057c7d6-e767-4d67-bcdb-3631ca97f983 None None] REQUEST_METHOD: `GET` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:00:21.735522 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a057c7d6-e767-4d67-bcdb-3631ca97f983 None None] SCRIPT_NAME: `/identity` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:00:21.735807 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a057c7d6-e767-4d67-bcdb-3631ca97f983 None None] PATH_INFO: `/v3` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:00:21.741024 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: [pid: 80377|app: 0|req: 1/1] 10.222.0.82 () {50 vars in 791 bytes} [Tue Aug 2 10:00:21 2022] GET /identity/v3/ => generated 255 bytes in 9 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 02 10:00:21.747755 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/saml2' {}] {{(pid=80378) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 02 10:00:21.748127 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/saml2/ecp' {}] {{(pid=80378) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 02 10:00:21.748489 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/websso/' {}] {{(pid=80378) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 02 10:00:21.748970 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/identity_providers//protocols//websso' {}] {{(pid=80378) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 02 10:00:21.763679 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.common [-] Adding standard routes to API credentials for `MethodViewType` (API Prefix: ) [/credentials, /credentials/] {{(pid=80378) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 02 10:00:21.769119 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.common [-] Adding standard routes to API domains for `MethodViewType` (API Prefix: ) [/domains, /domains/] {{(pid=80378) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 02 10:00:21.769552 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//config' {}] {{(pid=80378) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 02 10:00:21.769963 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//config/' {}] {{(pid=80378) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 02 10:00:21.770416 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//config//' {}] {{(pid=80378) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 02 10:00:21.770879 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains/config/default' {}] {{(pid=80378) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 02 10:00:21.771233 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains/config//default' {}] {{(pid=80378) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 02 10:00:21.771594 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains/config///default' {}] {{(pid=80378) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 02 10:00:21.772221 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//users//roles' {}] {{(pid=80378) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 02 10:00:21.772544 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//users//roles/' {}] {{(pid=80378) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 02 10:00:21.772839 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//groups//roles' {}] {{(pid=80378) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 02 10:00:21.773156 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//groups//roles/' {}] {{(pid=80378) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 02 10:00:21.812182 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.common [-] Adding resource routes to API ec2tokens: ['/ec2tokens' {}] {{(pid=80378) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 02 10:00:21.813846 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.common [-] Adding standard routes to API endpoints for `MethodViewType` (API Prefix: ) [/endpoints, /endpoints/] {{(pid=80378) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 02 10:00:21.814160 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.common [-] Adding resource routes to API endpoints: ['/endpoints//OS-ENDPOINT-POLICY/policy' {}] {{(pid=80378) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 02 10:00:21.820121 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.common [-] Adding standard routes to API groups for `MethodViewType` (API Prefix: ) [/groups, /groups/] {{(pid=80378) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 02 10:00:21.820501 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.common [-] Adding resource routes to API groups: ['/groups//users' {}] {{(pid=80378) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 02 10:00:21.820926 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.common [-] Adding resource routes to API groups: ['/groups//users/' {}] {{(pid=80378) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 02 10:00:21.828023 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.common [-] Adding standard routes to API limits for `MethodViewType` (API Prefix: ) [/limits, /limits/] {{(pid=80378) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 02 10:00:21.828313 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.common [-] Adding resource routes to API limits: ['/limits/model' {}] {{(pid=80378) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 02 10:00:21.833354 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.common [-] Adding standard routes to API OS-EP-FILTER for `MethodViewType` (API Prefix: /OS-EP-FILTER) [/endpoint_groups, /endpoint_groups/] {{(pid=80378) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 02 10:00:21.833663 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoints//projects' {}] {{(pid=80378) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 02 10:00:21.833977 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/projects//endpoints/' {}] {{(pid=80378) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 02 10:00:21.834380 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/projects//endpoints' {}] {{(pid=80378) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 02 10:00:21.834858 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/projects//endpoint_groups' {}] {{(pid=80378) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 02 10:00:21.835340 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoint_groups//endpoints' {}] {{(pid=80378) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 02 10:00:21.835841 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoint_groups//projects' {}] {{(pid=80378) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 02 10:00:21.836335 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoint_groups//projects/' {}] {{(pid=80378) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 02 10:00:21.858225 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-FEDERATION: ['/saml2/metadata' {}] {{(pid=80378) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 02 10:00:21.858528 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-FEDERATION: ['/identity_providers//protocols//auth' {}] {{(pid=80378) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 02 10:00:21.865227 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.common [-] Adding standard routes to API identity_providers for `MethodViewType` (API Prefix: /OS-FEDERATION) [/identity_providers, /identity_providers/] {{(pid=80378) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 02 10:00:21.873567 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.common [-] Adding resource routes to API protocols: ['/OS-FEDERATION/identity_providers//protocols/' {}] {{(pid=80378) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 02 10:00:21.873971 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.common [-] Adding resource routes to API protocols: ['/OS-FEDERATION/identity_providers//protocols' {}] {{(pid=80378) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 02 10:00:21.881514 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.common [-] Adding standard routes to API mappings for `MethodViewType` (API Prefix: /OS-FEDERATION) [/mappings, /mappings/] {{(pid=80378) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 02 10:00:21.889007 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.common [-] Adding standard routes to API service_providers for `MethodViewType` (API Prefix: /OS-FEDERATION) [/service_providers, /service_providers/] {{(pid=80378) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 02 10:00:21.895813 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//groups//roles//inherited_to_projects' {}] {{(pid=80378) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 02 10:00:21.896335 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//groups//roles/inherited_to_projects' {}] {{(pid=80378) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 02 10:00:21.896762 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//users//roles//inherited_to_projects' {}] {{(pid=80378) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 02 10:00:21.897192 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//users//roles/inherited_to_projects' {}] {{(pid=80378) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 02 10:00:21.897620 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/projects//users//roles//inherited_to_projects' {}] {{(pid=80378) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 02 10:00:21.898204 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/projects//groups//roles//inherited_to_projects' {}] {{(pid=80378) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 02 10:00:21.937532 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.common [-] Adding standard routes to API OS-OAUTH1 for `MethodViewType` (API Prefix: /OS-OAUTH1) [/consumers, /consumers/] {{(pid=80378) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 02 10:00:21.939588 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH1: ['/request_token' {}] {{(pid=80378) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 02 10:00:21.940125 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH1: ['/access_token' {}] {{(pid=80378) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 02 10:00:21.941133 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH1: ['/authorize/' {}] {{(pid=80378) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 02 10:00:21.955464 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.common [-] Adding resource routes to API events: ['/events' {}] {{(pid=80378) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 02 10:00:21.957994 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-SIMPLE-CERT: ['/OS-SIMPLE-CERT/ca' {}] {{(pid=80378) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 02 10:00:21.958404 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-SIMPLE-CERT: ['/OS-SIMPLE-CERT/certificates' {}] {{(pid=80378) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 02 10:00:21.963318 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.common [-] Adding standard routes to API policy for `MethodViewType` (API Prefix: ) [/policies, /policies/] {{(pid=80378) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 02 10:00:21.963818 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/endpoints' {}] {{(pid=80378) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 02 10:00:21.964240 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/endpoints/' {}] {{(pid=80378) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 02 10:00:21.964559 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/services/' {}] {{(pid=80378) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 02 10:00:21.964820 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/services//regions/' {}] {{(pid=80378) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 02 10:00:21.979169 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.common [-] Adding standard routes to API projects for `MethodViewType` (API Prefix: ) [/projects, /projects/] {{(pid=80378) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 02 10:00:21.979466 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//tags' {}] {{(pid=80378) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 02 10:00:21.979779 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//tags/' {}] {{(pid=80378) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 02 10:00:21.980040 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//users//roles/' {}] {{(pid=80378) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 02 10:00:21.980324 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//users//roles' {}] {{(pid=80378) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 02 10:00:21.980606 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//groups//roles/' {}] {{(pid=80378) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 02 10:00:21.980907 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//groups//roles' {}] {{(pid=80378) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 02 10:00:21.998019 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.common [-] Adding standard routes to API regions for `MethodViewType` (API Prefix: ) [/regions, /regions/] {{(pid=80378) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 02 10:00:22.001397 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.common [-] Adding standard routes to API registered_limit for `MethodViewType` (API Prefix: ) [/registered_limits, /registered_limits/] {{(pid=80378) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 02 10:00:22.005911 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.common [-] Adding resource routes to API role_assignments: ['/role_assignments' {}] {{(pid=80378) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 02 10:00:22.007542 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.common [-] Adding resource routes to API role_inferences: ['/role_inferences' {}] {{(pid=80378) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 02 10:00:22.009137 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.common [-] Adding standard routes to API roles for `MethodViewType` (API Prefix: ) [/roles, /roles/] {{(pid=80378) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 02 10:00:22.009500 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.common [-] Adding resource routes to API roles: ['/roles//implies' {}] {{(pid=80378) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 02 10:00:22.009919 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.common [-] Adding resource routes to API roles: ['/roles//implies/' {}] {{(pid=80378) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 02 10:00:22.017022 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.common [-] Adding resource routes to API s3tokens: ['/s3tokens' {}] {{(pid=80378) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 02 10:00:22.019563 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.common [-] Adding standard routes to API services for `MethodViewType` (API Prefix: ) [/services, /services/] {{(pid=80378) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 02 10:00:22.022928 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/users//roles' {}] {{(pid=80378) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 02 10:00:22.023266 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/users//roles/' {}] {{(pid=80378) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 02 10:00:22.023544 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/groups//roles' {}] {{(pid=80378) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 02 10:00:22.023825 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/groups//roles/' {}] {{(pid=80378) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 02 10:00:22.032460 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.common [-] Adding standard routes to API trusts for `MethodViewType` (API Prefix: /OS-TRUST) [/trusts, /trusts/] {{(pid=80378) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 02 10:00:22.032890 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.common [-] Adding resource routes to API trusts: ['/trusts//roles' {}] {{(pid=80378) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 02 10:00:22.033278 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.common [-] Adding resource routes to API trusts: ['/trusts//roles/' {}] {{(pid=80378) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 02 10:00:22.041640 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.common [-] Adding standard routes to API users for `MethodViewType` (API Prefix: ) [/users, /users/] {{(pid=80378) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 02 10:00:22.042083 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//password' {}] {{(pid=80378) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 02 10:00:22.042476 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//groups' {}] {{(pid=80378) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 02 10:00:22.042856 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//projects' {}] {{(pid=80378) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 02 10:00:22.043227 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//credentials/OS-EC2' {}] {{(pid=80378) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 02 10:00:22.043596 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//credentials/OS-EC2/' {}] {{(pid=80378) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 02 10:00:22.043966 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens' {}] {{(pid=80378) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 02 10:00:22.044355 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens/' {}] {{(pid=80378) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 02 10:00:22.044735 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens//roles' {}] {{(pid=80378) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 02 10:00:22.045120 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens//roles/' {}] {{(pid=80378) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 02 10:00:22.045497 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//application_credentials' {}] {{(pid=80378) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 02 10:00:22.045907 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//application_credentials/' {}] {{(pid=80378) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 02 10:00:22.046271 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//access_rules' {}] {{(pid=80378) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 02 10:00:22.046660 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//access_rules/' {}] {{(pid=80378) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 02 10:00:22.083765 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: WSGI app 0 (mountpoint='') ready in 9 seconds on interpreter 0x55bf6677a9c0 pid: 80378 (default app) Aug 02 10:00:30.893304 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-260f6129-edfe-4b57-be5f-af348b1dce81 None None] REQUEST_METHOD: `GET` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:00:30.893907 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-260f6129-edfe-4b57-be5f-af348b1dce81 None None] SCRIPT_NAME: `/identity` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:00:30.894334 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-260f6129-edfe-4b57-be5f-af348b1dce81 None None] PATH_INFO: `/` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:00:30.896831 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: [pid: 80377|app: 0|req: 2/2] 10.222.0.82 () {54 vars in 926 bytes} [Tue Aug 2 10:00:30 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 02 10:00:30.910893 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-057ef403-dcdb-4d4f-9241-d23ac122e29f None None] REQUEST_METHOD: `POST` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:00:30.911730 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-057ef403-dcdb-4d4f-9241-d23ac122e29f None None] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:00:30.911730 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-057ef403-dcdb-4d4f-9241-d23ac122e29f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:00:30.989566 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: INFO dbcounter [None req-057ef403-dcdb-4d4f-9241-d23ac122e29f None None] Registered counter for database keystone Aug 02 10:00:31.023594 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG oslo_db.sqlalchemy.engines [None req-057ef403-dcdb-4d4f-9241-d23ac122e29f None None] MySQL server mode set to STRICT_TRANS_TABLES,STRICT_ALL_TABLES,NO_ZERO_IN_DATE,NO_ZERO_DATE,ERROR_FOR_DIVISION_BY_ZERO,TRADITIONAL,NO_ENGINE_SUBSTITUTION {{(pid=80378) _check_effective_sql_mode /usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/engines.py:314}} Aug 02 10:00:31.032300 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG dbcounter [-] [80378] Writer thread running {{(pid=80378) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:99}} Aug 02 10:00:31.277339 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG passlib.handlers.bcrypt [None req-057ef403-dcdb-4d4f-9241-d23ac122e29f None None] detected 'bcrypt' backend, version '3.2.2' {{(pid=80378) _load_backend_mixin /usr/local/lib/python3.8/dist-packages/passlib/handlers/bcrypt.py:625}} Aug 02 10:00:31.278089 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG passlib.handlers.bcrypt [None req-057ef403-dcdb-4d4f-9241-d23ac122e29f None None] 'bcrypt' backend lacks $2$ support, enabling workaround {{(pid=80378) _finalize_backend_mixin /usr/local/lib/python3.8/dist-packages/passlib/handlers/bcrypt.py:406}} Aug 02 10:00:31.344156 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.auth.core [None req-057ef403-dcdb-4d4f-9241-d23ac122e29f None None] MFA Rules not processed for user `891005fa5ff74041b75747d16cc66798`. Rule list: `[]` (Enabled: `True`). {{(pid=80378) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 02 10:00:31.346377 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.fernet_utils [None req-057ef403-dcdb-4d4f-9241-d23ac122e29f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80378) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 10:00:31.550155 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 1/3] 10.222.0.82 () {58 vars in 1023 bytes} [Tue Aug 2 10:00:30 2022] POST /identity/v3/auth/tokens => generated 914 bytes in 647 msecs (HTTP/1.1 201) 6 headers in 384 bytes (1 switches on core 0) Aug 02 10:00:31.704190 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-260f6129-edfe-4b57-be5f-af348b1dce81 None None] Authenticating user token {{(pid=80377) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:00:31.788987 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: INFO dbcounter [None req-260f6129-edfe-4b57-be5f-af348b1dce81 None None] Registered counter for database keystone Aug 02 10:00:31.814283 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG oslo_db.sqlalchemy.engines [None req-260f6129-edfe-4b57-be5f-af348b1dce81 None None] MySQL server mode set to STRICT_TRANS_TABLES,STRICT_ALL_TABLES,NO_ZERO_IN_DATE,NO_ZERO_DATE,ERROR_FOR_DIVISION_BY_ZERO,TRADITIONAL,NO_ENGINE_SUBSTITUTION {{(pid=80377) _check_effective_sql_mode /usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/engines.py:314}} Aug 02 10:00:31.821376 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG dbcounter [-] [80377] Writer thread running {{(pid=80377) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:99}} Aug 02 10:00:31.982859 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-260f6129-edfe-4b57-be5f-af348b1dce81 None None] Validating token access rules against request {{(pid=80377) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:00:31.988822 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93295229-a409-4088-bc2e-eaa0d471d273 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '891005fa5ff74041b75747d16cc66798', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7047912c5acd4adeba5d87c3bf895e39', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80377) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:00:31.989489 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93295229-a409-4088-bc2e-eaa0d471d273 admin admin] REQUEST_METHOD: `GET` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:00:31.989775 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93295229-a409-4088-bc2e-eaa0d471d273 admin admin] SCRIPT_NAME: `/identity` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:00:31.990085 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93295229-a409-4088-bc2e-eaa0d471d273 admin admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:00:32.007769 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93295229-a409-4088-bc2e-eaa0d471d273 admin admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:00:32.072685 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: WARNING keystone.common.rbac_enforcer.enforcer [None req-93295229-a409-4088-bc2e-eaa0d471d273 admin admin] Deprecated policy rules found. Use oslopolicy-policy-generator and oslopolicy-policy-upgrade to detect and resolve deprecated policies in your configuration. Aug 02 10:00:32.073140 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93295229-a409-4088-bc2e-eaa0d471d273 admin admin] RBAC: Authorization granted {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:00:32.087579 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: [pid: 80377|app: 0|req: 3/4] 10.222.0.82 () {58 vars in 1324 bytes} [Tue Aug 2 10:00:31 2022] GET /identity/v3/auth/tokens => generated 914 bytes in 385 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Aug 02 10:00:32.094233 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-057ef403-dcdb-4d4f-9241-d23ac122e29f None None] Authenticating user token {{(pid=80378) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:00:32.115493 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-057ef403-dcdb-4d4f-9241-d23ac122e29f None None] Validating token access rules against request {{(pid=80378) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:00:32.120258 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75f89bb9-b74b-419c-a83e-0da7a4919b4a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '891005fa5ff74041b75747d16cc66798', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7047912c5acd4adeba5d87c3bf895e39', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80378) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:00:32.121632 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75f89bb9-b74b-419c-a83e-0da7a4919b4a admin admin] REQUEST_METHOD: `GET` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:00:32.121947 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75f89bb9-b74b-419c-a83e-0da7a4919b4a admin admin] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:00:32.122222 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75f89bb9-b74b-419c-a83e-0da7a4919b4a admin admin] PATH_INFO: `/v3/projects/7047912c5acd4adeba5d87c3bf895e39` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:00:32.123839 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75f89bb9-b74b-419c-a83e-0da7a4919b4a admin admin] RBAC: Authorizing `identity:get_project(project_id=7047912c5acd4adeba5d87c3bf895e39)` {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:00:32.182622 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: WARNING keystone.common.rbac_enforcer.enforcer [None req-75f89bb9-b74b-419c-a83e-0da7a4919b4a admin admin] Deprecated policy rules found. Use oslopolicy-policy-generator and oslopolicy-policy-upgrade to detect and resolve deprecated policies in your configuration. Aug 02 10:00:32.182956 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75f89bb9-b74b-419c-a83e-0da7a4919b4a admin admin] RBAC: Authorization granted {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:00:32.192299 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 2/5] 10.222.0.82 () {56 vars in 1207 bytes} [Tue Aug 2 10:00:32 2022] GET /identity/v3/projects/7047912c5acd4adeba5d87c3bf895e39 => generated 344 bytes in 99 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 02 10:00:34.637686 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2adb927-2c86-429c-a46f-fd1cdf4ee130 None None] REQUEST_METHOD: `GET` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:00:34.638572 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2adb927-2c86-429c-a46f-fd1cdf4ee130 None None] SCRIPT_NAME: `/identity` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:00:34.638572 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2adb927-2c86-429c-a46f-fd1cdf4ee130 None None] PATH_INFO: `/` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:00:34.642406 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: [pid: 80377|app: 0|req: 4/6] 10.222.0.82 () {54 vars in 926 bytes} [Tue Aug 2 10:00:34 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 02 10:00:34.650042 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d1407ca-b8ff-4eed-87c5-004388a7045c None None] REQUEST_METHOD: `POST` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:00:34.650561 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d1407ca-b8ff-4eed-87c5-004388a7045c None None] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:00:34.651043 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d1407ca-b8ff-4eed-87c5-004388a7045c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:00:34.682132 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.auth.core [None req-9d1407ca-b8ff-4eed-87c5-004388a7045c None None] MFA Rules not processed for user `891005fa5ff74041b75747d16cc66798`. Rule list: `[]` (Enabled: `True`). {{(pid=80378) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 02 10:00:34.683996 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.fernet_utils [None req-9d1407ca-b8ff-4eed-87c5-004388a7045c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80378) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 10:00:34.693809 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 3/7] 10.222.0.82 () {58 vars in 1023 bytes} [Tue Aug 2 10:00:34 2022] POST /identity/v3/auth/tokens => generated 914 bytes in 46 msecs (HTTP/1.1 201) 6 headers in 384 bytes (1 switches on core 0) Aug 02 10:00:34.721656 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2adb927-2c86-429c-a46f-fd1cdf4ee130 None None] Authenticating user token {{(pid=80377) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:00:34.736672 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2adb927-2c86-429c-a46f-fd1cdf4ee130 None None] Validating token access rules against request {{(pid=80377) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:00:34.741109 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1465c7ac-217b-4e6a-a2fc-5e9941bbc988 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '891005fa5ff74041b75747d16cc66798', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7047912c5acd4adeba5d87c3bf895e39', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80377) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:00:34.741749 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1465c7ac-217b-4e6a-a2fc-5e9941bbc988 admin admin] REQUEST_METHOD: `GET` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:00:34.742059 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1465c7ac-217b-4e6a-a2fc-5e9941bbc988 admin admin] SCRIPT_NAME: `/identity` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:00:34.742342 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1465c7ac-217b-4e6a-a2fc-5e9941bbc988 admin admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:00:34.753626 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1465c7ac-217b-4e6a-a2fc-5e9941bbc988 admin admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:00:34.756878 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1465c7ac-217b-4e6a-a2fc-5e9941bbc988 admin admin] RBAC: Authorization granted {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:00:34.770019 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: [pid: 80377|app: 0|req: 5/8] 10.222.0.82 () {58 vars in 1324 bytes} [Tue Aug 2 10:00:34 2022] GET /identity/v3/auth/tokens => generated 914 bytes in 49 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Aug 02 10:00:34.777493 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d1407ca-b8ff-4eed-87c5-004388a7045c None None] Authenticating user token {{(pid=80378) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:00:34.787986 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d1407ca-b8ff-4eed-87c5-004388a7045c None None] Validating token access rules against request {{(pid=80378) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:00:34.792911 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8166d4bd-f67a-42ff-b756-215f156116af admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '891005fa5ff74041b75747d16cc66798', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7047912c5acd4adeba5d87c3bf895e39', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80378) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:00:34.793577 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8166d4bd-f67a-42ff-b756-215f156116af admin admin] REQUEST_METHOD: `GET` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:00:34.793899 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8166d4bd-f67a-42ff-b756-215f156116af admin admin] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:00:34.794180 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8166d4bd-f67a-42ff-b756-215f156116af admin admin] PATH_INFO: `/v3/users/891005fa5ff74041b75747d16cc66798` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:00:34.795637 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8166d4bd-f67a-42ff-b756-215f156116af admin admin] RBAC: Authorizing `identity:get_user(user_id=891005fa5ff74041b75747d16cc66798)` {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:00:34.797528 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8166d4bd-f67a-42ff-b756-215f156116af admin admin] RBAC: Authorization granted {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:00:34.799269 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 4/9] 10.222.0.82 () {56 vars in 1198 bytes} [Tue Aug 2 10:00:34 2022] GET /identity/v3/users/891005fa5ff74041b75747d16cc66798 => generated 247 bytes in 22 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 02 10:00:37.476055 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c37e01be-1eff-42d1-ac4f-ded754d9c2d1 None None] REQUEST_METHOD: `GET` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:00:37.476931 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c37e01be-1eff-42d1-ac4f-ded754d9c2d1 None None] SCRIPT_NAME: `/identity` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:00:37.476931 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c37e01be-1eff-42d1-ac4f-ded754d9c2d1 None None] PATH_INFO: `/` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:00:37.477981 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: [pid: 80377|app: 0|req: 6/10] 10.222.0.82 () {54 vars in 926 bytes} [Tue Aug 2 10:00:37 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 02 10:00:37.484560 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6eabc768-b3ca-4cd3-923a-5658caca6c36 None None] REQUEST_METHOD: `POST` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:00:37.485229 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6eabc768-b3ca-4cd3-923a-5658caca6c36 None None] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:00:37.485450 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6eabc768-b3ca-4cd3-923a-5658caca6c36 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:00:37.506864 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.auth.core [None req-6eabc768-b3ca-4cd3-923a-5658caca6c36 None None] MFA Rules not processed for user `891005fa5ff74041b75747d16cc66798`. Rule list: `[]` (Enabled: `True`). {{(pid=80378) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 02 10:00:37.508728 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.fernet_utils [None req-6eabc768-b3ca-4cd3-923a-5658caca6c36 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80378) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 10:00:37.611314 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6369e814-8d3d-4c2b-980e-3ebb515c1aa1 None None] REQUEST_METHOD: `GET` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:00:37.611648 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6369e814-8d3d-4c2b-980e-3ebb515c1aa1 None None] SCRIPT_NAME: `/identity` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:00:37.612455 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6369e814-8d3d-4c2b-980e-3ebb515c1aa1 None None] PATH_INFO: `/` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:00:37.614295 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: [pid: 80377|app: 0|req: 7/11] 10.222.0.82 () {54 vars in 926 bytes} [Tue Aug 2 10:00:37 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 02 10:00:37.620435 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8248656c-e7da-45e5-af70-2750a47555a2 None None] REQUEST_METHOD: `POST` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:00:37.620719 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8248656c-e7da-45e5-af70-2750a47555a2 None None] SCRIPT_NAME: `/identity` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:00:37.620991 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8248656c-e7da-45e5-af70-2750a47555a2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:00:37.650377 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 5/12] 10.222.0.82 () {58 vars in 1023 bytes} [Tue Aug 2 10:00:37 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 168 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 02 10:00:37.686552 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6eabc768-b3ca-4cd3-923a-5658caca6c36 None None] Authenticating user token {{(pid=80378) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:00:37.697662 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG passlib.handlers.bcrypt [None req-8248656c-e7da-45e5-af70-2750a47555a2 None None] detected 'bcrypt' backend, version '3.2.2' {{(pid=80377) _load_backend_mixin /usr/local/lib/python3.8/dist-packages/passlib/handlers/bcrypt.py:625}} Aug 02 10:00:37.698016 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG passlib.handlers.bcrypt [None req-8248656c-e7da-45e5-af70-2750a47555a2 None None] 'bcrypt' backend lacks $2$ support, enabling workaround {{(pid=80377) _finalize_backend_mixin /usr/local/lib/python3.8/dist-packages/passlib/handlers/bcrypt.py:406}} Aug 02 10:00:37.715047 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG dbcounter [-] [80378] Writing DB stats keystone:SELECT=100 {{(pid=80378) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 10:00:37.748073 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.auth.core [None req-8248656c-e7da-45e5-af70-2750a47555a2 None None] MFA Rules not processed for user `891005fa5ff74041b75747d16cc66798`. Rule list: `[]` (Enabled: `True`). {{(pid=80377) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 02 10:00:37.749650 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.fernet_utils [None req-8248656c-e7da-45e5-af70-2750a47555a2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80377) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 10:00:37.855267 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6eabc768-b3ca-4cd3-923a-5658caca6c36 None None] Validating token access rules against request {{(pid=80378) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:00:37.885448 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: [pid: 80377|app: 0|req: 8/13] 10.222.0.82 () {58 vars in 1023 bytes} [Tue Aug 2 10:00:37 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 266 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Aug 02 10:00:37.901555 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96eca9d8-60a9-4f1b-8b86-150725d03aa8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '891005fa5ff74041b75747d16cc66798', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80378) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:00:37.902226 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96eca9d8-60a9-4f1b-8b86-150725d03aa8 None admin] REQUEST_METHOD: `GET` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:00:37.902512 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96eca9d8-60a9-4f1b-8b86-150725d03aa8 None admin] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:00:37.902788 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96eca9d8-60a9-4f1b-8b86-150725d03aa8 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:00:37.916419 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8248656c-e7da-45e5-af70-2750a47555a2 None None] Authenticating user token {{(pid=80377) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:00:37.994135 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96eca9d8-60a9-4f1b-8b86-150725d03aa8 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:00:37.996421 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96eca9d8-60a9-4f1b-8b86-150725d03aa8 None admin] RBAC: Authorization granted {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:00:38.038330 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG dbcounter [-] [80378] Writing DB stats keystone:SELECT=100 {{(pid=80378) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 10:00:38.083396 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8248656c-e7da-45e5-af70-2750a47555a2 None None] Validating token access rules against request {{(pid=80377) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:00:38.087449 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG dbcounter [-] [80377] Writing DB stats keystone:SELECT=100 {{(pid=80377) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 10:00:38.148467 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f70c191d-af11-4384-8ea9-c3c77180a490 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '891005fa5ff74041b75747d16cc66798', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80377) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:00:38.150344 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f70c191d-af11-4384-8ea9-c3c77180a490 None admin] REQUEST_METHOD: `GET` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:00:38.150813 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f70c191d-af11-4384-8ea9-c3c77180a490 None admin] SCRIPT_NAME: `/identity` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:00:38.151244 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f70c191d-af11-4384-8ea9-c3c77180a490 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:00:38.173468 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f70c191d-af11-4384-8ea9-c3c77180a490 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 02 10:00:38.181866 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 6/14] 10.222.0.82 () {58 vars in 1282 bytes} [Tue Aug 2 10:00:37 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 496 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 02 10:00:38.188179 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application [None req-f70c191d-af11-4384-8ea9-c3c77180a490 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 02 10:00:38.188179 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 02 10:00:38.188179 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 02 10:00:38.188179 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 02 10:00:38.188179 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 02 10:00:38.188179 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 02 10:00:38.188179 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 02 10:00:38.188179 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 02 10:00:38.188179 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 02 10:00:38.188179 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 02 10:00:38.188179 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 02 10:00:38.188179 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 02 10:00:38.188179 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 02 10:00:38.188179 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 02 10:00:38.188179 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 02 10:00:38.188179 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 02 10:00:38.190497 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 02 10:00:38.190497 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 02 10:00:38.190497 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 02 10:00:38.190497 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 02 10:00:38.190497 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 02 10:00:38.190497 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 02 10:00:38.190497 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 02 10:00:38.190497 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 02 10:00:38.190497 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 02 10:00:38.190497 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application return self.get_or_create( Aug 02 10:00:38.190497 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 02 10:00:38.190497 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application with Lock( Aug 02 10:00:38.190497 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 02 10:00:38.190497 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application return self._enter() Aug 02 10:00:38.190497 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 02 10:00:38.190497 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 02 10:00:38.204332 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 02 10:00:38.204332 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application return self.creator() Aug 02 10:00:38.204332 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 02 10:00:38.204332 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application created_value = creator( Aug 02 10:00:38.204332 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 02 10:00:38.204332 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 02 10:00:38.204332 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 02 10:00:38.204332 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 02 10:00:38.204332 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 02 10:00:38.204332 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 02 10:00:38.204332 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 02 10:00:38.204332 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application Aug 02 10:00:38.205805 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96eca9d8-60a9-4f1b-8b86-150725d03aa8 None admin] Authenticating user token {{(pid=80378) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:00:38.205896 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: [pid: 80377|app: 0|req: 9/15] 10.222.0.82 () {56 vars in 1096 bytes} [Tue Aug 2 10:00:37 2022] GET /identity/v3/roles/admin => generated 83 bytes in 275 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 02 10:00:38.209968 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f70c191d-af11-4384-8ea9-c3c77180a490 None admin] Authenticating user token {{(pid=80377) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:00:38.325140 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96eca9d8-60a9-4f1b-8b86-150725d03aa8 None admin] Validating token access rules against request {{(pid=80378) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:00:38.355430 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG dbcounter [-] [80378] Writing DB stats keystone:SELECT=100 {{(pid=80378) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 10:00:38.371262 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f70c191d-af11-4384-8ea9-c3c77180a490 None admin] Validating token access rules against request {{(pid=80377) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:00:38.374758 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-badaccf8-1bb9-4c42-947f-bf56278148cc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '891005fa5ff74041b75747d16cc66798', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80378) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:00:38.375958 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-badaccf8-1bb9-4c42-947f-bf56278148cc None admin] REQUEST_METHOD: `GET` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:00:38.376508 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-badaccf8-1bb9-4c42-947f-bf56278148cc None admin] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:00:38.379634 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-badaccf8-1bb9-4c42-947f-bf56278148cc None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:00:38.394342 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-badaccf8-1bb9-4c42-947f-bf56278148cc None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:00:38.395862 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-badaccf8-1bb9-4c42-947f-bf56278148cc None admin] RBAC: Authorization granted {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:00:38.416636 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application [None req-badaccf8-1bb9-4c42-947f-bf56278148cc None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 02 10:00:38.416636 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 02 10:00:38.416636 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 02 10:00:38.416636 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 02 10:00:38.416636 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 02 10:00:38.416636 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 02 10:00:38.416636 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 02 10:00:38.416636 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 02 10:00:38.416636 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 02 10:00:38.416636 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 02 10:00:38.416636 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 02 10:00:38.416636 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 02 10:00:38.416636 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 02 10:00:38.416636 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 02 10:00:38.416636 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 02 10:00:38.418000 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 02 10:00:38.418000 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 02 10:00:38.418000 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 02 10:00:38.418000 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 02 10:00:38.418000 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 02 10:00:38.418000 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 02 10:00:38.418000 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application return self.get_or_create( Aug 02 10:00:38.418000 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 02 10:00:38.418000 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application with Lock( Aug 02 10:00:38.418000 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 02 10:00:38.418000 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application return self._enter() Aug 02 10:00:38.418000 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 02 10:00:38.418000 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 02 10:00:38.418000 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 02 10:00:38.418000 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application return self.creator() Aug 02 10:00:38.418000 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 02 10:00:38.418000 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application created_value = creator( Aug 02 10:00:38.419360 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 02 10:00:38.419360 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 02 10:00:38.419360 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 02 10:00:38.419360 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application Aug 02 10:00:38.419360 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 7/16] 10.222.0.82 () {56 vars in 1108 bytes} [Tue Aug 2 10:00:38 2022] GET /identity/v3/domains/Default => generated 87 bytes in 230 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 02 10:00:38.424102 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd49edc0-ec41-4759-9c9a-4c05f9f98e62 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '891005fa5ff74041b75747d16cc66798', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80377) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:00:38.424759 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd49edc0-ec41-4759-9c9a-4c05f9f98e62 None admin] REQUEST_METHOD: `GET` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:00:38.425051 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd49edc0-ec41-4759-9c9a-4c05f9f98e62 None admin] SCRIPT_NAME: `/identity` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:00:38.425850 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd49edc0-ec41-4759-9c9a-4c05f9f98e62 None admin] PATH_INFO: `/v3/roles` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:00:38.426284 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd49edc0-ec41-4759-9c9a-4c05f9f98e62 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80377) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 02 10:00:38.426573 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd49edc0-ec41-4759-9c9a-4c05f9f98e62 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:00:38.427323 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-badaccf8-1bb9-4c42-947f-bf56278148cc None admin] Authenticating user token {{(pid=80378) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:00:38.428092 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd49edc0-ec41-4759-9c9a-4c05f9f98e62 None admin] RBAC: Authorization granted {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:00:38.443137 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: [pid: 80377|app: 0|req: 10/17] 10.222.0.82 () {56 vars in 1099 bytes} [Tue Aug 2 10:00:38 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 235 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 02 10:00:38.450201 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd49edc0-ec41-4759-9c9a-4c05f9f98e62 None admin] Authenticating user token {{(pid=80377) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:00:38.495340 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG dbcounter [-] [80377] Writing DB stats keystone:SELECT=100 {{(pid=80377) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 10:00:38.581667 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-badaccf8-1bb9-4c42-947f-bf56278148cc None admin] Validating token access rules against request {{(pid=80378) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:00:38.607743 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd49edc0-ec41-4759-9c9a-4c05f9f98e62 None admin] Validating token access rules against request {{(pid=80377) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:00:38.624066 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b2940ce-0782-44eb-8836-024cb10a67cb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '891005fa5ff74041b75747d16cc66798', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80378) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:00:38.624904 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b2940ce-0782-44eb-8836-024cb10a67cb None admin] REQUEST_METHOD: `GET` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:00:38.625229 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b2940ce-0782-44eb-8836-024cb10a67cb None admin] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:00:38.625632 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b2940ce-0782-44eb-8836-024cb10a67cb None admin] PATH_INFO: `/v3/domains` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:00:38.626618 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b2940ce-0782-44eb-8836-024cb10a67cb None admin] RBAC: Adding query filter params (name=Default) {{(pid=80378) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 02 10:00:38.627566 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b2940ce-0782-44eb-8836-024cb10a67cb None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:00:38.629299 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b2940ce-0782-44eb-8836-024cb10a67cb None admin] RBAC: Authorization granted {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:00:38.650363 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d2c8c63-caee-4a5f-9a80-f32e89760be8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '891005fa5ff74041b75747d16cc66798', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80377) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:00:38.650581 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 8/18] 10.222.0.82 () {56 vars in 1109 bytes} [Tue Aug 2 10:00:38 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 224 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 02 10:00:38.651729 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d2c8c63-caee-4a5f-9a80-f32e89760be8 None admin] REQUEST_METHOD: `GET` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:00:38.652019 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d2c8c63-caee-4a5f-9a80-f32e89760be8 None admin] SCRIPT_NAME: `/identity` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:00:38.652304 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d2c8c63-caee-4a5f-9a80-f32e89760be8 None admin] PATH_INFO: `/v3/users/891005fa5ff74041b75747d16cc66798` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:00:38.654052 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d2c8c63-caee-4a5f-9a80-f32e89760be8 None admin] RBAC: Authorizing `identity:get_user(user_id=891005fa5ff74041b75747d16cc66798)` {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:00:38.655842 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d2c8c63-caee-4a5f-9a80-f32e89760be8 None admin] RBAC: Authorization granted {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:00:38.657826 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: [pid: 80377|app: 0|req: 11/19] 10.222.0.82 () {56 vars in 1177 bytes} [Tue Aug 2 10:00:38 2022] GET /identity/v3/users/891005fa5ff74041b75747d16cc66798 => generated 247 bytes in 209 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 02 10:00:38.664593 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b2940ce-0782-44eb-8836-024cb10a67cb None admin] Authenticating user token {{(pid=80378) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:00:38.719759 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG dbcounter [-] [80378] Writing DB stats keystone:SELECT=100 {{(pid=80378) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 10:00:38.822908 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b2940ce-0782-44eb-8836-024cb10a67cb None admin] Validating token access rules against request {{(pid=80378) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:00:38.880322 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3e89ea4-aeb7-4a46-84e8-bcf3387f0d5b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '891005fa5ff74041b75747d16cc66798', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80378) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:00:38.880322 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3e89ea4-aeb7-4a46-84e8-bcf3387f0d5b None admin] REQUEST_METHOD: `GET` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:00:38.880322 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3e89ea4-aeb7-4a46-84e8-bcf3387f0d5b None admin] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:00:38.880671 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3e89ea4-aeb7-4a46-84e8-bcf3387f0d5b None admin] PATH_INFO: `/v3/domains/default` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:00:38.882746 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3e89ea4-aeb7-4a46-84e8-bcf3387f0d5b None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:00:38.885287 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3e89ea4-aeb7-4a46-84e8-bcf3387f0d5b None admin] RBAC: Authorization granted {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:00:38.887966 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 9/20] 10.222.0.82 () {56 vars in 1108 bytes} [Tue Aug 2 10:00:38 2022] GET /identity/v3/domains/default => generated 199 bytes in 224 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 02 10:00:38.893851 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d2c8c63-caee-4a5f-9a80-f32e89760be8 None admin] Authenticating user token {{(pid=80377) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:00:39.030522 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG dbcounter [-] [80377] Writing DB stats keystone:SELECT=100 {{(pid=80377) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 10:00:39.043038 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d2c8c63-caee-4a5f-9a80-f32e89760be8 None admin] Validating token access rules against request {{(pid=80377) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:00:39.102897 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2abeb9f7-ba94-4fd0-be0a-d6db64f08c2f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '891005fa5ff74041b75747d16cc66798', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80377) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:00:39.103477 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2abeb9f7-ba94-4fd0-be0a-d6db64f08c2f None admin] REQUEST_METHOD: `GET` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:00:39.103828 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2abeb9f7-ba94-4fd0-be0a-d6db64f08c2f None admin] SCRIPT_NAME: `/identity` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:00:39.104037 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2abeb9f7-ba94-4fd0-be0a-d6db64f08c2f None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:00:39.104860 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2abeb9f7-ba94-4fd0-be0a-d6db64f08c2f None admin] RBAC: Adding query filter params (role.id=237966442f05411191105bf336bc14f3, scope.domain.id=default, user.id=891005fa5ff74041b75747d16cc66798) {{(pid=80377) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 02 10:00:39.105172 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2abeb9f7-ba94-4fd0-be0a-d6db64f08c2f None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:00:39.106774 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2abeb9f7-ba94-4fd0-be0a-d6db64f08c2f None admin] RBAC: Authorization granted {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:00:39.118570 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: [pid: 80377|app: 0|req: 12/21] 10.222.0.82 () {56 vars in 1322 bytes} [Tue Aug 2 10:00:38 2022] GET /identity/v3/role_assignments?user.id=891005fa5ff74041b75747d16cc66798&scope.domain.id=default&role.id=237966442f05411191105bf336bc14f3 => generated 233 bytes in 225 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 02 10:00:41.761299 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9abc4230-c4c0-47d0-be46-39364312363f None None] REQUEST_METHOD: `GET` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:00:41.762388 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9abc4230-c4c0-47d0-be46-39364312363f None None] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:00:41.762891 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9abc4230-c4c0-47d0-be46-39364312363f None None] PATH_INFO: `/` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:00:41.764504 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 10/22] 10.222.0.82 () {54 vars in 926 bytes} [Tue Aug 2 10:00:41 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 02 10:00:41.772815 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bd1cdfc-6980-4e56-9c82-bce422074b74 None None] REQUEST_METHOD: `POST` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:00:41.773313 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bd1cdfc-6980-4e56-9c82-bce422074b74 None None] SCRIPT_NAME: `/identity` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:00:41.773697 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bd1cdfc-6980-4e56-9c82-bce422074b74 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:00:41.808914 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.auth.core [None req-5bd1cdfc-6980-4e56-9c82-bce422074b74 None None] MFA Rules not processed for user `891005fa5ff74041b75747d16cc66798`. Rule list: `[]` (Enabled: `True`). {{(pid=80377) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 02 10:00:41.810331 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.fernet_utils [None req-5bd1cdfc-6980-4e56-9c82-bce422074b74 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80377) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 10:00:41.976900 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: [pid: 80377|app: 0|req: 13/23] 10.222.0.82 () {58 vars in 1023 bytes} [Tue Aug 2 10:00:41 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 206 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 02 10:00:42.037555 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9abc4230-c4c0-47d0-be46-39364312363f None None] Authenticating user token {{(pid=80378) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:00:42.202375 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9abc4230-c4c0-47d0-be46-39364312363f None None] Validating token access rules against request {{(pid=80378) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:00:42.210798 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG dbcounter [-] [80378] Writing DB stats keystone:SELECT=100 {{(pid=80378) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 10:00:42.267217 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8081d627-bc2d-4fbd-95e2-81056d1751d8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '891005fa5ff74041b75747d16cc66798', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80378) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:00:42.268923 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8081d627-bc2d-4fbd-95e2-81056d1751d8 None admin] REQUEST_METHOD: `GET` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:00:42.269495 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8081d627-bc2d-4fbd-95e2-81056d1751d8 None admin] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:00:42.270176 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8081d627-bc2d-4fbd-95e2-81056d1751d8 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:00:42.300298 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8081d627-bc2d-4fbd-95e2-81056d1751d8 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:00:42.303434 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8081d627-bc2d-4fbd-95e2-81056d1751d8 None admin] RBAC: Authorization granted {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:00:42.334254 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application [None req-8081d627-bc2d-4fbd-95e2-81056d1751d8 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 02 10:00:42.334254 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 02 10:00:42.334254 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 02 10:00:42.334254 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 02 10:00:42.334254 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 02 10:00:42.334254 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 02 10:00:42.334254 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 02 10:00:42.334254 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 02 10:00:42.334254 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 02 10:00:42.334254 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 02 10:00:42.334254 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 02 10:00:42.334254 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 02 10:00:42.334254 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 02 10:00:42.334254 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 02 10:00:42.334254 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 02 10:00:42.335127 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 02 10:00:42.335127 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 02 10:00:42.335127 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 02 10:00:42.335127 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 02 10:00:42.335127 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 02 10:00:42.335127 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 02 10:00:42.335127 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application return self.get_or_create( Aug 02 10:00:42.335127 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 02 10:00:42.335127 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application with Lock( Aug 02 10:00:42.335127 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 02 10:00:42.335127 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application return self._enter() Aug 02 10:00:42.335127 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 02 10:00:42.335127 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 02 10:00:42.335127 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 02 10:00:42.335127 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application return self.creator() Aug 02 10:00:42.335127 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 02 10:00:42.335127 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application created_value = creator( Aug 02 10:00:42.335947 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 02 10:00:42.335947 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 02 10:00:42.335947 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 02 10:00:42.335947 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application Aug 02 10:00:42.338498 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 11/24] 10.222.0.82 () {56 vars in 1108 bytes} [Tue Aug 2 10:00:42 2022] GET /identity/v3/domains/Default => generated 87 bytes in 302 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 02 10:00:42.345401 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bd1cdfc-6980-4e56-9c82-bce422074b74 None None] Authenticating user token {{(pid=80377) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:00:42.536677 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG dbcounter [-] [80377] Writing DB stats keystone:SELECT=100 {{(pid=80377) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 10:00:42.564961 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bd1cdfc-6980-4e56-9c82-bce422074b74 None None] Validating token access rules against request {{(pid=80377) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:00:42.625067 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84cd8da2-4242-45e5-b8e1-ee654201409e None None] REQUEST_METHOD: `GET` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:00:42.625583 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84cd8da2-4242-45e5-b8e1-ee654201409e None None] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:00:42.626055 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84cd8da2-4242-45e5-b8e1-ee654201409e None None] PATH_INFO: `/` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:00:42.629176 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 12/25] 10.222.0.82 () {54 vars in 926 bytes} [Tue Aug 2 10:00:42 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 02 10:00:42.634394 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32d99c3e-24db-460c-8e02-c9de74cc906f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '891005fa5ff74041b75747d16cc66798', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80377) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:00:42.635468 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32d99c3e-24db-460c-8e02-c9de74cc906f None admin] REQUEST_METHOD: `GET` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:00:42.635980 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32d99c3e-24db-460c-8e02-c9de74cc906f None admin] SCRIPT_NAME: `/identity` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:00:42.636484 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32d99c3e-24db-460c-8e02-c9de74cc906f None admin] PATH_INFO: `/v3/domains` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:00:42.636609 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fddc21c-1501-41f3-a2fc-0b26439348d6 None None] REQUEST_METHOD: `POST` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:00:42.636799 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fddc21c-1501-41f3-a2fc-0b26439348d6 None None] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:00:42.637086 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fddc21c-1501-41f3-a2fc-0b26439348d6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:00:42.637772 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32d99c3e-24db-460c-8e02-c9de74cc906f None admin] RBAC: Adding query filter params (name=Default) {{(pid=80377) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 02 10:00:42.638200 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32d99c3e-24db-460c-8e02-c9de74cc906f None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:00:42.639762 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32d99c3e-24db-460c-8e02-c9de74cc906f None admin] RBAC: Authorization granted {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:00:42.686394 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.auth.core [None req-0fddc21c-1501-41f3-a2fc-0b26439348d6 None None] MFA Rules not processed for user `891005fa5ff74041b75747d16cc66798`. Rule list: `[]` (Enabled: `True`). {{(pid=80378) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 02 10:00:42.688482 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.fernet_utils [None req-0fddc21c-1501-41f3-a2fc-0b26439348d6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80378) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 10:00:42.697482 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: [pid: 80377|app: 0|req: 14/26] 10.222.0.82 () {56 vars in 1109 bytes} [Tue Aug 2 10:00:42 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 353 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 02 10:00:42.706898 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32d99c3e-24db-460c-8e02-c9de74cc906f None admin] Authenticating user token {{(pid=80377) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:00:42.899626 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32d99c3e-24db-460c-8e02-c9de74cc906f None admin] Validating token access rules against request {{(pid=80377) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:00:42.913958 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 13/27] 10.222.0.82 () {58 vars in 1023 bytes} [Tue Aug 2 10:00:42 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 279 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Aug 02 10:00:42.916357 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4e00ece-b246-498f-850c-06a56907c340 None None] REQUEST_METHOD: `GET` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:00:42.916901 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4e00ece-b246-498f-850c-06a56907c340 None None] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:00:42.917431 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4e00ece-b246-498f-850c-06a56907c340 None None] PATH_INFO: `/` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:00:42.920885 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 14/28] 10.222.0.82 () {54 vars in 926 bytes} [Tue Aug 2 10:00:42 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 02 10:00:42.927935 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-377c0e9d-8696-42a2-87cf-0ef4d9441b66 None None] REQUEST_METHOD: `GET` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:00:42.928685 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-377c0e9d-8696-42a2-87cf-0ef4d9441b66 None None] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:00:42.929347 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-377c0e9d-8696-42a2-87cf-0ef4d9441b66 None None] PATH_INFO: `/` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:00:42.931921 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 15/29] 10.222.0.82 () {54 vars in 926 bytes} [Tue Aug 2 10:00:42 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 02 10:00:42.935214 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdc57367-06f9-4472-b9c2-391d7aa79d11 None None] REQUEST_METHOD: `POST` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:00:42.935806 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdc57367-06f9-4472-b9c2-391d7aa79d11 None None] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:00:42.936298 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdc57367-06f9-4472-b9c2-391d7aa79d11 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:00:42.970192 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.auth.core [None req-bdc57367-06f9-4472-b9c2-391d7aa79d11 None None] MFA Rules not processed for user `891005fa5ff74041b75747d16cc66798`. Rule list: `[]` (Enabled: `True`). {{(pid=80378) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 02 10:00:42.974595 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.fernet_utils [None req-bdc57367-06f9-4472-b9c2-391d7aa79d11 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80378) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 10:00:42.985033 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b0d4240-b4ab-439c-820a-3e591bd59817 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '891005fa5ff74041b75747d16cc66798', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80377) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:00:42.986947 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b0d4240-b4ab-439c-820a-3e591bd59817 None admin] REQUEST_METHOD: `POST` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:00:42.987466 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b0d4240-b4ab-439c-820a-3e591bd59817 None admin] SCRIPT_NAME: `/identity` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:00:42.987890 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b0d4240-b4ab-439c-820a-3e591bd59817 None admin] PATH_INFO: `/v3/projects` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:00:42.988852 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b0d4240-b4ab-439c-820a-3e591bd59817 None admin] RBAC: Authorizing `identity:create_project()` {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:00:42.990655 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b0d4240-b4ab-439c-820a-3e591bd59817 None admin] RBAC: Authorization granted {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:00:43.119522 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 16/30] 10.222.0.82 () {58 vars in 1023 bytes} [Tue Aug 2 10:00:42 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 185 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 02 10:00:43.122065 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4a511d6-29d7-4be4-a1ea-ffb73171728e None None] REQUEST_METHOD: `POST` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:00:43.122916 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4a511d6-29d7-4be4-a1ea-ffb73171728e None None] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:00:43.123608 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4a511d6-29d7-4be4-a1ea-ffb73171728e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:00:43.160041 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.auth.core [None req-f4a511d6-29d7-4be4-a1ea-ffb73171728e None None] MFA Rules not processed for user `891005fa5ff74041b75747d16cc66798`. Rule list: `[]` (Enabled: `True`). {{(pid=80378) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 02 10:00:43.162691 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.fernet_utils [None req-f4a511d6-29d7-4be4-a1ea-ffb73171728e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80378) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 10:00:43.186043 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG dbcounter [-] [80378] Writing DB stats keystone:SELECT=100 {{(pid=80378) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 10:00:43.187617 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG dbcounter [-] [80377] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80377) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 10:00:43.325457 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 17/31] 10.222.0.82 () {58 vars in 1023 bytes} [Tue Aug 2 10:00:43 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 205 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 02 10:00:43.327173 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4a511d6-29d7-4be4-a1ea-ffb73171728e None None] Authenticating user token {{(pid=80378) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:00:43.444320 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: [pid: 80377|app: 0|req: 15/32] 10.222.0.82 () {60 vars in 1141 bytes} [Tue Aug 2 10:00:42 2022] POST /identity/v3/projects => generated 301 bytes in 738 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 02 10:00:43.446347 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b0d4240-b4ab-439c-820a-3e591bd59817 None admin] Authenticating user token {{(pid=80377) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:00:43.467641 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4a511d6-29d7-4be4-a1ea-ffb73171728e None None] Validating token access rules against request {{(pid=80378) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:00:43.518631 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0afbc64f-a408-4823-afcf-d53058b33c01 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '891005fa5ff74041b75747d16cc66798', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80378) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:00:43.524507 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0afbc64f-a408-4823-afcf-d53058b33c01 None admin] REQUEST_METHOD: `POST` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:00:43.526872 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0afbc64f-a408-4823-afcf-d53058b33c01 None admin] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:00:43.529084 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0afbc64f-a408-4823-afcf-d53058b33c01 None admin] PATH_INFO: `/v3/roles` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:00:43.530562 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0afbc64f-a408-4823-afcf-d53058b33c01 None admin] RBAC: Authorizing `identity:create_role()` {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:00:43.533992 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0afbc64f-a408-4823-afcf-d53058b33c01 None admin] RBAC: Authorization granted {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:00:43.650808 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b0d4240-b4ab-439c-820a-3e591bd59817 None admin] Validating token access rules against request {{(pid=80377) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:00:43.723999 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-232b01b1-1c8a-4bee-b246-d5b8591a0446 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '891005fa5ff74041b75747d16cc66798', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80377) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:00:43.725101 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-232b01b1-1c8a-4bee-b246-d5b8591a0446 None admin] REQUEST_METHOD: `POST` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:00:43.725615 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-232b01b1-1c8a-4bee-b246-d5b8591a0446 None admin] SCRIPT_NAME: `/identity` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:00:43.726142 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-232b01b1-1c8a-4bee-b246-d5b8591a0446 None admin] PATH_INFO: `/v3/roles` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:00:43.727373 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-232b01b1-1c8a-4bee-b246-d5b8591a0446 None admin] RBAC: Authorizing `identity:create_role()` {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:00:43.729314 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-232b01b1-1c8a-4bee-b246-d5b8591a0446 None admin] RBAC: Authorization granted {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:00:43.767884 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: [pid: 80377|app: 0|req: 16/33] 10.222.0.82 () {60 vars in 1131 bytes} [Tue Aug 2 10:00:43 2022] POST /identity/v3/roles => generated 219 bytes in 323 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 02 10:00:43.769223 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-232b01b1-1c8a-4bee-b246-d5b8591a0446 None admin] Authenticating user token {{(pid=80377) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:00:43.888592 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG dbcounter [-] [80377] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80377) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 10:00:43.934925 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 18/34] 10.222.0.82 () {60 vars in 1131 bytes} [Tue Aug 2 10:00:43 2022] POST /identity/v3/roles => generated 225 bytes in 608 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 02 10:00:43.938251 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc372b69-a8c4-4077-877f-f28e71c3c698 None None] REQUEST_METHOD: `GET` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:00:43.939731 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc372b69-a8c4-4077-877f-f28e71c3c698 None None] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:00:43.940361 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc372b69-a8c4-4077-877f-f28e71c3c698 None None] PATH_INFO: `/` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:00:43.943956 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 19/35] 10.222.0.82 () {54 vars in 926 bytes} [Tue Aug 2 10:00:43 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 02 10:00:43.951400 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42eb1454-0c1f-4942-9db3-2a629cda8ea5 None None] REQUEST_METHOD: `POST` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:00:43.952264 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42eb1454-0c1f-4942-9db3-2a629cda8ea5 None None] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:00:43.952794 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42eb1454-0c1f-4942-9db3-2a629cda8ea5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:00:43.982652 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG dbcounter [-] [80378] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80378) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 10:00:43.986309 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-232b01b1-1c8a-4bee-b246-d5b8591a0446 None admin] Validating token access rules against request {{(pid=80377) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:00:44.005973 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.auth.core [None req-42eb1454-0c1f-4942-9db3-2a629cda8ea5 None None] MFA Rules not processed for user `891005fa5ff74041b75747d16cc66798`. Rule list: `[]` (Enabled: `True`). {{(pid=80378) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 02 10:00:44.008740 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.fernet_utils [None req-42eb1454-0c1f-4942-9db3-2a629cda8ea5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80378) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 10:00:44.087477 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b743c92-ee0a-4594-b6d4-539821a61cd0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '891005fa5ff74041b75747d16cc66798', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80377) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:00:44.088442 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b743c92-ee0a-4594-b6d4-539821a61cd0 None admin] REQUEST_METHOD: `GET` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:00:44.088736 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b743c92-ee0a-4594-b6d4-539821a61cd0 None admin] SCRIPT_NAME: `/identity` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:00:44.089019 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b743c92-ee0a-4594-b6d4-539821a61cd0 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:00:44.105797 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b743c92-ee0a-4594-b6d4-539821a61cd0 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 02 10:00:44.113579 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application [None req-0b743c92-ee0a-4594-b6d4-539821a61cd0 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 02 10:00:44.113579 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 02 10:00:44.113579 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 02 10:00:44.113579 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 02 10:00:44.113579 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 02 10:00:44.113579 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 02 10:00:44.113579 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 02 10:00:44.113579 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 02 10:00:44.113579 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 02 10:00:44.113579 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 02 10:00:44.113579 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 02 10:00:44.113579 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 02 10:00:44.113579 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 02 10:00:44.113579 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 02 10:00:44.113579 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 02 10:00:44.113579 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 02 10:00:44.115497 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 02 10:00:44.115497 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 02 10:00:44.115497 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 02 10:00:44.115497 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 02 10:00:44.115497 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 02 10:00:44.115497 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 02 10:00:44.115497 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 02 10:00:44.115497 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 02 10:00:44.115497 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 02 10:00:44.115497 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application return self.get_or_create( Aug 02 10:00:44.115497 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 02 10:00:44.115497 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application with Lock( Aug 02 10:00:44.115497 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 02 10:00:44.115497 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application return self._enter() Aug 02 10:00:44.115497 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 02 10:00:44.115497 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 02 10:00:44.117175 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 02 10:00:44.117175 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application return self.creator() Aug 02 10:00:44.117175 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 02 10:00:44.117175 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application created_value = creator( Aug 02 10:00:44.117175 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 02 10:00:44.117175 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 02 10:00:44.117175 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 02 10:00:44.117175 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 02 10:00:44.117175 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 02 10:00:44.117175 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 02 10:00:44.117175 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 02 10:00:44.117175 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application Aug 02 10:00:44.123506 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: [pid: 80377|app: 0|req: 17/36] 10.222.0.82 () {56 vars in 1096 bytes} [Tue Aug 2 10:00:43 2022] GET /identity/v3/roles/admin => generated 83 bytes in 354 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 02 10:00:44.141994 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b743c92-ee0a-4594-b6d4-539821a61cd0 None admin] Authenticating user token {{(pid=80377) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:00:44.166573 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 20/37] 10.222.0.82 () {58 vars in 1023 bytes} [Tue Aug 2 10:00:43 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 216 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Aug 02 10:00:44.203467 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42eb1454-0c1f-4942-9db3-2a629cda8ea5 None None] Authenticating user token {{(pid=80378) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:00:44.355898 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b743c92-ee0a-4594-b6d4-539821a61cd0 None admin] Validating token access rules against request {{(pid=80377) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:00:44.381922 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42eb1454-0c1f-4942-9db3-2a629cda8ea5 None None] Validating token access rules against request {{(pid=80378) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:00:44.385085 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG dbcounter [-] [80377] Writing DB stats keystone:SELECT=100 {{(pid=80377) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 10:00:44.406356 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c152184-4d1b-48bc-bd1a-d0caa4816f3c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '891005fa5ff74041b75747d16cc66798', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80377) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:00:44.407000 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c152184-4d1b-48bc-bd1a-d0caa4816f3c None admin] REQUEST_METHOD: `GET` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:00:44.407395 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c152184-4d1b-48bc-bd1a-d0caa4816f3c None admin] SCRIPT_NAME: `/identity` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:00:44.407662 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c152184-4d1b-48bc-bd1a-d0caa4816f3c None admin] PATH_INFO: `/v3/roles` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:00:44.409007 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c152184-4d1b-48bc-bd1a-d0caa4816f3c None admin] RBAC: Adding query filter params (name=admin) {{(pid=80377) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 02 10:00:44.409326 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c152184-4d1b-48bc-bd1a-d0caa4816f3c None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:00:44.410821 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c152184-4d1b-48bc-bd1a-d0caa4816f3c None admin] RBAC: Authorization granted {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:00:44.421378 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: [pid: 80377|app: 0|req: 18/38] 10.222.0.82 () {56 vars in 1099 bytes} [Tue Aug 2 10:00:44 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 280 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 02 10:00:44.423200 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b239221-1979-4a36-ae5a-c4d441da84fb None None] REQUEST_METHOD: `GET` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:00:44.423719 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b239221-1979-4a36-ae5a-c4d441da84fb None None] SCRIPT_NAME: `/identity` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:00:44.424220 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b239221-1979-4a36-ae5a-c4d441da84fb None None] PATH_INFO: `/` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:00:44.427147 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: [pid: 80377|app: 0|req: 19/39] 10.222.0.82 () {54 vars in 926 bytes} [Tue Aug 2 10:00:44 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 02 10:00:44.429986 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b239221-1979-4a36-ae5a-c4d441da84fb None None] Authenticating user token {{(pid=80377) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:00:44.448596 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e6e3637-c9f3-40d1-a15e-07403bad4ce0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '891005fa5ff74041b75747d16cc66798', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80378) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:00:44.449659 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e6e3637-c9f3-40d1-a15e-07403bad4ce0 None admin] REQUEST_METHOD: `POST` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:00:44.450046 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e6e3637-c9f3-40d1-a15e-07403bad4ce0 None admin] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:00:44.450461 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e6e3637-c9f3-40d1-a15e-07403bad4ce0 None admin] PATH_INFO: `/v3/roles` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:00:44.451669 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e6e3637-c9f3-40d1-a15e-07403bad4ce0 None admin] RBAC: Authorizing `identity:create_role()` {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:00:44.453671 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e6e3637-c9f3-40d1-a15e-07403bad4ce0 None admin] RBAC: Authorization granted {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:00:44.458516 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG dbcounter [-] [80378] Writing DB stats keystone:SELECT=100 {{(pid=80378) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 10:00:44.478751 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 21/40] 10.222.0.82 () {60 vars in 1131 bytes} [Tue Aug 2 10:00:44 2022] POST /identity/v3/roles => generated 223 bytes in 276 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 02 10:00:44.479927 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19692723-8cf8-4162-89b9-25c35534a52d None None] REQUEST_METHOD: `POST` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:00:44.479927 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19692723-8cf8-4162-89b9-25c35534a52d None None] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:00:44.480199 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19692723-8cf8-4162-89b9-25c35534a52d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:00:44.503471 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.auth.core [None req-19692723-8cf8-4162-89b9-25c35534a52d None None] MFA Rules not processed for user `891005fa5ff74041b75747d16cc66798`. Rule list: `[]` (Enabled: `True`). {{(pid=80378) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 02 10:00:44.505086 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.fernet_utils [None req-19692723-8cf8-4162-89b9-25c35534a52d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80378) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 10:00:44.624119 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b239221-1979-4a36-ae5a-c4d441da84fb None None] Validating token access rules against request {{(pid=80377) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:00:44.630220 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 22/41] 10.222.0.82 () {58 vars in 1023 bytes} [Tue Aug 2 10:00:44 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 151 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 02 10:00:44.651598 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19692723-8cf8-4162-89b9-25c35534a52d None None] Authenticating user token {{(pid=80378) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:00:44.692153 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51ad2ef7-92eb-4800-a251-8a115c8046fb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '891005fa5ff74041b75747d16cc66798', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80377) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:00:44.693156 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51ad2ef7-92eb-4800-a251-8a115c8046fb None admin] REQUEST_METHOD: `GET` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:00:44.693609 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51ad2ef7-92eb-4800-a251-8a115c8046fb None admin] SCRIPT_NAME: `/identity` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:00:44.694031 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51ad2ef7-92eb-4800-a251-8a115c8046fb None admin] PATH_INFO: `/v3/users/891005fa5ff74041b75747d16cc66798` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:00:44.695399 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51ad2ef7-92eb-4800-a251-8a115c8046fb None admin] RBAC: Authorizing `identity:get_user(user_id=891005fa5ff74041b75747d16cc66798)` {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:00:44.696921 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51ad2ef7-92eb-4800-a251-8a115c8046fb None admin] RBAC: Authorization granted {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:00:44.698665 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: [pid: 80377|app: 0|req: 20/42] 10.222.0.82 () {56 vars in 1177 bytes} [Tue Aug 2 10:00:44 2022] GET /identity/v3/users/891005fa5ff74041b75747d16cc66798 => generated 247 bytes in 270 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 02 10:00:44.705946 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51ad2ef7-92eb-4800-a251-8a115c8046fb None admin] Authenticating user token {{(pid=80377) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:00:44.817981 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG dbcounter [-] [80377] Writing DB stats keystone:SELECT=100 {{(pid=80377) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 10:00:44.847240 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19692723-8cf8-4162-89b9-25c35534a52d None None] Validating token access rules against request {{(pid=80378) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:00:44.887283 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG dbcounter [-] [80378] Writing DB stats keystone:INSERT=1,keystone:SELECT=99 {{(pid=80378) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 10:00:44.909518 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51ad2ef7-92eb-4800-a251-8a115c8046fb None admin] Validating token access rules against request {{(pid=80377) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:00:44.916633 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-adb88a67-e1b3-4cc9-aec9-d352ec4a8871 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '891005fa5ff74041b75747d16cc66798', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80378) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:00:44.917315 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adb88a67-e1b3-4cc9-aec9-d352ec4a8871 None admin] REQUEST_METHOD: `GET` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:00:44.917607 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adb88a67-e1b3-4cc9-aec9-d352ec4a8871 None admin] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:00:44.917924 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adb88a67-e1b3-4cc9-aec9-d352ec4a8871 None admin] PATH_INFO: `/v3/domains/default` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:00:44.919379 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-adb88a67-e1b3-4cc9-aec9-d352ec4a8871 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:00:44.921444 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-adb88a67-e1b3-4cc9-aec9-d352ec4a8871 None admin] RBAC: Authorization granted {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:00:44.925380 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 23/43] 10.222.0.82 () {56 vars in 1108 bytes} [Tue Aug 2 10:00:44 2022] GET /identity/v3/domains/default => generated 199 bytes in 276 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 02 10:00:44.933192 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-adb88a67-e1b3-4cc9-aec9-d352ec4a8871 None admin] Authenticating user token {{(pid=80378) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:00:44.957940 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8713962-84af-46a4-a64b-c5a912c817bd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '891005fa5ff74041b75747d16cc66798', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80377) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:00:44.958775 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8713962-84af-46a4-a64b-c5a912c817bd None admin] REQUEST_METHOD: `GET` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:00:44.959296 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8713962-84af-46a4-a64b-c5a912c817bd None admin] SCRIPT_NAME: `/identity` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:00:44.959815 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8713962-84af-46a4-a64b-c5a912c817bd None admin] PATH_INFO: `/v3/domains/default` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:00:44.961637 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8713962-84af-46a4-a64b-c5a912c817bd None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:00:44.963228 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8713962-84af-46a4-a64b-c5a912c817bd None admin] RBAC: Authorization granted {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:00:44.965073 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: [pid: 80377|app: 0|req: 21/44] 10.222.0.82 () {56 vars in 1108 bytes} [Tue Aug 2 10:00:44 2022] GET /identity/v3/domains/default => generated 199 bytes in 261 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 02 10:00:44.974582 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8713962-84af-46a4-a64b-c5a912c817bd None admin] Authenticating user token {{(pid=80377) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:00:45.077273 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-adb88a67-e1b3-4cc9-aec9-d352ec4a8871 None admin] Validating token access rules against request {{(pid=80378) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:00:45.116494 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8713962-84af-46a4-a64b-c5a912c817bd None admin] Validating token access rules against request {{(pid=80377) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:00:45.126981 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b806876-4ff7-4f74-964b-5716f85b17db None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '891005fa5ff74041b75747d16cc66798', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80378) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:00:45.127964 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b806876-4ff7-4f74-964b-5716f85b17db None admin] REQUEST_METHOD: `POST` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:00:45.128404 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b806876-4ff7-4f74-964b-5716f85b17db None admin] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:00:45.128866 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b806876-4ff7-4f74-964b-5716f85b17db None admin] PATH_INFO: `/v3/projects` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:00:45.130054 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b806876-4ff7-4f74-964b-5716f85b17db None admin] RBAC: Authorizing `identity:create_project()` {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:00:45.131779 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b806876-4ff7-4f74-964b-5716f85b17db None admin] RBAC: Authorization granted {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:00:45.140823 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG dbcounter [-] [80377] Writing DB stats keystone:SELECT=100 {{(pid=80377) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 10:00:45.164851 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 24/45] 10.222.0.82 () {60 vars in 1141 bytes} [Tue Aug 2 10:00:44 2022] POST /identity/v3/projects => generated 312 bytes in 233 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 02 10:00:45.181632 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c1944ae-bed7-4217-a70a-b11a2a172a73 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '891005fa5ff74041b75747d16cc66798', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80377) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:00:45.182635 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c1944ae-bed7-4217-a70a-b11a2a172a73 None admin] REQUEST_METHOD: `PUT` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:00:45.182992 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c1944ae-bed7-4217-a70a-b11a2a172a73 None admin] SCRIPT_NAME: `/identity` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:00:45.183435 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c1944ae-bed7-4217-a70a-b11a2a172a73 None admin] PATH_INFO: `/v3/domains/default/users/891005fa5ff74041b75747d16cc66798/roles/237966442f05411191105bf336bc14f3` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:00:45.186031 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c1944ae-bed7-4217-a70a-b11a2a172a73 None admin] RBAC: Authorizing `identity:create_grant(domain_id=default, user_id=891005fa5ff74041b75747d16cc66798, role_id=237966442f05411191105bf336bc14f3)` {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:00:45.187660 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c1944ae-bed7-4217-a70a-b11a2a172a73 None admin] RBAC: Authorization granted {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:00:45.212919 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: [pid: 80377|app: 0|req: 22/46] 10.222.0.82 () {58 vars in 1361 bytes} [Tue Aug 2 10:00:44 2022] PUT /identity/v3/domains/default/users/891005fa5ff74041b75747d16cc66798/roles/237966442f05411191105bf336bc14f3 => generated 0 bytes in 239 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 02 10:00:47.436988 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-645e0d86-dde4-4770-a507-87fffccd7c12 None None] REQUEST_METHOD: `GET` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:00:47.437704 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-645e0d86-dde4-4770-a507-87fffccd7c12 None None] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:00:47.437704 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-645e0d86-dde4-4770-a507-87fffccd7c12 None None] PATH_INFO: `/` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:00:47.439033 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 25/47] 10.222.0.82 () {54 vars in 926 bytes} [Tue Aug 2 10:00:47 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 02 10:00:47.448466 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ace963f5-0f9c-4ca6-93a4-b88c9dd6ac34 None None] REQUEST_METHOD: `POST` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:00:47.449140 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ace963f5-0f9c-4ca6-93a4-b88c9dd6ac34 None None] SCRIPT_NAME: `/identity` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:00:47.449417 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ace963f5-0f9c-4ca6-93a4-b88c9dd6ac34 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:00:47.475716 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.auth.core [None req-ace963f5-0f9c-4ca6-93a4-b88c9dd6ac34 None None] MFA Rules not processed for user `891005fa5ff74041b75747d16cc66798`. Rule list: `[]` (Enabled: `True`). {{(pid=80377) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 02 10:00:47.478474 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.fernet_utils [None req-ace963f5-0f9c-4ca6-93a4-b88c9dd6ac34 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80377) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 10:00:47.587510 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: [pid: 80377|app: 0|req: 23/48] 10.222.0.82 () {58 vars in 1023 bytes} [Tue Aug 2 10:00:47 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 141 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 02 10:00:47.614323 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-645e0d86-dde4-4770-a507-87fffccd7c12 None None] Authenticating user token {{(pid=80378) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:00:47.696077 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04d463d8-3b4b-4b36-ab71-f5e35b214f2b None None] REQUEST_METHOD: `GET` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:00:47.696453 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04d463d8-3b4b-4b36-ab71-f5e35b214f2b None None] SCRIPT_NAME: `/identity` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:00:47.696659 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04d463d8-3b4b-4b36-ab71-f5e35b214f2b None None] PATH_INFO: `/` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:00:47.698168 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: [pid: 80377|app: 0|req: 24/49] 10.222.0.82 () {54 vars in 926 bytes} [Tue Aug 2 10:00:47 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 02 10:00:47.698276 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG dbcounter [-] [80378] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80378) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 10:00:47.710743 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5328f6bc-091a-49a7-927a-88d2e97b477a None None] REQUEST_METHOD: `POST` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:00:47.710743 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5328f6bc-091a-49a7-927a-88d2e97b477a None None] SCRIPT_NAME: `/identity` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:00:47.710743 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5328f6bc-091a-49a7-927a-88d2e97b477a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:00:47.726595 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.auth.core [None req-5328f6bc-091a-49a7-927a-88d2e97b477a None None] MFA Rules not processed for user `891005fa5ff74041b75747d16cc66798`. Rule list: `[]` (Enabled: `True`). {{(pid=80377) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 02 10:00:47.727816 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.fernet_utils [None req-5328f6bc-091a-49a7-927a-88d2e97b477a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80377) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 10:00:47.777412 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-645e0d86-dde4-4770-a507-87fffccd7c12 None None] Validating token access rules against request {{(pid=80378) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:00:47.826643 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbd9c46c-1329-4fa2-9986-ba6377f1bbe4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '891005fa5ff74041b75747d16cc66798', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80378) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:00:47.827299 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbd9c46c-1329-4fa2-9986-ba6377f1bbe4 None admin] REQUEST_METHOD: `GET` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:00:47.827590 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbd9c46c-1329-4fa2-9986-ba6377f1bbe4 None admin] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:00:47.827865 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbd9c46c-1329-4fa2-9986-ba6377f1bbe4 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:00:47.833929 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: [pid: 80377|app: 0|req: 25/50] 10.222.0.82 () {58 vars in 1023 bytes} [Tue Aug 2 10:00:47 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 130 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Aug 02 10:00:47.843145 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fbd9c46c-1329-4fa2-9986-ba6377f1bbe4 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 02 10:00:47.850275 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application [None req-fbd9c46c-1329-4fa2-9986-ba6377f1bbe4 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 02 10:00:47.850275 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 02 10:00:47.850275 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 02 10:00:47.850275 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 02 10:00:47.850275 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 02 10:00:47.850275 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 02 10:00:47.850275 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 02 10:00:47.850275 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 02 10:00:47.850275 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 02 10:00:47.850275 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 02 10:00:47.850275 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 02 10:00:47.850275 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 02 10:00:47.850275 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 02 10:00:47.850275 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 02 10:00:47.850275 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 02 10:00:47.850275 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 02 10:00:47.851186 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 02 10:00:47.851186 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 02 10:00:47.851186 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 02 10:00:47.851186 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 02 10:00:47.851186 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 02 10:00:47.851186 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 02 10:00:47.851186 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 02 10:00:47.851186 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 02 10:00:47.851186 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 02 10:00:47.851186 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application return self.get_or_create( Aug 02 10:00:47.851186 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 02 10:00:47.851186 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application with Lock( Aug 02 10:00:47.851186 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 02 10:00:47.851186 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application return self._enter() Aug 02 10:00:47.851186 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 02 10:00:47.851186 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 02 10:00:47.853067 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 02 10:00:47.853067 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application return self.creator() Aug 02 10:00:47.853067 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 02 10:00:47.853067 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application created_value = creator( Aug 02 10:00:47.853067 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 02 10:00:47.853067 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 02 10:00:47.853067 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 02 10:00:47.853067 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 02 10:00:47.853067 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 02 10:00:47.853067 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 02 10:00:47.853067 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 02 10:00:47.853067 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application Aug 02 10:00:47.853067 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 26/51] 10.222.0.82 () {56 vars in 1096 bytes} [Tue Aug 2 10:00:47 2022] GET /identity/v3/roles/admin => generated 83 bytes in 238 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 02 10:00:47.857524 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5328f6bc-091a-49a7-927a-88d2e97b477a None None] Authenticating user token {{(pid=80377) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:00:47.858781 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbd9c46c-1329-4fa2-9986-ba6377f1bbe4 None admin] Authenticating user token {{(pid=80378) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:00:47.941221 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG dbcounter [-] [80377] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80377) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 10:00:48.013475 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbd9c46c-1329-4fa2-9986-ba6377f1bbe4 None admin] Validating token access rules against request {{(pid=80378) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:00:48.017973 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG dbcounter [-] [80378] Writing DB stats keystone:SELECT=100 {{(pid=80378) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 10:00:48.024440 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5328f6bc-091a-49a7-927a-88d2e97b477a None None] Validating token access rules against request {{(pid=80377) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:00:48.066130 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-035fa240-89f2-43dd-9ebf-58c0b633b728 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '891005fa5ff74041b75747d16cc66798', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80378) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:00:48.067220 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-035fa240-89f2-43dd-9ebf-58c0b633b728 None admin] REQUEST_METHOD: `GET` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:00:48.067723 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-035fa240-89f2-43dd-9ebf-58c0b633b728 None admin] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:00:48.068201 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-035fa240-89f2-43dd-9ebf-58c0b633b728 None admin] PATH_INFO: `/v3/roles` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:00:48.069541 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-035fa240-89f2-43dd-9ebf-58c0b633b728 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80378) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 02 10:00:48.070064 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-035fa240-89f2-43dd-9ebf-58c0b633b728 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:00:48.072567 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-035fa240-89f2-43dd-9ebf-58c0b633b728 None admin] RBAC: Authorization granted {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:00:48.080231 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-822e4f97-ddbb-4ff0-aaf7-6021b7bd7b70 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '891005fa5ff74041b75747d16cc66798', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80377) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:00:48.080914 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-822e4f97-ddbb-4ff0-aaf7-6021b7bd7b70 None admin] REQUEST_METHOD: `GET` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:00:48.081206 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-822e4f97-ddbb-4ff0-aaf7-6021b7bd7b70 None admin] SCRIPT_NAME: `/identity` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:00:48.081494 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-822e4f97-ddbb-4ff0-aaf7-6021b7bd7b70 None admin] PATH_INFO: `/v3/domains/default` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:00:48.082788 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-822e4f97-ddbb-4ff0-aaf7-6021b7bd7b70 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:00:48.084273 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-822e4f97-ddbb-4ff0-aaf7-6021b7bd7b70 None admin] RBAC: Authorization granted {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:00:48.086253 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: [pid: 80377|app: 0|req: 26/52] 10.222.0.82 () {56 vars in 1108 bytes} [Tue Aug 2 10:00:47 2022] GET /identity/v3/domains/default => generated 199 bytes in 229 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 02 10:00:48.091943 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 27/53] 10.222.0.82 () {56 vars in 1099 bytes} [Tue Aug 2 10:00:47 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 234 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 02 10:00:48.093302 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-822e4f97-ddbb-4ff0-aaf7-6021b7bd7b70 None admin] Authenticating user token {{(pid=80377) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:00:48.098304 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-035fa240-89f2-43dd-9ebf-58c0b633b728 None admin] Authenticating user token {{(pid=80378) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:00:48.245638 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-822e4f97-ddbb-4ff0-aaf7-6021b7bd7b70 None admin] Validating token access rules against request {{(pid=80377) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:00:48.249093 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-035fa240-89f2-43dd-9ebf-58c0b633b728 None admin] Validating token access rules against request {{(pid=80378) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:00:48.254882 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG dbcounter [-] [80377] Writing DB stats keystone:SELECT=100 {{(pid=80377) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 10:00:48.297600 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b244ee2-d215-4d34-a390-8d3331b1182e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '891005fa5ff74041b75747d16cc66798', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80377) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:00:48.298269 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b244ee2-d215-4d34-a390-8d3331b1182e None admin] REQUEST_METHOD: `POST` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:00:48.299117 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b244ee2-d215-4d34-a390-8d3331b1182e None admin] SCRIPT_NAME: `/identity` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:00:48.299409 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b244ee2-d215-4d34-a390-8d3331b1182e None admin] PATH_INFO: `/v3/projects` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:00:48.300115 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b244ee2-d215-4d34-a390-8d3331b1182e None admin] RBAC: Authorizing `identity:create_project()` {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:00:48.301639 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b244ee2-d215-4d34-a390-8d3331b1182e None admin] RBAC: Authorization granted {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:00:48.304096 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a55409a9-a9c4-457b-835a-1bf28a37184f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '891005fa5ff74041b75747d16cc66798', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80378) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:00:48.305122 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a55409a9-a9c4-457b-835a-1bf28a37184f None admin] REQUEST_METHOD: `GET` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:00:48.305559 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a55409a9-a9c4-457b-835a-1bf28a37184f None admin] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:00:48.306040 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a55409a9-a9c4-457b-835a-1bf28a37184f None admin] PATH_INFO: `/v3/users/891005fa5ff74041b75747d16cc66798` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:00:48.308030 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a55409a9-a9c4-457b-835a-1bf28a37184f None admin] RBAC: Authorizing `identity:get_user(user_id=891005fa5ff74041b75747d16cc66798)` {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:00:48.310402 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a55409a9-a9c4-457b-835a-1bf28a37184f None admin] RBAC: Authorization granted {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:00:48.317932 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 28/54] 10.222.0.82 () {56 vars in 1177 bytes} [Tue Aug 2 10:00:48 2022] GET /identity/v3/users/891005fa5ff74041b75747d16cc66798 => generated 247 bytes in 216 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 02 10:00:48.320481 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a55409a9-a9c4-457b-835a-1bf28a37184f None admin] Authenticating user token {{(pid=80378) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:00:48.341226 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: [pid: 80377|app: 0|req: 27/55] 10.222.0.82 () {60 vars in 1140 bytes} [Tue Aug 2 10:00:48 2022] POST /identity/v3/projects => generated 298 bytes in 249 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 02 10:00:48.387694 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG dbcounter [-] [80378] Writing DB stats keystone:SELECT=100 {{(pid=80378) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 10:00:48.501266 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a55409a9-a9c4-457b-835a-1bf28a37184f None admin] Validating token access rules against request {{(pid=80378) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:00:48.568978 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb9dd0a3-557a-4dc5-a129-25b1d84a62ac None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '891005fa5ff74041b75747d16cc66798', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80378) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:00:48.569577 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb9dd0a3-557a-4dc5-a129-25b1d84a62ac None admin] REQUEST_METHOD: `GET` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:00:48.569883 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb9dd0a3-557a-4dc5-a129-25b1d84a62ac None admin] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:00:48.570165 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb9dd0a3-557a-4dc5-a129-25b1d84a62ac None admin] PATH_INFO: `/v3/domains/default` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:00:48.571368 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb9dd0a3-557a-4dc5-a129-25b1d84a62ac None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:00:48.572852 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb9dd0a3-557a-4dc5-a129-25b1d84a62ac None admin] RBAC: Authorization granted {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:00:48.574589 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 29/56] 10.222.0.82 () {56 vars in 1108 bytes} [Tue Aug 2 10:00:48 2022] GET /identity/v3/domains/default => generated 199 bytes in 255 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 02 10:00:48.585500 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b244ee2-d215-4d34-a390-8d3331b1182e None admin] Authenticating user token {{(pid=80377) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:00:48.737285 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b244ee2-d215-4d34-a390-8d3331b1182e None admin] Validating token access rules against request {{(pid=80377) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:00:48.782934 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-001dcd32-891f-46be-803f-596e2b6f3709 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '891005fa5ff74041b75747d16cc66798', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80377) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:00:48.784394 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-001dcd32-891f-46be-803f-596e2b6f3709 None admin] REQUEST_METHOD: `GET` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:00:48.784895 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-001dcd32-891f-46be-803f-596e2b6f3709 None admin] SCRIPT_NAME: `/identity` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:00:48.785274 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-001dcd32-891f-46be-803f-596e2b6f3709 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:00:48.786243 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-001dcd32-891f-46be-803f-596e2b6f3709 None admin] RBAC: Adding query filter params (role.id=237966442f05411191105bf336bc14f3, scope.domain.id=default, user.id=891005fa5ff74041b75747d16cc66798) {{(pid=80377) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 02 10:00:48.786652 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-001dcd32-891f-46be-803f-596e2b6f3709 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:00:48.788225 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-001dcd32-891f-46be-803f-596e2b6f3709 None admin] RBAC: Authorization granted {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:00:48.795213 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: [pid: 80377|app: 0|req: 28/57] 10.222.0.82 () {56 vars in 1322 bytes} [Tue Aug 2 10:00:48 2022] GET /identity/v3/role_assignments?user.id=891005fa5ff74041b75747d16cc66798&scope.domain.id=default&role.id=237966442f05411191105bf336bc14f3 => generated 530 bytes in 211 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 02 10:00:50.656676 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9660ec2d-3159-444b-9378-310a5291de40 None None] REQUEST_METHOD: `GET` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:00:50.657233 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9660ec2d-3159-444b-9378-310a5291de40 None None] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:00:50.657233 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9660ec2d-3159-444b-9378-310a5291de40 None None] PATH_INFO: `/` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:00:50.659164 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 30/58] 10.222.0.82 () {54 vars in 926 bytes} [Tue Aug 2 10:00:50 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 02 10:00:50.664824 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38060e92-5048-48d8-8d7b-a87a5319bf47 None None] REQUEST_METHOD: `POST` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:00:50.665646 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38060e92-5048-48d8-8d7b-a87a5319bf47 None None] SCRIPT_NAME: `/identity` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:00:50.665646 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38060e92-5048-48d8-8d7b-a87a5319bf47 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:00:50.687377 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.auth.core [None req-38060e92-5048-48d8-8d7b-a87a5319bf47 None None] MFA Rules not processed for user `891005fa5ff74041b75747d16cc66798`. Rule list: `[]` (Enabled: `True`). {{(pid=80377) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 02 10:00:50.688788 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.fernet_utils [None req-38060e92-5048-48d8-8d7b-a87a5319bf47 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80377) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 10:00:50.712012 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG dbcounter [-] [80377] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80377) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 10:00:50.787063 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: [pid: 80377|app: 0|req: 29/59] 10.222.0.82 () {58 vars in 1023 bytes} [Tue Aug 2 10:00:50 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 123 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Aug 02 10:00:50.811496 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9660ec2d-3159-444b-9378-310a5291de40 None None] Authenticating user token {{(pid=80378) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:00:50.948436 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9660ec2d-3159-444b-9378-310a5291de40 None None] Validating token access rules against request {{(pid=80378) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:00:50.951019 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG dbcounter [-] [80378] Writing DB stats keystone:SELECT=100 {{(pid=80378) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 10:00:51.010048 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd95122e-9ac5-4457-bec3-65b0f6b913b2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '891005fa5ff74041b75747d16cc66798', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80378) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:00:51.010708 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd95122e-9ac5-4457-bec3-65b0f6b913b2 None admin] REQUEST_METHOD: `GET` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:00:51.010990 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd95122e-9ac5-4457-bec3-65b0f6b913b2 None admin] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:00:51.011267 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd95122e-9ac5-4457-bec3-65b0f6b913b2 None admin] PATH_INFO: `/v3/domains/default` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:00:51.012500 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd95122e-9ac5-4457-bec3-65b0f6b913b2 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:00:51.014014 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd95122e-9ac5-4457-bec3-65b0f6b913b2 None admin] RBAC: Authorization granted {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:00:51.015740 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 31/60] 10.222.0.82 () {56 vars in 1108 bytes} [Tue Aug 2 10:00:50 2022] GET /identity/v3/domains/default => generated 199 bytes in 205 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 02 10:00:51.024310 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38060e92-5048-48d8-8d7b-a87a5319bf47 None None] Authenticating user token {{(pid=80377) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:00:51.163651 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38060e92-5048-48d8-8d7b-a87a5319bf47 None None] Validating token access rules against request {{(pid=80377) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:00:51.203352 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3be5797-e355-47fb-b00a-49da410bff27 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '891005fa5ff74041b75747d16cc66798', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80377) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:00:51.203903 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3be5797-e355-47fb-b00a-49da410bff27 None admin] REQUEST_METHOD: `POST` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:00:51.204190 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3be5797-e355-47fb-b00a-49da410bff27 None admin] SCRIPT_NAME: `/identity` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:00:51.204479 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3be5797-e355-47fb-b00a-49da410bff27 None admin] PATH_INFO: `/v3/users` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:00:51.205195 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3be5797-e355-47fb-b00a-49da410bff27 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:00:51.206789 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3be5797-e355-47fb-b00a-49da410bff27 None admin] RBAC: Authorization granted {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:00:51.245642 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: [pid: 80377|app: 0|req: 30/61] 10.222.0.82 () {60 vars in 1132 bytes} [Tue Aug 2 10:00:51 2022] POST /identity/v3/users => generated 275 bytes in 222 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 02 10:00:55.471865 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0a7e5e4-373a-41d5-a997-b7cf02faff5e None None] REQUEST_METHOD: `GET` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:00:55.472677 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0a7e5e4-373a-41d5-a997-b7cf02faff5e None None] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:00:55.472787 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0a7e5e4-373a-41d5-a997-b7cf02faff5e None None] PATH_INFO: `/` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:00:55.474862 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 32/62] 10.222.0.82 () {54 vars in 926 bytes} [Tue Aug 2 10:00:55 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 02 10:00:55.481768 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-086bb661-b5d9-438c-8d95-706ce54a0620 None None] REQUEST_METHOD: `POST` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:00:55.482397 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-086bb661-b5d9-438c-8d95-706ce54a0620 None None] SCRIPT_NAME: `/identity` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:00:55.482397 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-086bb661-b5d9-438c-8d95-706ce54a0620 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:00:55.489718 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-304746ef-652f-4a51-8c42-0e1fe4194ba3 None None] REQUEST_METHOD: `GET` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:00:55.490200 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-304746ef-652f-4a51-8c42-0e1fe4194ba3 None None] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:00:55.490597 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-304746ef-652f-4a51-8c42-0e1fe4194ba3 None None] PATH_INFO: `/` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:00:55.492576 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 33/63] 10.222.0.82 () {54 vars in 926 bytes} [Tue Aug 2 10:00:55 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 02 10:00:55.499485 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1dc03a66-8e7b-4ed2-bcaa-e8686cf1fcc0 None None] REQUEST_METHOD: `POST` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:00:55.499977 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1dc03a66-8e7b-4ed2-bcaa-e8686cf1fcc0 None None] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:00:55.500423 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1dc03a66-8e7b-4ed2-bcaa-e8686cf1fcc0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:00:55.527660 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.auth.core [None req-086bb661-b5d9-438c-8d95-706ce54a0620 None None] MFA Rules not processed for user `891005fa5ff74041b75747d16cc66798`. Rule list: `[]` (Enabled: `True`). {{(pid=80377) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 02 10:00:55.529810 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.fernet_utils [None req-086bb661-b5d9-438c-8d95-706ce54a0620 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80377) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 10:00:55.536347 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG dbcounter [-] [80377] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80377) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 10:00:55.537048 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.auth.core [None req-1dc03a66-8e7b-4ed2-bcaa-e8686cf1fcc0 None None] MFA Rules not processed for user `891005fa5ff74041b75747d16cc66798`. Rule list: `[]` (Enabled: `True`). {{(pid=80378) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 02 10:00:55.538910 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.fernet_utils [None req-1dc03a66-8e7b-4ed2-bcaa-e8686cf1fcc0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80378) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 10:00:55.671260 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: [pid: 80377|app: 0|req: 31/64] 10.222.0.82 () {58 vars in 1023 bytes} [Tue Aug 2 10:00:55 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 191 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 02 10:00:55.702407 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 34/65] 10.222.0.82 () {58 vars in 1023 bytes} [Tue Aug 2 10:00:55 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 205 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 02 10:00:55.706545 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-086bb661-b5d9-438c-8d95-706ce54a0620 None None] Authenticating user token {{(pid=80377) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:00:55.729575 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1dc03a66-8e7b-4ed2-bcaa-e8686cf1fcc0 None None] Authenticating user token {{(pid=80378) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:00:55.909226 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG dbcounter [-] [80378] Writing DB stats keystone:SELECT=100 {{(pid=80378) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 10:00:55.914892 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1dc03a66-8e7b-4ed2-bcaa-e8686cf1fcc0 None None] Validating token access rules against request {{(pid=80378) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:00:55.997506 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ece74c10-44db-4253-8df7-5166bcaaca3c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '891005fa5ff74041b75747d16cc66798', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80378) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:00:55.998150 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ece74c10-44db-4253-8df7-5166bcaaca3c None admin] REQUEST_METHOD: `GET` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:00:55.998437 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ece74c10-44db-4253-8df7-5166bcaaca3c None admin] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:00:55.999176 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ece74c10-44db-4253-8df7-5166bcaaca3c None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:00:56.002771 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-086bb661-b5d9-438c-8d95-706ce54a0620 None None] Validating token access rules against request {{(pid=80377) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:00:56.015828 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ece74c10-44db-4253-8df7-5166bcaaca3c None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 02 10:00:56.024841 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application [None req-ece74c10-44db-4253-8df7-5166bcaaca3c None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 02 10:00:56.024841 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 02 10:00:56.024841 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 02 10:00:56.024841 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 02 10:00:56.024841 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 02 10:00:56.024841 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 02 10:00:56.024841 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 02 10:00:56.024841 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 02 10:00:56.024841 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 02 10:00:56.024841 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 02 10:00:56.024841 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 02 10:00:56.024841 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 02 10:00:56.024841 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 02 10:00:56.024841 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 02 10:00:56.024841 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 02 10:00:56.024841 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 02 10:00:56.026317 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 02 10:00:56.026317 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 02 10:00:56.026317 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 02 10:00:56.026317 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 02 10:00:56.026317 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 02 10:00:56.026317 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 02 10:00:56.026317 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 02 10:00:56.026317 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 02 10:00:56.026317 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 02 10:00:56.026317 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application return self.get_or_create( Aug 02 10:00:56.026317 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 02 10:00:56.026317 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application with Lock( Aug 02 10:00:56.026317 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 02 10:00:56.026317 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application return self._enter() Aug 02 10:00:56.026317 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 02 10:00:56.026317 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 02 10:00:56.027666 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 02 10:00:56.027666 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application return self.creator() Aug 02 10:00:56.027666 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 02 10:00:56.027666 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application created_value = creator( Aug 02 10:00:56.027666 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 02 10:00:56.027666 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 02 10:00:56.027666 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 02 10:00:56.027666 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 02 10:00:56.027666 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 02 10:00:56.027666 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 02 10:00:56.027666 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 02 10:00:56.027666 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application Aug 02 10:00:56.027666 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 35/66] 10.222.0.82 () {56 vars in 1096 bytes} [Tue Aug 2 10:00:55 2022] GET /identity/v3/roles/admin => generated 83 bytes in 297 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 02 10:00:56.027666 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7363aa17-1dde-4cf7-bb78-3ebff2188680 None None] REQUEST_METHOD: `GET` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:00:56.028628 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7363aa17-1dde-4cf7-bb78-3ebff2188680 None None] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:00:56.028628 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7363aa17-1dde-4cf7-bb78-3ebff2188680 None None] PATH_INFO: `/` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:00:56.035434 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 36/67] 10.222.0.82 () {54 vars in 926 bytes} [Tue Aug 2 10:00:56 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 02 10:00:56.036421 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7363aa17-1dde-4cf7-bb78-3ebff2188680 None None] Authenticating user token {{(pid=80378) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:00:56.085300 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb42d303-9f4a-4ab8-8048-2c55984f8817 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '891005fa5ff74041b75747d16cc66798', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80377) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:00:56.086307 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb42d303-9f4a-4ab8-8048-2c55984f8817 None admin] REQUEST_METHOD: `GET` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:00:56.086757 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb42d303-9f4a-4ab8-8048-2c55984f8817 None admin] SCRIPT_NAME: `/identity` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:00:56.087151 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb42d303-9f4a-4ab8-8048-2c55984f8817 None admin] PATH_INFO: `/v3/roles/member` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:00:56.098735 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG dbcounter [-] [80377] Writing DB stats keystone:SELECT=100 {{(pid=80377) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 10:00:56.107707 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb42d303-9f4a-4ab8-8048-2c55984f8817 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 02 10:00:56.128014 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application [None req-fb42d303-9f4a-4ab8-8048-2c55984f8817 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Aug 02 10:00:56.128014 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 02 10:00:56.128014 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 02 10:00:56.128014 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 02 10:00:56.128014 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 02 10:00:56.128014 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 02 10:00:56.128014 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 02 10:00:56.128014 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 02 10:00:56.128014 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 02 10:00:56.128014 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 02 10:00:56.128014 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 02 10:00:56.128014 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 02 10:00:56.128014 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 02 10:00:56.128014 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 02 10:00:56.128014 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 02 10:00:56.128014 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 02 10:00:56.128995 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 02 10:00:56.128995 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 02 10:00:56.128995 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 02 10:00:56.128995 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 02 10:00:56.128995 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 02 10:00:56.128995 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 02 10:00:56.128995 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 02 10:00:56.128995 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 02 10:00:56.128995 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 02 10:00:56.128995 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application return self.get_or_create( Aug 02 10:00:56.128995 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 02 10:00:56.128995 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application with Lock( Aug 02 10:00:56.128995 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 02 10:00:56.128995 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application return self._enter() Aug 02 10:00:56.128995 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 02 10:00:56.128995 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 02 10:00:56.129958 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 02 10:00:56.129958 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application return self.creator() Aug 02 10:00:56.129958 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 02 10:00:56.129958 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application created_value = creator( Aug 02 10:00:56.129958 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 02 10:00:56.129958 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 02 10:00:56.129958 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 02 10:00:56.129958 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 02 10:00:56.129958 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 02 10:00:56.129958 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 02 10:00:56.129958 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Aug 02 10:00:56.129958 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application Aug 02 10:00:56.132945 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: [pid: 80377|app: 0|req: 32/68] 10.222.0.82 () {56 vars in 1099 bytes} [Tue Aug 2 10:00:55 2022] GET /identity/v3/roles/member => generated 84 bytes in 428 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 02 10:00:56.135051 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85c805f8-292b-4e81-a03f-81b7779fbe6e None None] REQUEST_METHOD: `POST` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:00:56.135577 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85c805f8-292b-4e81-a03f-81b7779fbe6e None None] SCRIPT_NAME: `/identity` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:00:56.136080 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85c805f8-292b-4e81-a03f-81b7779fbe6e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:00:56.156469 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.auth.core [None req-85c805f8-292b-4e81-a03f-81b7779fbe6e None None] MFA Rules not processed for user `891005fa5ff74041b75747d16cc66798`. Rule list: `[]` (Enabled: `True`). {{(pid=80377) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 02 10:00:56.159078 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.fernet_utils [None req-85c805f8-292b-4e81-a03f-81b7779fbe6e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80377) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 10:00:56.235425 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7363aa17-1dde-4cf7-bb78-3ebff2188680 None None] Validating token access rules against request {{(pid=80378) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:00:56.256456 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: [pid: 80377|app: 0|req: 33/69] 10.222.0.82 () {58 vars in 1023 bytes} [Tue Aug 2 10:00:56 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 123 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 02 10:00:56.257262 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85c805f8-292b-4e81-a03f-81b7779fbe6e None None] Authenticating user token {{(pid=80377) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:00:56.310647 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-784c7709-da06-44bb-8359-eab9e20e52d1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '891005fa5ff74041b75747d16cc66798', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80378) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:00:56.311313 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-784c7709-da06-44bb-8359-eab9e20e52d1 None admin] REQUEST_METHOD: `GET` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:00:56.311603 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-784c7709-da06-44bb-8359-eab9e20e52d1 None admin] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:00:56.311887 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-784c7709-da06-44bb-8359-eab9e20e52d1 None admin] PATH_INFO: `/v3/roles` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:00:56.312640 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-784c7709-da06-44bb-8359-eab9e20e52d1 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80378) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 02 10:00:56.312956 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-784c7709-da06-44bb-8359-eab9e20e52d1 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:00:56.314546 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-784c7709-da06-44bb-8359-eab9e20e52d1 None admin] RBAC: Authorization granted {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:00:56.328154 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 37/70] 10.222.0.82 () {56 vars in 1099 bytes} [Tue Aug 2 10:00:56 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 292 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 02 10:00:56.330399 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-784c7709-da06-44bb-8359-eab9e20e52d1 None admin] Authenticating user token {{(pid=80378) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:00:56.386386 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG dbcounter [-] [80378] Writing DB stats keystone:SELECT=100 {{(pid=80378) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 10:00:56.429475 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85c805f8-292b-4e81-a03f-81b7779fbe6e None None] Validating token access rules against request {{(pid=80377) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:00:56.478589 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG dbcounter [-] [80377] Writing DB stats keystone:SELECT=100 {{(pid=80377) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 10:00:56.490605 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a266c9d-5498-48ad-9f2e-df4eb4405a3b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '891005fa5ff74041b75747d16cc66798', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80377) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:00:56.491560 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a266c9d-5498-48ad-9f2e-df4eb4405a3b None admin] REQUEST_METHOD: `GET` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:00:56.491649 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a266c9d-5498-48ad-9f2e-df4eb4405a3b None admin] SCRIPT_NAME: `/identity` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:00:56.492046 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a266c9d-5498-48ad-9f2e-df4eb4405a3b None admin] PATH_INFO: `/v3/roles` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:00:56.492886 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a266c9d-5498-48ad-9f2e-df4eb4405a3b None admin] RBAC: Adding query filter params (name=member) {{(pid=80377) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 02 10:00:56.493280 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a266c9d-5498-48ad-9f2e-df4eb4405a3b None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:00:56.494953 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a266c9d-5498-48ad-9f2e-df4eb4405a3b None admin] RBAC: Authorization granted {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:00:56.508194 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: [pid: 80377|app: 0|req: 34/71] 10.222.0.82 () {56 vars in 1101 bytes} [Tue Aug 2 10:00:56 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 251 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 02 10:00:56.508835 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a266c9d-5498-48ad-9f2e-df4eb4405a3b None admin] Authenticating user token {{(pid=80377) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:00:56.559793 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-784c7709-da06-44bb-8359-eab9e20e52d1 None admin] Validating token access rules against request {{(pid=80378) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:00:56.629749 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb1bf19b-9118-4caa-a63f-482a285a0d3a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '891005fa5ff74041b75747d16cc66798', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80378) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:00:56.631185 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb1bf19b-9118-4caa-a63f-482a285a0d3a None admin] REQUEST_METHOD: `GET` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:00:56.631701 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb1bf19b-9118-4caa-a63f-482a285a0d3a None admin] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:00:56.632188 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb1bf19b-9118-4caa-a63f-482a285a0d3a None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:00:56.652841 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb1bf19b-9118-4caa-a63f-482a285a0d3a None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 02 10:00:56.653076 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a266c9d-5498-48ad-9f2e-df4eb4405a3b None admin] Validating token access rules against request {{(pid=80377) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:00:56.665108 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application [None req-bb1bf19b-9118-4caa-a63f-482a285a0d3a None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Aug 02 10:00:56.665108 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 02 10:00:56.665108 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 02 10:00:56.665108 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 02 10:00:56.665108 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 02 10:00:56.665108 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 02 10:00:56.665108 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 02 10:00:56.665108 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 02 10:00:56.665108 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 02 10:00:56.665108 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 02 10:00:56.665108 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 02 10:00:56.665108 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 02 10:00:56.665108 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 02 10:00:56.665108 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 02 10:00:56.665108 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 02 10:00:56.665108 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 02 10:00:56.666577 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 02 10:00:56.666577 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 02 10:00:56.666577 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 02 10:00:56.666577 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 02 10:00:56.666577 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 02 10:00:56.666577 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 02 10:00:56.666577 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 02 10:00:56.666577 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 02 10:00:56.666577 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 02 10:00:56.666577 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application return self.get_or_create( Aug 02 10:00:56.666577 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 02 10:00:56.666577 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application with Lock( Aug 02 10:00:56.666577 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 02 10:00:56.666577 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application return self._enter() Aug 02 10:00:56.666577 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 02 10:00:56.666577 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 02 10:00:56.667939 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 02 10:00:56.667939 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application return self.creator() Aug 02 10:00:56.667939 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 02 10:00:56.667939 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application created_value = creator( Aug 02 10:00:56.667939 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 02 10:00:56.667939 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 02 10:00:56.667939 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 02 10:00:56.667939 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 02 10:00:56.667939 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 02 10:00:56.667939 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 02 10:00:56.667939 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Aug 02 10:00:56.667939 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application Aug 02 10:00:56.667939 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 38/72] 10.222.0.82 () {56 vars in 1114 bytes} [Tue Aug 2 10:00:56 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 339 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 02 10:00:56.669578 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb1bf19b-9118-4caa-a63f-482a285a0d3a None admin] Authenticating user token {{(pid=80378) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:00:56.714397 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77d0162b-5512-4391-9d3a-60b8dbd460c7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '891005fa5ff74041b75747d16cc66798', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80377) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:00:56.715331 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77d0162b-5512-4391-9d3a-60b8dbd460c7 None admin] REQUEST_METHOD: `GET` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:00:56.715623 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77d0162b-5512-4391-9d3a-60b8dbd460c7 None admin] SCRIPT_NAME: `/identity` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:00:56.715903 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77d0162b-5512-4391-9d3a-60b8dbd460c7 None admin] PATH_INFO: `/v3/users/891005fa5ff74041b75747d16cc66798` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:00:56.717261 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77d0162b-5512-4391-9d3a-60b8dbd460c7 None admin] RBAC: Authorizing `identity:get_user(user_id=891005fa5ff74041b75747d16cc66798)` {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:00:56.718881 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77d0162b-5512-4391-9d3a-60b8dbd460c7 None admin] RBAC: Authorization granted {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:00:56.720673 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: [pid: 80377|app: 0|req: 35/73] 10.222.0.82 () {56 vars in 1177 bytes} [Tue Aug 2 10:00:56 2022] GET /identity/v3/users/891005fa5ff74041b75747d16cc66798 => generated 247 bytes in 212 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 02 10:00:56.726612 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18d73c49-1bea-465a-be4f-a79654614154 None None] REQUEST_METHOD: `GET` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:00:56.726936 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18d73c49-1bea-465a-be4f-a79654614154 None None] SCRIPT_NAME: `/identity` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:00:56.727191 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18d73c49-1bea-465a-be4f-a79654614154 None None] PATH_INFO: `/` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:00:56.728765 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: [pid: 80377|app: 0|req: 36/74] 10.222.0.82 () {54 vars in 926 bytes} [Tue Aug 2 10:00:56 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 02 10:00:56.729253 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18d73c49-1bea-465a-be4f-a79654614154 None None] Authenticating user token {{(pid=80377) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:00:56.825743 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG dbcounter [-] [80377] Writing DB stats keystone:SELECT=100 {{(pid=80377) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 10:00:56.828037 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG dbcounter [-] [80378] Writing DB stats keystone:SELECT=100 {{(pid=80378) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 10:00:56.854280 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb1bf19b-9118-4caa-a63f-482a285a0d3a None admin] Validating token access rules against request {{(pid=80378) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:00:56.893224 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18d73c49-1bea-465a-be4f-a79654614154 None None] Validating token access rules against request {{(pid=80377) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:00:56.901022 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28cc53b6-08e9-4272-890e-d1b19a97fdd1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '891005fa5ff74041b75747d16cc66798', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80378) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:00:56.901919 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28cc53b6-08e9-4272-890e-d1b19a97fdd1 None admin] REQUEST_METHOD: `GET` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:00:56.902278 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28cc53b6-08e9-4272-890e-d1b19a97fdd1 None admin] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:00:56.902604 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28cc53b6-08e9-4272-890e-d1b19a97fdd1 None admin] PATH_INFO: `/v3/users/71a4a6ffabe74df39f0a7ca7ea7b33b8` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:00:56.930289 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28cc53b6-08e9-4272-890e-d1b19a97fdd1 None admin] RBAC: Authorizing `identity:get_user(user_id=71a4a6ffabe74df39f0a7ca7ea7b33b8)` {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:00:56.932146 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28cc53b6-08e9-4272-890e-d1b19a97fdd1 None admin] RBAC: Authorization granted {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:00:56.934560 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 39/75] 10.222.0.82 () {56 vars in 1177 bytes} [Tue Aug 2 10:00:56 2022] GET /identity/v3/users/71a4a6ffabe74df39f0a7ca7ea7b33b8 => generated 275 bytes in 266 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 02 10:00:56.935963 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28cc53b6-08e9-4272-890e-d1b19a97fdd1 None admin] Authenticating user token {{(pid=80378) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:00:56.951248 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8703b6b3-3244-44b4-a6fc-f0e3a30fb283 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '891005fa5ff74041b75747d16cc66798', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80377) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:00:56.951984 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8703b6b3-3244-44b4-a6fc-f0e3a30fb283 None admin] REQUEST_METHOD: `GET` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:00:56.952352 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8703b6b3-3244-44b4-a6fc-f0e3a30fb283 None admin] SCRIPT_NAME: `/identity` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:00:56.952707 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8703b6b3-3244-44b4-a6fc-f0e3a30fb283 None admin] PATH_INFO: `/v3/roles` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:00:56.953460 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8703b6b3-3244-44b4-a6fc-f0e3a30fb283 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=80377) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 02 10:00:56.953861 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8703b6b3-3244-44b4-a6fc-f0e3a30fb283 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:00:56.955920 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8703b6b3-3244-44b4-a6fc-f0e3a30fb283 None admin] RBAC: Authorization granted {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:00:56.966757 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: [pid: 80377|app: 0|req: 37/76] 10.222.0.82 () {56 vars in 1111 bytes} [Tue Aug 2 10:00:56 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 237 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 02 10:00:56.967794 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb3bdbef-477a-4cdd-802d-901542eafbcf None None] REQUEST_METHOD: `POST` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:00:56.968178 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb3bdbef-477a-4cdd-802d-901542eafbcf None None] SCRIPT_NAME: `/identity` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:00:56.968464 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb3bdbef-477a-4cdd-802d-901542eafbcf None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:00:56.989822 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.auth.core [None req-fb3bdbef-477a-4cdd-802d-901542eafbcf None None] MFA Rules not processed for user `891005fa5ff74041b75747d16cc66798`. Rule list: `[]` (Enabled: `True`). {{(pid=80377) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 02 10:00:56.991071 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.fernet_utils [None req-fb3bdbef-477a-4cdd-802d-901542eafbcf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80377) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 10:00:57.083037 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: [pid: 80377|app: 0|req: 38/77] 10.222.0.82 () {58 vars in 1023 bytes} [Tue Aug 2 10:00:56 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 116 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 02 10:00:57.084482 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c367604-a88a-4b4b-a7ca-d7877c9acfc0 None None] REQUEST_METHOD: `GET` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:00:57.084842 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c367604-a88a-4b4b-a7ca-d7877c9acfc0 None None] SCRIPT_NAME: `/identity` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:00:57.085126 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c367604-a88a-4b4b-a7ca-d7877c9acfc0 None None] PATH_INFO: `/` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:00:57.089839 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: [pid: 80377|app: 0|req: 39/78] 10.222.0.82 () {54 vars in 926 bytes} [Tue Aug 2 10:00:57 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 02 10:00:57.089839 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c367604-a88a-4b4b-a7ca-d7877c9acfc0 None None] Authenticating user token {{(pid=80377) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:00:57.090102 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28cc53b6-08e9-4272-890e-d1b19a97fdd1 None admin] Validating token access rules against request {{(pid=80378) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:00:57.139285 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9073b416-455f-488a-aab5-cce9aab01a5e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '891005fa5ff74041b75747d16cc66798', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80378) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:00:57.139930 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9073b416-455f-488a-aab5-cce9aab01a5e None admin] REQUEST_METHOD: `GET` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:00:57.140219 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9073b416-455f-488a-aab5-cce9aab01a5e None admin] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:00:57.140500 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9073b416-455f-488a-aab5-cce9aab01a5e None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:00:57.160711 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG dbcounter [-] [80378] Writing DB stats keystone:SELECT=100 {{(pid=80378) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 10:00:57.163813 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG dbcounter [-] [80377] Writing DB stats keystone:SELECT=100 {{(pid=80377) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 10:00:57.233374 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c367604-a88a-4b4b-a7ca-d7877c9acfc0 None None] Validating token access rules against request {{(pid=80377) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:00:57.253690 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9073b416-455f-488a-aab5-cce9aab01a5e None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:00:57.256084 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9073b416-455f-488a-aab5-cce9aab01a5e None admin] RBAC: Authorization granted {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:00:57.278317 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fec6c38-0183-4f96-bc4f-4a2842fe333f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '891005fa5ff74041b75747d16cc66798', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80377) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:00:57.278961 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fec6c38-0183-4f96-bc4f-4a2842fe333f None admin] REQUEST_METHOD: `GET` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:00:57.279249 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fec6c38-0183-4f96-bc4f-4a2842fe333f None admin] SCRIPT_NAME: `/identity` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:00:57.279532 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fec6c38-0183-4f96-bc4f-4a2842fe333f None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:00:57.384188 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8fec6c38-0183-4f96-bc4f-4a2842fe333f None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:00:57.385591 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8fec6c38-0183-4f96-bc4f-4a2842fe333f None admin] RBAC: Authorization granted {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:00:57.407224 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 40/79] 10.222.0.82 () {58 vars in 1282 bytes} [Tue Aug 2 10:00:56 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 472 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 02 10:00:57.408201 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9073b416-455f-488a-aab5-cce9aab01a5e None admin] Authenticating user token {{(pid=80378) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:00:57.462441 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG dbcounter [-] [80378] Writing DB stats keystone:SELECT=100 {{(pid=80378) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 10:00:57.466449 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG dbcounter [-] [80377] Writing DB stats keystone:SELECT=100 {{(pid=80377) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 10:00:57.545341 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9073b416-455f-488a-aab5-cce9aab01a5e None admin] Validating token access rules against request {{(pid=80378) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:00:57.548407 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: [pid: 80377|app: 0|req: 40/80] 10.222.0.82 () {58 vars in 1282 bytes} [Tue Aug 2 10:00:57 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 461 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 02 10:00:57.550449 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52ad255f-38cc-4751-9e5f-96959f75df33 None None] REQUEST_METHOD: `POST` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:00:57.550912 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52ad255f-38cc-4751-9e5f-96959f75df33 None None] SCRIPT_NAME: `/identity` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:00:57.551391 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52ad255f-38cc-4751-9e5f-96959f75df33 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:00:57.573284 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.auth.core [None req-52ad255f-38cc-4751-9e5f-96959f75df33 None None] MFA Rules not processed for user `891005fa5ff74041b75747d16cc66798`. Rule list: `[]` (Enabled: `True`). {{(pid=80377) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 02 10:00:57.574567 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.fernet_utils [None req-52ad255f-38cc-4751-9e5f-96959f75df33 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80377) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 10:00:57.595755 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0d5388d-a895-4a66-b621-5b6d039344e0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '891005fa5ff74041b75747d16cc66798', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80378) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:00:57.596489 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0d5388d-a895-4a66-b621-5b6d039344e0 None admin] REQUEST_METHOD: `GET` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:00:57.596789 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0d5388d-a895-4a66-b621-5b6d039344e0 None admin] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:00:57.597077 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0d5388d-a895-4a66-b621-5b6d039344e0 None admin] PATH_INFO: `/v3/users/71a4a6ffabe74df39f0a7ca7ea7b33b8` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:00:57.598543 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0d5388d-a895-4a66-b621-5b6d039344e0 None admin] RBAC: Authorizing `identity:get_user(user_id=71a4a6ffabe74df39f0a7ca7ea7b33b8)` {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:00:57.600077 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0d5388d-a895-4a66-b621-5b6d039344e0 None admin] RBAC: Authorization granted {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:00:57.601848 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 41/81] 10.222.0.82 () {56 vars in 1177 bytes} [Tue Aug 2 10:00:57 2022] GET /identity/v3/users/71a4a6ffabe74df39f0a7ca7ea7b33b8 => generated 275 bytes in 194 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 02 10:00:57.602899 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0d5388d-a895-4a66-b621-5b6d039344e0 None admin] Authenticating user token {{(pid=80378) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:00:57.687835 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: [pid: 80377|app: 0|req: 41/82] 10.222.0.82 () {58 vars in 1023 bytes} [Tue Aug 2 10:00:57 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 139 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 02 10:00:57.688624 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52ad255f-38cc-4751-9e5f-96959f75df33 None None] Authenticating user token {{(pid=80377) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:00:57.766809 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0d5388d-a895-4a66-b621-5b6d039344e0 None admin] Validating token access rules against request {{(pid=80378) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:00:57.780772 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG dbcounter [-] [80378] Writing DB stats keystone:SELECT=100 {{(pid=80378) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 10:00:57.797706 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG dbcounter [-] [80377] Writing DB stats keystone:SELECT=100 {{(pid=80377) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 10:00:57.812659 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1890f49b-e18b-48c9-acfc-2bfa48fb3f60 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '891005fa5ff74041b75747d16cc66798', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80378) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:00:57.813514 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1890f49b-e18b-48c9-acfc-2bfa48fb3f60 None admin] REQUEST_METHOD: `GET` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:00:57.814506 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1890f49b-e18b-48c9-acfc-2bfa48fb3f60 None admin] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:00:57.814875 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1890f49b-e18b-48c9-acfc-2bfa48fb3f60 None admin] PATH_INFO: `/v3/domains/default` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:00:57.816294 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1890f49b-e18b-48c9-acfc-2bfa48fb3f60 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:00:57.817906 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1890f49b-e18b-48c9-acfc-2bfa48fb3f60 None admin] RBAC: Authorization granted {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:00:57.819577 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 42/83] 10.222.0.82 () {56 vars in 1108 bytes} [Tue Aug 2 10:00:57 2022] GET /identity/v3/domains/default => generated 199 bytes in 217 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 02 10:00:57.820390 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1890f49b-e18b-48c9-acfc-2bfa48fb3f60 None admin] Authenticating user token {{(pid=80378) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:00:57.823283 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52ad255f-38cc-4751-9e5f-96959f75df33 None None] Validating token access rules against request {{(pid=80377) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:00:57.873041 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0bfd58b7-2e2d-49df-bc5e-ab1999315d83 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '891005fa5ff74041b75747d16cc66798', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80377) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:00:57.874211 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bfd58b7-2e2d-49df-bc5e-ab1999315d83 None admin] REQUEST_METHOD: `GET` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:00:57.874499 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bfd58b7-2e2d-49df-bc5e-ab1999315d83 None admin] SCRIPT_NAME: `/identity` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:00:57.874781 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bfd58b7-2e2d-49df-bc5e-ab1999315d83 None admin] PATH_INFO: `/v3/projects/c1809c9e380b45ec803130fe38eeaf24` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:00:57.876018 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0bfd58b7-2e2d-49df-bc5e-ab1999315d83 None admin] RBAC: Authorizing `identity:get_project(project_id=c1809c9e380b45ec803130fe38eeaf24)` {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:00:57.878123 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0bfd58b7-2e2d-49df-bc5e-ab1999315d83 None admin] RBAC: Authorization granted {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:00:57.879846 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: [pid: 80377|app: 0|req: 42/84] 10.222.0.82 () {56 vars in 1186 bytes} [Tue Aug 2 10:00:57 2022] GET /identity/v3/projects/c1809c9e380b45ec803130fe38eeaf24 => generated 298 bytes in 192 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 02 10:00:57.880631 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0bfd58b7-2e2d-49df-bc5e-ab1999315d83 None admin] Authenticating user token {{(pid=80377) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:00:57.960072 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1890f49b-e18b-48c9-acfc-2bfa48fb3f60 None admin] Validating token access rules against request {{(pid=80378) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:00:57.998432 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0bfd58b7-2e2d-49df-bc5e-ab1999315d83 None admin] Validating token access rules against request {{(pid=80377) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:00:57.998828 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-968f7862-f03d-43d9-b68d-37ccdc7b3954 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '891005fa5ff74041b75747d16cc66798', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80378) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:00:57.999495 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-968f7862-f03d-43d9-b68d-37ccdc7b3954 None admin] REQUEST_METHOD: `GET` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:00:57.999788 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-968f7862-f03d-43d9-b68d-37ccdc7b3954 None admin] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:00:58.000066 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-968f7862-f03d-43d9-b68d-37ccdc7b3954 None admin] PATH_INFO: `/v3/projects/c1809c9e380b45ec803130fe38eeaf24` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:00:58.001908 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-968f7862-f03d-43d9-b68d-37ccdc7b3954 None admin] RBAC: Authorizing `identity:get_project(project_id=c1809c9e380b45ec803130fe38eeaf24)` {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:00:58.003405 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-968f7862-f03d-43d9-b68d-37ccdc7b3954 None admin] RBAC: Authorization granted {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:00:58.005891 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 43/85] 10.222.0.82 () {56 vars in 1186 bytes} [Tue Aug 2 10:00:57 2022] GET /identity/v3/projects/c1809c9e380b45ec803130fe38eeaf24 => generated 298 bytes in 185 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 02 10:00:58.006080 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-968f7862-f03d-43d9-b68d-37ccdc7b3954 None admin] Authenticating user token {{(pid=80378) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:00:58.041432 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4a2f321-55a2-44c2-afdc-b1e8ec30e91d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '891005fa5ff74041b75747d16cc66798', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80377) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:00:58.042124 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4a2f321-55a2-44c2-afdc-b1e8ec30e91d None admin] REQUEST_METHOD: `GET` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:00:58.042410 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4a2f321-55a2-44c2-afdc-b1e8ec30e91d None admin] SCRIPT_NAME: `/identity` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:00:58.042688 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4a2f321-55a2-44c2-afdc-b1e8ec30e91d None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:00:58.062138 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG dbcounter [-] [80378] Writing DB stats keystone:SELECT=100 {{(pid=80378) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 10:00:58.062778 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG dbcounter [-] [80377] Writing DB stats keystone:SELECT=100 {{(pid=80377) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 10:00:58.131776 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4a2f321-55a2-44c2-afdc-b1e8ec30e91d None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:00:58.134155 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4a2f321-55a2-44c2-afdc-b1e8ec30e91d None admin] RBAC: Authorization granted {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:00:58.160352 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-968f7862-f03d-43d9-b68d-37ccdc7b3954 None admin] Validating token access rules against request {{(pid=80378) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:00:58.216641 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b58689f8-9246-4194-b59b-bdfc3002f1b3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '891005fa5ff74041b75747d16cc66798', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80378) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:00:58.217621 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b58689f8-9246-4194-b59b-bdfc3002f1b3 None admin] REQUEST_METHOD: `GET` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:00:58.218136 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b58689f8-9246-4194-b59b-bdfc3002f1b3 None admin] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:00:58.218612 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b58689f8-9246-4194-b59b-bdfc3002f1b3 None admin] PATH_INFO: `/v3/roles/member` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:00:58.238321 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b58689f8-9246-4194-b59b-bdfc3002f1b3 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 02 10:00:58.247763 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application [None req-b58689f8-9246-4194-b59b-bdfc3002f1b3 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Aug 02 10:00:58.247763 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 02 10:00:58.247763 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 02 10:00:58.247763 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 02 10:00:58.247763 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 02 10:00:58.247763 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 02 10:00:58.247763 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 02 10:00:58.247763 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 02 10:00:58.247763 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 02 10:00:58.247763 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 02 10:00:58.247763 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 02 10:00:58.247763 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 02 10:00:58.247763 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 02 10:00:58.247763 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 02 10:00:58.247763 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 02 10:00:58.247763 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 02 10:00:58.248712 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 02 10:00:58.248712 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 02 10:00:58.248712 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 02 10:00:58.248712 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 02 10:00:58.248712 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 02 10:00:58.248712 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 02 10:00:58.248712 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 02 10:00:58.248712 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 02 10:00:58.248712 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 02 10:00:58.248712 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application return self.get_or_create( Aug 02 10:00:58.248712 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 02 10:00:58.248712 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application with Lock( Aug 02 10:00:58.248712 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 02 10:00:58.248712 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application return self._enter() Aug 02 10:00:58.248712 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 02 10:00:58.248712 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 02 10:00:58.249491 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 02 10:00:58.249491 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application return self.creator() Aug 02 10:00:58.249491 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 02 10:00:58.249491 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application created_value = creator( Aug 02 10:00:58.249491 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 02 10:00:58.249491 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 02 10:00:58.249491 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 02 10:00:58.249491 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 02 10:00:58.249491 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 02 10:00:58.249491 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 02 10:00:58.249491 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Aug 02 10:00:58.249491 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application Aug 02 10:00:58.250144 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 44/86] 10.222.0.82 () {56 vars in 1099 bytes} [Tue Aug 2 10:00:58 2022] GET /identity/v3/roles/member => generated 84 bytes in 244 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 02 10:00:58.251340 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b58689f8-9246-4194-b59b-bdfc3002f1b3 None admin] Authenticating user token {{(pid=80378) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:00:58.256155 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: [pid: 80377|app: 0|req: 43/87] 10.222.0.82 () {58 vars in 1282 bytes} [Tue Aug 2 10:00:57 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 376 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 02 10:00:58.256951 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4a2f321-55a2-44c2-afdc-b1e8ec30e91d None admin] Authenticating user token {{(pid=80377) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:00:58.319309 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG dbcounter [-] [80377] Writing DB stats keystone:SELECT=100 {{(pid=80377) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 10:00:58.392579 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b58689f8-9246-4194-b59b-bdfc3002f1b3 None admin] Validating token access rules against request {{(pid=80378) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:00:58.396350 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG dbcounter [-] [80378] Writing DB stats keystone:SELECT=100 {{(pid=80378) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 10:00:58.416452 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4a2f321-55a2-44c2-afdc-b1e8ec30e91d None admin] Validating token access rules against request {{(pid=80377) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:00:58.431691 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b1dc806-ab36-4510-8c13-fb6a2bf62735 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '891005fa5ff74041b75747d16cc66798', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80378) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:00:58.432330 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b1dc806-ab36-4510-8c13-fb6a2bf62735 None admin] REQUEST_METHOD: `POST` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:00:58.432624 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b1dc806-ab36-4510-8c13-fb6a2bf62735 None admin] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:00:58.432907 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b1dc806-ab36-4510-8c13-fb6a2bf62735 None admin] PATH_INFO: `/v3/users` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:00:58.433612 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b1dc806-ab36-4510-8c13-fb6a2bf62735 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:00:58.435126 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b1dc806-ab36-4510-8c13-fb6a2bf62735 None admin] RBAC: Authorization granted {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:00:58.467815 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 45/88] 10.222.0.82 () {60 vars in 1132 bytes} [Tue Aug 2 10:00:58 2022] POST /identity/v3/users => generated 289 bytes in 217 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 02 10:00:58.468960 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b1dc806-ab36-4510-8c13-fb6a2bf62735 None admin] Authenticating user token {{(pid=80378) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:00:58.491866 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c7bd942-244d-49c4-95a1-cee3703c5fd8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '891005fa5ff74041b75747d16cc66798', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80377) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:00:58.492729 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c7bd942-244d-49c4-95a1-cee3703c5fd8 None admin] REQUEST_METHOD: `GET` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:00:58.493025 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c7bd942-244d-49c4-95a1-cee3703c5fd8 None admin] SCRIPT_NAME: `/identity` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:00:58.493312 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c7bd942-244d-49c4-95a1-cee3703c5fd8 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:00:58.494181 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c7bd942-244d-49c4-95a1-cee3703c5fd8 None admin] RBAC: Adding query filter params (role.id=237966442f05411191105bf336bc14f3, scope.project.id=c1809c9e380b45ec803130fe38eeaf24, user.id=891005fa5ff74041b75747d16cc66798) {{(pid=80377) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 02 10:00:58.494504 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c7bd942-244d-49c4-95a1-cee3703c5fd8 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:00:58.496015 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c7bd942-244d-49c4-95a1-cee3703c5fd8 None admin] RBAC: Authorization granted {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:00:58.502642 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: [pid: 80377|app: 0|req: 44/89] 10.222.0.82 () {56 vars in 1374 bytes} [Tue Aug 2 10:00:58 2022] GET /identity/v3/role_assignments?user.id=891005fa5ff74041b75747d16cc66798&scope.project.id=c1809c9e380b45ec803130fe38eeaf24&role.id=237966442f05411191105bf336bc14f3 => generated 259 bytes in 246 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 02 10:00:58.504132 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c7bd942-244d-49c4-95a1-cee3703c5fd8 None admin] Authenticating user token {{(pid=80377) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:00:58.669301 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c7bd942-244d-49c4-95a1-cee3703c5fd8 None admin] Validating token access rules against request {{(pid=80377) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:00:58.677823 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b1dc806-ab36-4510-8c13-fb6a2bf62735 None admin] Validating token access rules against request {{(pid=80378) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:00:58.688997 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG dbcounter [-] [80377] Writing DB stats keystone:SELECT=100 {{(pid=80377) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 10:00:58.726239 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4e7658c-fec3-47bc-bc5a-a2a2649c08eb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '891005fa5ff74041b75747d16cc66798', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80377) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:00:58.728050 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4e7658c-fec3-47bc-bc5a-a2a2649c08eb None admin] REQUEST_METHOD: `GET` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:00:58.728669 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4e7658c-fec3-47bc-bc5a-a2a2649c08eb None admin] SCRIPT_NAME: `/identity` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:00:58.729739 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4e7658c-fec3-47bc-bc5a-a2a2649c08eb None admin] PATH_INFO: `/v3/roles` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:00:58.731082 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4e7658c-fec3-47bc-bc5a-a2a2649c08eb None admin] RBAC: Adding query filter params (name=member) {{(pid=80377) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 02 10:00:58.731658 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4e7658c-fec3-47bc-bc5a-a2a2649c08eb None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:00:58.733217 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4e7658c-fec3-47bc-bc5a-a2a2649c08eb None admin] RBAC: Authorization granted {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:00:58.739083 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a09df94-445a-43ba-b7a0-44fb806f252b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '891005fa5ff74041b75747d16cc66798', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80378) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:00:58.740091 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a09df94-445a-43ba-b7a0-44fb806f252b None admin] REQUEST_METHOD: `GET` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:00:58.740587 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a09df94-445a-43ba-b7a0-44fb806f252b None admin] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:00:58.741077 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a09df94-445a-43ba-b7a0-44fb806f252b None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:00:58.743721 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a09df94-445a-43ba-b7a0-44fb806f252b None admin] RBAC: Adding query filter params (role.id=e1c04b19fc4543b8b54f58bbcffbc598, scope.project.id=c1809c9e380b45ec803130fe38eeaf24, user.id=71a4a6ffabe74df39f0a7ca7ea7b33b8) {{(pid=80378) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 02 10:00:58.744270 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a09df94-445a-43ba-b7a0-44fb806f252b None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:00:58.744679 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: [pid: 80377|app: 0|req: 45/90] 10.222.0.82 () {56 vars in 1101 bytes} [Tue Aug 2 10:00:58 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 241 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 02 10:00:58.745547 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4e7658c-fec3-47bc-bc5a-a2a2649c08eb None admin] Authenticating user token {{(pid=80377) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:00:58.747388 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a09df94-445a-43ba-b7a0-44fb806f252b None admin] RBAC: Authorization granted {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:00:58.764291 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 46/91] 10.222.0.82 () {56 vars in 1374 bytes} [Tue Aug 2 10:00:58 2022] GET /identity/v3/role_assignments?user.id=71a4a6ffabe74df39f0a7ca7ea7b33b8&scope.project.id=c1809c9e380b45ec803130fe38eeaf24&role.id=e1c04b19fc4543b8b54f58bbcffbc598 => generated 259 bytes in 296 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 02 10:00:58.765747 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a09df94-445a-43ba-b7a0-44fb806f252b None admin] Authenticating user token {{(pid=80378) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:00:58.814537 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG dbcounter [-] [80378] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80378) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 10:00:58.949837 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4e7658c-fec3-47bc-bc5a-a2a2649c08eb None admin] Validating token access rules against request {{(pid=80377) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:00:58.980694 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a09df94-445a-43ba-b7a0-44fb806f252b None admin] Validating token access rules against request {{(pid=80378) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:00:59.031245 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-179a622c-30d4-4975-b2c9-099da2d4b613 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '891005fa5ff74041b75747d16cc66798', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80377) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:00:59.032160 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-179a622c-30d4-4975-b2c9-099da2d4b613 None admin] REQUEST_METHOD: `GET` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:00:59.032462 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-179a622c-30d4-4975-b2c9-099da2d4b613 None admin] SCRIPT_NAME: `/identity` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:00:59.033294 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-179a622c-30d4-4975-b2c9-099da2d4b613 None admin] PATH_INFO: `/v3/projects/c1809c9e380b45ec803130fe38eeaf24` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:00:59.035544 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-179a622c-30d4-4975-b2c9-099da2d4b613 None admin] RBAC: Authorizing `identity:get_project(project_id=c1809c9e380b45ec803130fe38eeaf24)` {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:00:59.037187 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-179a622c-30d4-4975-b2c9-099da2d4b613 None admin] RBAC: Authorization granted {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:00:59.039628 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: [pid: 80377|app: 0|req: 46/92] 10.222.0.82 () {56 vars in 1186 bytes} [Tue Aug 2 10:00:58 2022] GET /identity/v3/projects/c1809c9e380b45ec803130fe38eeaf24 => generated 298 bytes in 294 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 02 10:00:59.051633 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-179a622c-30d4-4975-b2c9-099da2d4b613 None admin] Authenticating user token {{(pid=80377) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:00:59.062119 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95a164df-95eb-418b-bd91-682e8481eccd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '891005fa5ff74041b75747d16cc66798', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80378) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:00:59.063142 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95a164df-95eb-418b-bd91-682e8481eccd None admin] REQUEST_METHOD: `GET` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:00:59.063615 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95a164df-95eb-418b-bd91-682e8481eccd None admin] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:00:59.064092 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95a164df-95eb-418b-bd91-682e8481eccd None admin] PATH_INFO: `/v3/users/71a4a6ffabe74df39f0a7ca7ea7b33b8` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:00:59.065977 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95a164df-95eb-418b-bd91-682e8481eccd None admin] RBAC: Authorizing `identity:get_user(user_id=71a4a6ffabe74df39f0a7ca7ea7b33b8)` {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:00:59.067826 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95a164df-95eb-418b-bd91-682e8481eccd None admin] RBAC: Authorization granted {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:00:59.071087 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 47/93] 10.222.0.82 () {56 vars in 1177 bytes} [Tue Aug 2 10:00:58 2022] GET /identity/v3/users/71a4a6ffabe74df39f0a7ca7ea7b33b8 => generated 275 bytes in 306 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 02 10:00:59.080072 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95a164df-95eb-418b-bd91-682e8481eccd None admin] Authenticating user token {{(pid=80378) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:00:59.144878 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG dbcounter [-] [80377] Writing DB stats keystone:SELECT=100 {{(pid=80377) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 10:00:59.329023 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-179a622c-30d4-4975-b2c9-099da2d4b613 None admin] Validating token access rules against request {{(pid=80377) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:00:59.332931 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG dbcounter [-] [80378] Writing DB stats keystone:SELECT=100 {{(pid=80378) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 10:00:59.350811 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95a164df-95eb-418b-bd91-682e8481eccd None admin] Validating token access rules against request {{(pid=80378) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:00:59.418694 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e58262b2-c249-4211-9fde-416efa1b9453 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '891005fa5ff74041b75747d16cc66798', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80378) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:00:59.419360 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e58262b2-c249-4211-9fde-416efa1b9453 None admin] REQUEST_METHOD: `GET` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:00:59.419777 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e58262b2-c249-4211-9fde-416efa1b9453 None admin] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:00:59.420294 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e58262b2-c249-4211-9fde-416efa1b9453 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:00:59.428196 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac688265-3b90-4c9c-904c-1ad7e7f84e02 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '891005fa5ff74041b75747d16cc66798', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80377) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:00:59.429131 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac688265-3b90-4c9c-904c-1ad7e7f84e02 None admin] REQUEST_METHOD: `GET` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:00:59.429542 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac688265-3b90-4c9c-904c-1ad7e7f84e02 None admin] SCRIPT_NAME: `/identity` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:00:59.429933 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac688265-3b90-4c9c-904c-1ad7e7f84e02 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:00:59.431006 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac688265-3b90-4c9c-904c-1ad7e7f84e02 None admin] RBAC: Adding query filter params (role.id=649993d933d44c50b268df7fdf336fbf, scope.project.id=c1809c9e380b45ec803130fe38eeaf24, user.id=71a4a6ffabe74df39f0a7ca7ea7b33b8) {{(pid=80377) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 02 10:00:59.431402 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac688265-3b90-4c9c-904c-1ad7e7f84e02 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:00:59.433304 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac688265-3b90-4c9c-904c-1ad7e7f84e02 None admin] RBAC: Authorization granted {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:00:59.457811 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: [pid: 80377|app: 0|req: 47/94] 10.222.0.82 () {56 vars in 1374 bytes} [Tue Aug 2 10:00:59 2022] GET /identity/v3/role_assignments?user.id=71a4a6ffabe74df39f0a7ca7ea7b33b8&scope.project.id=c1809c9e380b45ec803130fe38eeaf24&role.id=649993d933d44c50b268df7fdf336fbf => generated 259 bytes in 409 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 02 10:00:59.589603 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e58262b2-c249-4211-9fde-416efa1b9453 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:00:59.591627 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e58262b2-c249-4211-9fde-416efa1b9453 None admin] RBAC: Authorization granted {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:00:59.766986 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG dbcounter [-] [80378] Writing DB stats keystone:SELECT=100 {{(pid=80378) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 10:00:59.781318 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 48/95] 10.222.0.82 () {58 vars in 1282 bytes} [Tue Aug 2 10:00:59 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 702 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 02 10:00:59.791358 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac688265-3b90-4c9c-904c-1ad7e7f84e02 None admin] Authenticating user token {{(pid=80377) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:01:00.020170 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac688265-3b90-4c9c-904c-1ad7e7f84e02 None admin] Validating token access rules against request {{(pid=80377) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:01:00.032283 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG dbcounter [-] [80377] Writing DB stats keystone:SELECT=100 {{(pid=80377) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 10:01:00.109833 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b1745f2-7768-417f-9259-8059d1e9130b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '891005fa5ff74041b75747d16cc66798', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80377) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:01:00.110472 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b1745f2-7768-417f-9259-8059d1e9130b None admin] REQUEST_METHOD: `GET` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:01:00.111345 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b1745f2-7768-417f-9259-8059d1e9130b None admin] SCRIPT_NAME: `/identity` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:01:00.111729 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b1745f2-7768-417f-9259-8059d1e9130b None admin] PATH_INFO: `/v3/projects/354cc945e1c64e598aefaee72be3cffc` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:01:00.113392 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b1745f2-7768-417f-9259-8059d1e9130b None admin] RBAC: Authorizing `identity:get_project(project_id=354cc945e1c64e598aefaee72be3cffc)` {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:01:00.114952 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b1745f2-7768-417f-9259-8059d1e9130b None admin] RBAC: Authorization granted {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:01:00.117116 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: [pid: 80377|app: 0|req: 48/96] 10.222.0.82 () {56 vars in 1186 bytes} [Tue Aug 2 10:00:59 2022] GET /identity/v3/projects/354cc945e1c64e598aefaee72be3cffc => generated 312 bytes in 327 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 02 10:01:00.125693 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e58262b2-c249-4211-9fde-416efa1b9453 None admin] Authenticating user token {{(pid=80378) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:01:00.323842 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e58262b2-c249-4211-9fde-416efa1b9453 None admin] Validating token access rules against request {{(pid=80378) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:01:00.388168 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb081bca-4357-4370-82b0-ebabc55676ff None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '891005fa5ff74041b75747d16cc66798', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80378) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:01:00.388732 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb081bca-4357-4370-82b0-ebabc55676ff None admin] REQUEST_METHOD: `GET` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:01:00.389019 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb081bca-4357-4370-82b0-ebabc55676ff None admin] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:01:00.389295 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb081bca-4357-4370-82b0-ebabc55676ff None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:01:00.390129 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb081bca-4357-4370-82b0-ebabc55676ff None admin] RBAC: Adding query filter params (role.id=e1c04b19fc4543b8b54f58bbcffbc598, scope.project.id=354cc945e1c64e598aefaee72be3cffc, user.id=71a4a6ffabe74df39f0a7ca7ea7b33b8) {{(pid=80378) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 02 10:01:00.390443 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb081bca-4357-4370-82b0-ebabc55676ff None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:01:00.391978 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb081bca-4357-4370-82b0-ebabc55676ff None admin] RBAC: Authorization granted {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:01:00.402593 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 49/97] 10.222.0.82 () {56 vars in 1374 bytes} [Tue Aug 2 10:01:00 2022] GET /identity/v3/role_assignments?user.id=71a4a6ffabe74df39f0a7ca7ea7b33b8&scope.project.id=354cc945e1c64e598aefaee72be3cffc&role.id=e1c04b19fc4543b8b54f58bbcffbc598 => generated 259 bytes in 277 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 02 10:01:02.291943 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-002432d7-750d-47d2-a440-614868f41d68 None None] REQUEST_METHOD: `GET` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:01:02.293003 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-002432d7-750d-47d2-a440-614868f41d68 None None] SCRIPT_NAME: `/identity` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:01:02.293587 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-002432d7-750d-47d2-a440-614868f41d68 None None] PATH_INFO: `/` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:01:02.295484 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: [pid: 80377|app: 0|req: 49/98] 10.222.0.82 () {54 vars in 926 bytes} [Tue Aug 2 10:01:02 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 02 10:01:02.302273 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd500623-26a8-4e63-b759-0e24bda01c3f None None] REQUEST_METHOD: `POST` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:01:02.303293 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd500623-26a8-4e63-b759-0e24bda01c3f None None] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:01:02.303799 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd500623-26a8-4e63-b759-0e24bda01c3f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:01:02.313347 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-816a4baf-010a-4b82-b53c-c094cff174ac None None] REQUEST_METHOD: `GET` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:01:02.314394 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-816a4baf-010a-4b82-b53c-c094cff174ac None None] SCRIPT_NAME: `/identity` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:01:02.314889 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-816a4baf-010a-4b82-b53c-c094cff174ac None None] PATH_INFO: `/` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:01:02.316434 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: [pid: 80377|app: 0|req: 50/99] 10.222.0.82 () {54 vars in 926 bytes} [Tue Aug 2 10:01:02 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 02 10:01:02.323153 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd209caa-2b34-4401-90f7-76d87eaecb93 None None] REQUEST_METHOD: `POST` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:01:02.324009 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd209caa-2b34-4401-90f7-76d87eaecb93 None None] SCRIPT_NAME: `/identity` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:01:02.324730 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd209caa-2b34-4401-90f7-76d87eaecb93 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:01:02.342104 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.auth.core [None req-dd500623-26a8-4e63-b759-0e24bda01c3f None None] MFA Rules not processed for user `891005fa5ff74041b75747d16cc66798`. Rule list: `[]` (Enabled: `True`). {{(pid=80378) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 02 10:01:02.346699 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.fernet_utils [None req-dd500623-26a8-4e63-b759-0e24bda01c3f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80378) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 10:01:02.360344 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.auth.core [None req-dd209caa-2b34-4401-90f7-76d87eaecb93 None None] MFA Rules not processed for user `891005fa5ff74041b75747d16cc66798`. Rule list: `[]` (Enabled: `True`). {{(pid=80377) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 02 10:01:02.364114 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.fernet_utils [None req-dd209caa-2b34-4401-90f7-76d87eaecb93 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80377) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 10:01:02.451114 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG dbcounter [-] [80378] Writing DB stats keystone:SELECT=100 {{(pid=80378) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 10:01:02.483550 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 50/100] 10.222.0.82 () {58 vars in 1023 bytes} [Tue Aug 2 10:01:02 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 182 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 02 10:01:02.499296 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: [pid: 80377|app: 0|req: 51/101] 10.222.0.82 () {58 vars in 1023 bytes} [Tue Aug 2 10:01:02 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 177 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 02 10:01:02.520231 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd500623-26a8-4e63-b759-0e24bda01c3f None None] Authenticating user token {{(pid=80378) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:01:02.540364 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd209caa-2b34-4401-90f7-76d87eaecb93 None None] Authenticating user token {{(pid=80377) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:01:02.753518 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd500623-26a8-4e63-b759-0e24bda01c3f None None] Validating token access rules against request {{(pid=80378) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:01:02.761323 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd209caa-2b34-4401-90f7-76d87eaecb93 None None] Validating token access rules against request {{(pid=80377) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:01:02.769991 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG dbcounter [-] [80377] Writing DB stats keystone:SELECT=100 {{(pid=80377) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 10:01:02.800154 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91a64ce4-8f8e-4b48-ab0a-75d9d23a4007 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '891005fa5ff74041b75747d16cc66798', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80378) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:01:02.801163 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91a64ce4-8f8e-4b48-ab0a-75d9d23a4007 None admin] REQUEST_METHOD: `GET` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:01:02.801679 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91a64ce4-8f8e-4b48-ab0a-75d9d23a4007 None admin] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:01:02.802183 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91a64ce4-8f8e-4b48-ab0a-75d9d23a4007 None admin] PATH_INFO: `/v3/roles/reader` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:01:02.813644 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91a64ce4-8f8e-4b48-ab0a-75d9d23a4007 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 02 10:01:02.819696 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application [None req-91a64ce4-8f8e-4b48-ab0a-75d9d23a4007 None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Aug 02 10:01:02.819696 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 02 10:01:02.819696 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 02 10:01:02.819696 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 02 10:01:02.819696 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 02 10:01:02.819696 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 02 10:01:02.819696 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 02 10:01:02.819696 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 02 10:01:02.819696 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 02 10:01:02.819696 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 02 10:01:02.819696 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 02 10:01:02.819696 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 02 10:01:02.819696 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 02 10:01:02.819696 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 02 10:01:02.819696 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 02 10:01:02.819696 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 02 10:01:02.820867 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 02 10:01:02.820867 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 02 10:01:02.820867 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 02 10:01:02.820867 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 02 10:01:02.820867 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 02 10:01:02.820867 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 02 10:01:02.820867 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 02 10:01:02.820867 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 02 10:01:02.820867 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 02 10:01:02.820867 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application return self.get_or_create( Aug 02 10:01:02.820867 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 02 10:01:02.820867 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application with Lock( Aug 02 10:01:02.820867 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 02 10:01:02.820867 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application return self._enter() Aug 02 10:01:02.820867 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 02 10:01:02.820867 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 02 10:01:02.821751 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 02 10:01:02.821751 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application return self.creator() Aug 02 10:01:02.821751 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 02 10:01:02.821751 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application created_value = creator( Aug 02 10:01:02.821751 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 02 10:01:02.821751 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 02 10:01:02.821751 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 02 10:01:02.821751 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 02 10:01:02.821751 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 02 10:01:02.821751 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 02 10:01:02.821751 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Aug 02 10:01:02.821751 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application Aug 02 10:01:02.824499 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 51/102] 10.222.0.82 () {56 vars in 1099 bytes} [Tue Aug 2 10:01:02 2022] GET /identity/v3/roles/reader => generated 84 bytes in 305 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 02 10:01:02.827456 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2997eaf6-f581-4781-a939-1e85f5979f5b None None] REQUEST_METHOD: `GET` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:01:02.828133 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2997eaf6-f581-4781-a939-1e85f5979f5b None None] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:01:02.828699 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2997eaf6-f581-4781-a939-1e85f5979f5b None None] PATH_INFO: `/` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:01:02.830647 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d2a6e62-daa2-4ddd-b953-360d996e1280 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '891005fa5ff74041b75747d16cc66798', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80377) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:01:02.831372 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 52/103] 10.222.0.82 () {54 vars in 926 bytes} [Tue Aug 2 10:01:02 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 02 10:01:02.831568 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d2a6e62-daa2-4ddd-b953-360d996e1280 None admin] REQUEST_METHOD: `GET` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:01:02.831877 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d2a6e62-daa2-4ddd-b953-360d996e1280 None admin] SCRIPT_NAME: `/identity` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:01:02.832184 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d2a6e62-daa2-4ddd-b953-360d996e1280 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:01:02.833451 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1b306a1-8be4-4177-86a6-6acab0a89472 None None] REQUEST_METHOD: `GET` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:01:02.833994 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1b306a1-8be4-4177-86a6-6acab0a89472 None None] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:01:02.834491 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1b306a1-8be4-4177-86a6-6acab0a89472 None None] PATH_INFO: `/` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:01:02.836063 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 53/104] 10.222.0.82 () {54 vars in 926 bytes} [Tue Aug 2 10:01:02 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 02 10:01:02.837032 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1b306a1-8be4-4177-86a6-6acab0a89472 None None] Authenticating user token {{(pid=80378) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:01:02.855303 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d2a6e62-daa2-4ddd-b953-360d996e1280 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 02 10:01:02.866495 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application [None req-9d2a6e62-daa2-4ddd-b953-360d996e1280 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 02 10:01:02.866495 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 02 10:01:02.866495 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 02 10:01:02.866495 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 02 10:01:02.866495 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 02 10:01:02.866495 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 02 10:01:02.866495 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 02 10:01:02.866495 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 02 10:01:02.866495 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 02 10:01:02.866495 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 02 10:01:02.866495 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 02 10:01:02.866495 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 02 10:01:02.866495 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 02 10:01:02.866495 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 02 10:01:02.866495 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 02 10:01:02.866495 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 02 10:01:02.868194 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 02 10:01:02.868194 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 02 10:01:02.868194 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 02 10:01:02.868194 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 02 10:01:02.868194 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 02 10:01:02.868194 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 02 10:01:02.868194 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 02 10:01:02.868194 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 02 10:01:02.868194 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 02 10:01:02.868194 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application return self.get_or_create( Aug 02 10:01:02.868194 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 02 10:01:02.868194 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application with Lock( Aug 02 10:01:02.868194 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 02 10:01:02.868194 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application return self._enter() Aug 02 10:01:02.868194 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 02 10:01:02.868194 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 02 10:01:02.869705 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 02 10:01:02.869705 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application return self.creator() Aug 02 10:01:02.869705 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 02 10:01:02.869705 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application created_value = creator( Aug 02 10:01:02.869705 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 02 10:01:02.869705 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 02 10:01:02.869705 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 02 10:01:02.869705 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 02 10:01:02.869705 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 02 10:01:02.869705 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 02 10:01:02.869705 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 02 10:01:02.869705 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application Aug 02 10:01:02.874202 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: [pid: 80377|app: 0|req: 52/105] 10.222.0.82 () {56 vars in 1096 bytes} [Tue Aug 2 10:01:02 2022] GET /identity/v3/roles/admin => generated 83 bytes in 336 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 02 10:01:02.877752 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a038d3e8-4b61-407a-afc3-b23b6a9759b6 None None] REQUEST_METHOD: `POST` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:01:02.878646 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a038d3e8-4b61-407a-afc3-b23b6a9759b6 None None] SCRIPT_NAME: `/identity` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:01:02.879360 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a038d3e8-4b61-407a-afc3-b23b6a9759b6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:01:02.898490 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG dbcounter [-] [80378] Writing DB stats keystone:SELECT=100 {{(pid=80378) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 10:01:02.916314 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.auth.core [None req-a038d3e8-4b61-407a-afc3-b23b6a9759b6 None None] MFA Rules not processed for user `891005fa5ff74041b75747d16cc66798`. Rule list: `[]` (Enabled: `True`). {{(pid=80377) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 02 10:01:02.920037 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.fernet_utils [None req-a038d3e8-4b61-407a-afc3-b23b6a9759b6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80377) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 10:01:02.998327 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1b306a1-8be4-4177-86a6-6acab0a89472 None None] Validating token access rules against request {{(pid=80378) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:01:03.037921 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: [pid: 80377|app: 0|req: 53/106] 10.222.0.82 () {58 vars in 1023 bytes} [Tue Aug 2 10:01:02 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 157 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 02 10:01:03.037921 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcef0581-258a-429d-b59d-b8d044c74cf2 None None] REQUEST_METHOD: `POST` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:01:03.037921 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcef0581-258a-429d-b59d-b8d044c74cf2 None None] SCRIPT_NAME: `/identity` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:01:03.037921 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcef0581-258a-429d-b59d-b8d044c74cf2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:01:03.069397 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.auth.core [None req-bcef0581-258a-429d-b59d-b8d044c74cf2 None None] MFA Rules not processed for user `891005fa5ff74041b75747d16cc66798`. Rule list: `[]` (Enabled: `True`). {{(pid=80377) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 02 10:01:03.070636 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.fernet_utils [None req-bcef0581-258a-429d-b59d-b8d044c74cf2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80377) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 10:01:03.098466 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c52945b7-f5ff-49d0-926f-ab846189fd00 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '891005fa5ff74041b75747d16cc66798', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80378) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:01:03.101072 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c52945b7-f5ff-49d0-926f-ab846189fd00 None admin] REQUEST_METHOD: `GET` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:01:03.101903 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c52945b7-f5ff-49d0-926f-ab846189fd00 None admin] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:01:03.102957 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c52945b7-f5ff-49d0-926f-ab846189fd00 None admin] PATH_INFO: `/v3/roles` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:01:03.105046 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c52945b7-f5ff-49d0-926f-ab846189fd00 None admin] RBAC: Adding query filter params (name=reader) {{(pid=80378) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 02 10:01:03.105934 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c52945b7-f5ff-49d0-926f-ab846189fd00 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:01:03.110326 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c52945b7-f5ff-49d0-926f-ab846189fd00 None admin] RBAC: Authorization granted {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:01:03.129001 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 54/107] 10.222.0.82 () {56 vars in 1101 bytes} [Tue Aug 2 10:01:02 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 292 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 02 10:01:03.130634 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c52945b7-f5ff-49d0-926f-ab846189fd00 None admin] Authenticating user token {{(pid=80378) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:01:03.181976 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: [pid: 80377|app: 0|req: 54/108] 10.222.0.82 () {58 vars in 1023 bytes} [Tue Aug 2 10:01:03 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 149 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 02 10:01:03.183470 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bcef0581-258a-429d-b59d-b8d044c74cf2 None None] Authenticating user token {{(pid=80377) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:01:03.214327 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG dbcounter [-] [80377] Writing DB stats keystone:SELECT=100 {{(pid=80377) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 10:01:03.285363 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c52945b7-f5ff-49d0-926f-ab846189fd00 None admin] Validating token access rules against request {{(pid=80378) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:01:03.295781 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG dbcounter [-] [80378] Writing DB stats keystone:SELECT=100 {{(pid=80378) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 10:01:03.307923 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bcef0581-258a-429d-b59d-b8d044c74cf2 None None] Validating token access rules against request {{(pid=80377) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:01:03.342276 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37d29d1d-04e0-4e68-af06-431e939dafca None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '891005fa5ff74041b75747d16cc66798', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80378) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:01:03.343089 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37d29d1d-04e0-4e68-af06-431e939dafca None admin] REQUEST_METHOD: `GET` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:01:03.343217 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37d29d1d-04e0-4e68-af06-431e939dafca None admin] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:01:03.343502 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37d29d1d-04e0-4e68-af06-431e939dafca None admin] PATH_INFO: `/v3/roles` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:01:03.344327 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37d29d1d-04e0-4e68-af06-431e939dafca None admin] RBAC: Adding query filter params (name=admin) {{(pid=80378) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 02 10:01:03.344640 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37d29d1d-04e0-4e68-af06-431e939dafca None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:01:03.346444 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37d29d1d-04e0-4e68-af06-431e939dafca None admin] RBAC: Authorization granted {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:01:03.359738 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 55/109] 10.222.0.82 () {56 vars in 1099 bytes} [Tue Aug 2 10:01:03 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 229 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 02 10:01:03.361030 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44d855f5-6f9a-4221-9f2a-d3931793d21a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '891005fa5ff74041b75747d16cc66798', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80377) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:01:03.361425 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37d29d1d-04e0-4e68-af06-431e939dafca None admin] Authenticating user token {{(pid=80378) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:01:03.362763 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44d855f5-6f9a-4221-9f2a-d3931793d21a None admin] REQUEST_METHOD: `GET` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:01:03.363364 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44d855f5-6f9a-4221-9f2a-d3931793d21a None admin] SCRIPT_NAME: `/identity` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:01:03.363941 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44d855f5-6f9a-4221-9f2a-d3931793d21a None admin] PATH_INFO: `/v3/domains/default` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:01:03.366464 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44d855f5-6f9a-4221-9f2a-d3931793d21a None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:01:03.368856 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44d855f5-6f9a-4221-9f2a-d3931793d21a None admin] RBAC: Authorization granted {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:01:03.371983 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: [pid: 80377|app: 0|req: 55/110] 10.222.0.82 () {56 vars in 1108 bytes} [Tue Aug 2 10:01:03 2022] GET /identity/v3/domains/default => generated 199 bytes in 188 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 02 10:01:03.372458 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44d855f5-6f9a-4221-9f2a-d3931793d21a None admin] Authenticating user token {{(pid=80377) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:01:03.508827 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG dbcounter [-] [80377] Writing DB stats keystone:SELECT=100 {{(pid=80377) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 10:01:03.528728 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44d855f5-6f9a-4221-9f2a-d3931793d21a None admin] Validating token access rules against request {{(pid=80377) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:01:03.539114 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37d29d1d-04e0-4e68-af06-431e939dafca None admin] Validating token access rules against request {{(pid=80378) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:01:03.573730 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e114105c-6511-414a-ab22-ee5479dd9d49 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '891005fa5ff74041b75747d16cc66798', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80377) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:01:03.574507 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e114105c-6511-414a-ab22-ee5479dd9d49 None admin] REQUEST_METHOD: `GET` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:01:03.574857 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e114105c-6511-414a-ab22-ee5479dd9d49 None admin] SCRIPT_NAME: `/identity` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:01:03.575224 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e114105c-6511-414a-ab22-ee5479dd9d49 None admin] PATH_INFO: `/v3/roles/member` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:01:03.597818 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e114105c-6511-414a-ab22-ee5479dd9d49 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 02 10:01:03.601235 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bddb5b47-c8e1-49d8-8eca-94b6690cbf20 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '891005fa5ff74041b75747d16cc66798', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80378) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:01:03.601960 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bddb5b47-c8e1-49d8-8eca-94b6690cbf20 None admin] REQUEST_METHOD: `GET` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:01:03.602317 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bddb5b47-c8e1-49d8-8eca-94b6690cbf20 None admin] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:01:03.602709 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bddb5b47-c8e1-49d8-8eca-94b6690cbf20 None admin] PATH_INFO: `/v3/users/1c51a65536174654945f89334229c254` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:01:03.608268 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application [None req-e114105c-6511-414a-ab22-ee5479dd9d49 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Aug 02 10:01:03.608268 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 02 10:01:03.608268 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 02 10:01:03.608268 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 02 10:01:03.608268 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 02 10:01:03.608268 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 02 10:01:03.608268 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 02 10:01:03.608268 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 02 10:01:03.608268 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 02 10:01:03.608268 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 02 10:01:03.608268 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 02 10:01:03.608268 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 02 10:01:03.608268 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 02 10:01:03.608268 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 02 10:01:03.608268 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 02 10:01:03.608268 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 02 10:01:03.609775 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 02 10:01:03.609775 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 02 10:01:03.609775 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 02 10:01:03.609775 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 02 10:01:03.609775 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 02 10:01:03.609775 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 02 10:01:03.609775 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 02 10:01:03.609775 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 02 10:01:03.609775 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 02 10:01:03.609775 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application return self.get_or_create( Aug 02 10:01:03.609775 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 02 10:01:03.609775 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application with Lock( Aug 02 10:01:03.609775 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 02 10:01:03.609775 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application return self._enter() Aug 02 10:01:03.609775 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 02 10:01:03.609775 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 02 10:01:03.612188 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 02 10:01:03.612188 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application return self.creator() Aug 02 10:01:03.612188 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 02 10:01:03.612188 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application created_value = creator( Aug 02 10:01:03.612188 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 02 10:01:03.612188 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 02 10:01:03.612188 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 02 10:01:03.612188 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 02 10:01:03.612188 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 02 10:01:03.612188 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 02 10:01:03.612188 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Aug 02 10:01:03.612188 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application Aug 02 10:01:03.612188 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: [pid: 80377|app: 0|req: 56/111] 10.222.0.82 () {56 vars in 1099 bytes} [Tue Aug 2 10:01:03 2022] GET /identity/v3/roles/member => generated 84 bytes in 239 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 02 10:01:03.616268 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe1c648f-0b0f-4819-b25f-f65cea6362ce None None] REQUEST_METHOD: `GET` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:01:03.616268 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe1c648f-0b0f-4819-b25f-f65cea6362ce None None] SCRIPT_NAME: `/identity` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:01:03.616268 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe1c648f-0b0f-4819-b25f-f65cea6362ce None None] PATH_INFO: `/` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:01:03.621830 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: [pid: 80377|app: 0|req: 57/112] 10.222.0.82 () {54 vars in 926 bytes} [Tue Aug 2 10:01:03 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 02 10:01:03.621830 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe1c648f-0b0f-4819-b25f-f65cea6362ce None None] Authenticating user token {{(pid=80377) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:01:03.629587 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bddb5b47-c8e1-49d8-8eca-94b6690cbf20 None admin] RBAC: Authorizing `identity:get_user(user_id=1c51a65536174654945f89334229c254)` {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:01:03.631159 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bddb5b47-c8e1-49d8-8eca-94b6690cbf20 None admin] RBAC: Authorization granted {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:01:03.632949 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 56/113] 10.222.0.82 () {56 vars in 1177 bytes} [Tue Aug 2 10:01:03 2022] GET /identity/v3/users/1c51a65536174654945f89334229c254 => generated 289 bytes in 273 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 02 10:01:03.633958 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bddb5b47-c8e1-49d8-8eca-94b6690cbf20 None admin] Authenticating user token {{(pid=80378) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:01:03.672596 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG dbcounter [-] [80378] Writing DB stats keystone:SELECT=100 {{(pid=80378) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 10:01:03.801168 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bddb5b47-c8e1-49d8-8eca-94b6690cbf20 None admin] Validating token access rules against request {{(pid=80378) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:01:03.811760 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe1c648f-0b0f-4819-b25f-f65cea6362ce None None] Validating token access rules against request {{(pid=80377) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:01:03.861477 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a591bd67-0520-4a0b-aa04-36d32d0481b4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '891005fa5ff74041b75747d16cc66798', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80378) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:01:03.862027 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a591bd67-0520-4a0b-aa04-36d32d0481b4 None admin] REQUEST_METHOD: `POST` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:01:03.862311 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a591bd67-0520-4a0b-aa04-36d32d0481b4 None admin] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:01:03.862586 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a591bd67-0520-4a0b-aa04-36d32d0481b4 None admin] PATH_INFO: `/v3/projects` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:01:03.863328 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a591bd67-0520-4a0b-aa04-36d32d0481b4 None admin] RBAC: Authorizing `identity:create_project()` {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:01:03.864828 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a591bd67-0520-4a0b-aa04-36d32d0481b4 None admin] RBAC: Authorization granted {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:01:03.867797 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff0eb95f-9e4c-4400-9ce4-8b54bfcb7bdb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '891005fa5ff74041b75747d16cc66798', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80377) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:01:03.868854 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff0eb95f-9e4c-4400-9ce4-8b54bfcb7bdb None admin] REQUEST_METHOD: `GET` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:01:03.869375 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff0eb95f-9e4c-4400-9ce4-8b54bfcb7bdb None admin] SCRIPT_NAME: `/identity` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:01:03.869910 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff0eb95f-9e4c-4400-9ce4-8b54bfcb7bdb None admin] PATH_INFO: `/v3/users/891005fa5ff74041b75747d16cc66798` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:01:03.872682 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff0eb95f-9e4c-4400-9ce4-8b54bfcb7bdb None admin] RBAC: Authorizing `identity:get_user(user_id=891005fa5ff74041b75747d16cc66798)` {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:01:03.875452 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff0eb95f-9e4c-4400-9ce4-8b54bfcb7bdb None admin] RBAC: Authorization granted {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:01:03.883931 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: [pid: 80377|app: 0|req: 58/114] 10.222.0.82 () {56 vars in 1177 bytes} [Tue Aug 2 10:01:03 2022] GET /identity/v3/users/891005fa5ff74041b75747d16cc66798 => generated 247 bytes in 260 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 02 10:01:03.883931 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff0eb95f-9e4c-4400-9ce4-8b54bfcb7bdb None admin] Authenticating user token {{(pid=80377) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:01:03.901320 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 57/115] 10.222.0.82 () {60 vars in 1141 bytes} [Tue Aug 2 10:01:03 2022] POST /identity/v3/projects => generated 302 bytes in 268 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 02 10:01:03.902827 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81013521-aa25-4a36-8945-0e1750ea0c36 None None] REQUEST_METHOD: `POST` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:01:03.903122 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81013521-aa25-4a36-8945-0e1750ea0c36 None None] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:01:03.903430 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81013521-aa25-4a36-8945-0e1750ea0c36 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:01:03.916225 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG dbcounter [-] [80377] Writing DB stats keystone:SELECT=100 {{(pid=80377) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 10:01:03.941526 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.auth.core [None req-81013521-aa25-4a36-8945-0e1750ea0c36 None None] MFA Rules not processed for user `891005fa5ff74041b75747d16cc66798`. Rule list: `[]` (Enabled: `True`). {{(pid=80378) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 02 10:01:03.942894 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.fernet_utils [None req-81013521-aa25-4a36-8945-0e1750ea0c36 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80378) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 10:01:04.069662 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 58/116] 10.222.0.82 () {58 vars in 1023 bytes} [Tue Aug 2 10:01:03 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 168 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 02 10:01:04.071495 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81013521-aa25-4a36-8945-0e1750ea0c36 None None] Authenticating user token {{(pid=80378) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:01:04.080349 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff0eb95f-9e4c-4400-9ce4-8b54bfcb7bdb None admin] Validating token access rules against request {{(pid=80377) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:01:04.094627 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG dbcounter [-] [80378] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80378) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 10:01:04.149674 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6441efb1-06f8-4d50-91b3-97de6127a63c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '891005fa5ff74041b75747d16cc66798', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80377) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:01:04.150507 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6441efb1-06f8-4d50-91b3-97de6127a63c None admin] REQUEST_METHOD: `GET` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:01:04.151022 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6441efb1-06f8-4d50-91b3-97de6127a63c None admin] SCRIPT_NAME: `/identity` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:01:04.151521 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6441efb1-06f8-4d50-91b3-97de6127a63c None admin] PATH_INFO: `/v3/roles` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:01:04.153709 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6441efb1-06f8-4d50-91b3-97de6127a63c None admin] RBAC: Adding query filter params (name=member) {{(pid=80377) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 02 10:01:04.154325 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6441efb1-06f8-4d50-91b3-97de6127a63c None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:01:04.158371 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6441efb1-06f8-4d50-91b3-97de6127a63c None admin] RBAC: Authorization granted {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:01:04.174714 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: [pid: 80377|app: 0|req: 59/117] 10.222.0.82 () {56 vars in 1101 bytes} [Tue Aug 2 10:01:03 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 296 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 02 10:01:04.176551 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13713fba-c466-4b67-bf91-ef0605ac19c8 None None] REQUEST_METHOD: `GET` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:01:04.176948 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13713fba-c466-4b67-bf91-ef0605ac19c8 None None] SCRIPT_NAME: `/identity` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:01:04.177383 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13713fba-c466-4b67-bf91-ef0605ac19c8 None None] PATH_INFO: `/` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:01:04.179942 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: [pid: 80377|app: 0|req: 60/118] 10.222.0.82 () {54 vars in 926 bytes} [Tue Aug 2 10:01:04 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 02 10:01:04.181288 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13713fba-c466-4b67-bf91-ef0605ac19c8 None None] Authenticating user token {{(pid=80377) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:01:04.259137 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81013521-aa25-4a36-8945-0e1750ea0c36 None None] Validating token access rules against request {{(pid=80378) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:01:04.318766 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-793e1182-030e-40bd-ad4f-bac6dfa714f4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '891005fa5ff74041b75747d16cc66798', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80378) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:01:04.320298 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-793e1182-030e-40bd-ad4f-bac6dfa714f4 None admin] REQUEST_METHOD: `GET` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:01:04.320791 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-793e1182-030e-40bd-ad4f-bac6dfa714f4 None admin] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:01:04.321286 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-793e1182-030e-40bd-ad4f-bac6dfa714f4 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:01:04.325435 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG dbcounter [-] [80377] Writing DB stats keystone:SELECT=100 {{(pid=80377) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 10:01:04.353197 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13713fba-c466-4b67-bf91-ef0605ac19c8 None None] Validating token access rules against request {{(pid=80377) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:01:04.416242 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5147c24-25c3-4942-9e86-3d2cc4282765 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '891005fa5ff74041b75747d16cc66798', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80377) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:01:04.417239 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5147c24-25c3-4942-9e86-3d2cc4282765 None admin] REQUEST_METHOD: `GET` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:01:04.417723 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5147c24-25c3-4942-9e86-3d2cc4282765 None admin] SCRIPT_NAME: `/identity` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:01:04.417847 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-793e1182-030e-40bd-ad4f-bac6dfa714f4 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:01:04.418265 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5147c24-25c3-4942-9e86-3d2cc4282765 None admin] PATH_INFO: `/v3/projects/c1809c9e380b45ec803130fe38eeaf24` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:01:04.419279 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-793e1182-030e-40bd-ad4f-bac6dfa714f4 None admin] RBAC: Authorization granted {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:01:04.420310 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5147c24-25c3-4942-9e86-3d2cc4282765 None admin] RBAC: Authorizing `identity:get_project(project_id=c1809c9e380b45ec803130fe38eeaf24)` {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:01:04.423106 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5147c24-25c3-4942-9e86-3d2cc4282765 None admin] RBAC: Authorization granted {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:01:04.435789 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: [pid: 80377|app: 0|req: 61/119] 10.222.0.82 () {56 vars in 1186 bytes} [Tue Aug 2 10:01:04 2022] GET /identity/v3/projects/c1809c9e380b45ec803130fe38eeaf24 => generated 298 bytes in 247 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 02 10:01:04.435789 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5147c24-25c3-4942-9e86-3d2cc4282765 None admin] Authenticating user token {{(pid=80377) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:01:04.450778 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG dbcounter [-] [80378] Writing DB stats keystone:SELECT=100 {{(pid=80378) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 10:01:04.605710 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 59/120] 10.222.0.82 () {58 vars in 1282 bytes} [Tue Aug 2 10:01:04 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 535 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 02 10:01:04.611853 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-793e1182-030e-40bd-ad4f-bac6dfa714f4 None admin] Authenticating user token {{(pid=80378) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:01:04.628978 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5147c24-25c3-4942-9e86-3d2cc4282765 None admin] Validating token access rules against request {{(pid=80377) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:01:04.688081 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03aae6b4-0575-4dd4-81c4-608705a3d6b1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '891005fa5ff74041b75747d16cc66798', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80377) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:01:04.689196 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03aae6b4-0575-4dd4-81c4-608705a3d6b1 None admin] REQUEST_METHOD: `GET` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:01:04.689597 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03aae6b4-0575-4dd4-81c4-608705a3d6b1 None admin] SCRIPT_NAME: `/identity` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:01:04.690099 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03aae6b4-0575-4dd4-81c4-608705a3d6b1 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:01:04.707501 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03aae6b4-0575-4dd4-81c4-608705a3d6b1 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 02 10:01:04.718604 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application [None req-03aae6b4-0575-4dd4-81c4-608705a3d6b1 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Aug 02 10:01:04.718604 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 02 10:01:04.718604 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 02 10:01:04.718604 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 02 10:01:04.718604 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 02 10:01:04.718604 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 02 10:01:04.718604 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 02 10:01:04.718604 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 02 10:01:04.718604 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 02 10:01:04.718604 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 02 10:01:04.718604 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 02 10:01:04.718604 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 02 10:01:04.718604 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 02 10:01:04.718604 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 02 10:01:04.718604 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 02 10:01:04.718604 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 02 10:01:04.720685 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 02 10:01:04.720685 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 02 10:01:04.720685 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 02 10:01:04.720685 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 02 10:01:04.720685 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 02 10:01:04.720685 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 02 10:01:04.720685 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 02 10:01:04.720685 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 02 10:01:04.720685 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 02 10:01:04.720685 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application return self.get_or_create( Aug 02 10:01:04.720685 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 02 10:01:04.720685 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application with Lock( Aug 02 10:01:04.720685 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 02 10:01:04.720685 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application return self._enter() Aug 02 10:01:04.720685 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 02 10:01:04.720685 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 02 10:01:04.722122 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 02 10:01:04.722122 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application return self.creator() Aug 02 10:01:04.722122 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 02 10:01:04.722122 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application created_value = creator( Aug 02 10:01:04.722122 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 02 10:01:04.722122 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 02 10:01:04.722122 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 02 10:01:04.722122 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 02 10:01:04.722122 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 02 10:01:04.722122 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 02 10:01:04.722122 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Aug 02 10:01:04.722122 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application Aug 02 10:01:04.726189 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: [pid: 80377|app: 0|req: 62/121] 10.222.0.82 () {56 vars in 1114 bytes} [Tue Aug 2 10:01:04 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 298 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 02 10:01:04.728129 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63afd335-072c-490a-92f9-d3873203b380 None None] REQUEST_METHOD: `POST` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:01:04.728425 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63afd335-072c-490a-92f9-d3873203b380 None None] SCRIPT_NAME: `/identity` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:01:04.728708 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63afd335-072c-490a-92f9-d3873203b380 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:01:04.754406 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.auth.core [None req-63afd335-072c-490a-92f9-d3873203b380 None None] MFA Rules not processed for user `891005fa5ff74041b75747d16cc66798`. Rule list: `[]` (Enabled: `True`). {{(pid=80377) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 02 10:01:04.755760 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.fernet_utils [None req-63afd335-072c-490a-92f9-d3873203b380 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80377) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 10:01:04.760065 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG dbcounter [-] [80377] Writing DB stats keystone:SELECT=100 {{(pid=80377) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 10:01:04.768299 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-793e1182-030e-40bd-ad4f-bac6dfa714f4 None admin] Validating token access rules against request {{(pid=80378) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:01:04.787366 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG dbcounter [-] [80378] Writing DB stats keystone:SELECT=100 {{(pid=80378) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 10:01:04.847028 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d1e1def-d02a-400a-954a-9fd0bf66741e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '891005fa5ff74041b75747d16cc66798', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80378) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:01:04.848671 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d1e1def-d02a-400a-954a-9fd0bf66741e None admin] REQUEST_METHOD: `GET` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:01:04.849101 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d1e1def-d02a-400a-954a-9fd0bf66741e None admin] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:01:04.849519 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d1e1def-d02a-400a-954a-9fd0bf66741e None admin] PATH_INFO: `/v3/users/71a4a6ffabe74df39f0a7ca7ea7b33b8` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:01:04.851402 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d1e1def-d02a-400a-954a-9fd0bf66741e None admin] RBAC: Authorizing `identity:get_user(user_id=71a4a6ffabe74df39f0a7ca7ea7b33b8)` {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:01:04.853312 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d1e1def-d02a-400a-954a-9fd0bf66741e None admin] RBAC: Authorization granted {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:01:04.856177 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 60/122] 10.222.0.82 () {56 vars in 1177 bytes} [Tue Aug 2 10:01:04 2022] GET /identity/v3/users/71a4a6ffabe74df39f0a7ca7ea7b33b8 => generated 275 bytes in 250 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 02 10:01:04.857340 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d1e1def-d02a-400a-954a-9fd0bf66741e None admin] Authenticating user token {{(pid=80378) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:01:04.886561 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: [pid: 80377|app: 0|req: 63/123] 10.222.0.82 () {58 vars in 1023 bytes} [Tue Aug 2 10:01:04 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 159 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 02 10:01:04.887718 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63afd335-072c-490a-92f9-d3873203b380 None None] Authenticating user token {{(pid=80377) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:01:05.038790 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d1e1def-d02a-400a-954a-9fd0bf66741e None admin] Validating token access rules against request {{(pid=80378) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:01:05.061929 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63afd335-072c-490a-92f9-d3873203b380 None None] Validating token access rules against request {{(pid=80377) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:01:05.090501 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22a77407-561a-417a-aae1-0f6fe98dd6b7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '891005fa5ff74041b75747d16cc66798', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80378) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:01:05.092049 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22a77407-561a-417a-aae1-0f6fe98dd6b7 None admin] REQUEST_METHOD: `PUT` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:01:05.092818 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22a77407-561a-417a-aae1-0f6fe98dd6b7 None admin] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:01:05.093328 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22a77407-561a-417a-aae1-0f6fe98dd6b7 None admin] PATH_INFO: `/v3/projects/c1809c9e380b45ec803130fe38eeaf24/users/891005fa5ff74041b75747d16cc66798/roles/237966442f05411191105bf336bc14f3` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:01:05.096776 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22a77407-561a-417a-aae1-0f6fe98dd6b7 None admin] RBAC: Authorizing `identity:create_grant(project_id=c1809c9e380b45ec803130fe38eeaf24, user_id=891005fa5ff74041b75747d16cc66798, role_id=237966442f05411191105bf336bc14f3)` {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:01:05.098825 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22a77407-561a-417a-aae1-0f6fe98dd6b7 None admin] RBAC: Authorization granted {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:01:05.128085 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 61/124] 10.222.0.82 () {58 vars in 1439 bytes} [Tue Aug 2 10:01:04 2022] PUT /identity/v3/projects/c1809c9e380b45ec803130fe38eeaf24/users/891005fa5ff74041b75747d16cc66798/roles/237966442f05411191105bf336bc14f3 => generated 0 bytes in 271 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 02 10:01:05.130508 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22a77407-561a-417a-aae1-0f6fe98dd6b7 None admin] Authenticating user token {{(pid=80378) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:01:05.132543 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b1594ff-26bb-49da-88ea-34bec7f532c2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '891005fa5ff74041b75747d16cc66798', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80377) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:01:05.134437 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b1594ff-26bb-49da-88ea-34bec7f532c2 None admin] REQUEST_METHOD: `GET` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:01:05.135546 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b1594ff-26bb-49da-88ea-34bec7f532c2 None admin] SCRIPT_NAME: `/identity` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:01:05.136122 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b1594ff-26bb-49da-88ea-34bec7f532c2 None admin] PATH_INFO: `/v3/projects/c1809c9e380b45ec803130fe38eeaf24` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:01:05.138623 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b1594ff-26bb-49da-88ea-34bec7f532c2 None admin] RBAC: Authorizing `identity:get_project(project_id=c1809c9e380b45ec803130fe38eeaf24)` {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:01:05.141569 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b1594ff-26bb-49da-88ea-34bec7f532c2 None admin] RBAC: Authorization granted {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:01:05.152942 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: [pid: 80377|app: 0|req: 64/125] 10.222.0.82 () {56 vars in 1186 bytes} [Tue Aug 2 10:01:04 2022] GET /identity/v3/projects/c1809c9e380b45ec803130fe38eeaf24 => generated 298 bytes in 258 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 02 10:01:05.152942 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b1594ff-26bb-49da-88ea-34bec7f532c2 None admin] Authenticating user token {{(pid=80377) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:01:05.166171 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG dbcounter [-] [80377] Writing DB stats keystone:SELECT=100 {{(pid=80377) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 10:01:05.222253 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG dbcounter [-] [80378] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80378) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 10:01:05.335237 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22a77407-561a-417a-aae1-0f6fe98dd6b7 None admin] Validating token access rules against request {{(pid=80378) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:01:05.350694 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b1594ff-26bb-49da-88ea-34bec7f532c2 None admin] Validating token access rules against request {{(pid=80377) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:01:05.404281 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-059b93f2-6ed9-4794-b9d0-f304acb381ee None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '891005fa5ff74041b75747d16cc66798', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80378) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:01:05.404940 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-059b93f2-6ed9-4794-b9d0-f304acb381ee None admin] REQUEST_METHOD: `GET` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:01:05.405559 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-059b93f2-6ed9-4794-b9d0-f304acb381ee None admin] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:01:05.405880 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-059b93f2-6ed9-4794-b9d0-f304acb381ee None admin] PATH_INFO: `/v3/roles` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:01:05.406618 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-059b93f2-6ed9-4794-b9d0-f304acb381ee None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=80378) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 02 10:01:05.406929 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-059b93f2-6ed9-4794-b9d0-f304acb381ee None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:01:05.409449 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-059b93f2-6ed9-4794-b9d0-f304acb381ee None admin] RBAC: Authorization granted {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:01:05.423248 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 62/126] 10.222.0.82 () {56 vars in 1111 bytes} [Tue Aug 2 10:01:05 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 294 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 02 10:01:05.424780 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-059b93f2-6ed9-4794-b9d0-f304acb381ee None admin] Authenticating user token {{(pid=80378) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:01:05.431765 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06899864-a4ae-4f1e-874a-9ff27fa53510 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '891005fa5ff74041b75747d16cc66798', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80377) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:01:05.432811 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06899864-a4ae-4f1e-874a-9ff27fa53510 None admin] REQUEST_METHOD: `GET` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:01:05.433461 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06899864-a4ae-4f1e-874a-9ff27fa53510 None admin] SCRIPT_NAME: `/identity` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:01:05.435061 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06899864-a4ae-4f1e-874a-9ff27fa53510 None admin] PATH_INFO: `/v3/projects/c1809c9e380b45ec803130fe38eeaf24` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:01:05.436973 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06899864-a4ae-4f1e-874a-9ff27fa53510 None admin] RBAC: Authorizing `identity:get_project(project_id=c1809c9e380b45ec803130fe38eeaf24)` {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:01:05.439824 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06899864-a4ae-4f1e-874a-9ff27fa53510 None admin] RBAC: Authorization granted {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:01:05.442816 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: [pid: 80377|app: 0|req: 65/127] 10.222.0.82 () {56 vars in 1186 bytes} [Tue Aug 2 10:01:05 2022] GET /identity/v3/projects/c1809c9e380b45ec803130fe38eeaf24 => generated 298 bytes in 297 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 02 10:01:05.443915 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06899864-a4ae-4f1e-874a-9ff27fa53510 None admin] Authenticating user token {{(pid=80377) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:01:05.568624 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG dbcounter [-] [80377] Writing DB stats keystone:SELECT=100 {{(pid=80377) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 10:01:05.592252 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG dbcounter [-] [80378] Writing DB stats keystone:SELECT=100 {{(pid=80378) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 10:01:05.598531 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-059b93f2-6ed9-4794-b9d0-f304acb381ee None admin] Validating token access rules against request {{(pid=80378) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:01:05.604233 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06899864-a4ae-4f1e-874a-9ff27fa53510 None admin] Validating token access rules against request {{(pid=80377) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:01:05.643862 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b62e76b-6a64-4bb1-83e4-3f2c3f2c21a4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '891005fa5ff74041b75747d16cc66798', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80378) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:01:05.645394 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b62e76b-6a64-4bb1-83e4-3f2c3f2c21a4 None admin] REQUEST_METHOD: `GET` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:01:05.646180 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b62e76b-6a64-4bb1-83e4-3f2c3f2c21a4 None admin] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:01:05.646668 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b62e76b-6a64-4bb1-83e4-3f2c3f2c21a4 None admin] PATH_INFO: `/v3/roles/member` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:01:05.661722 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b62e76b-6a64-4bb1-83e4-3f2c3f2c21a4 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 02 10:01:05.662073 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8378f97b-02e0-492d-b310-cdaacb4fb753 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '891005fa5ff74041b75747d16cc66798', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80377) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:01:05.662782 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8378f97b-02e0-492d-b310-cdaacb4fb753 None admin] REQUEST_METHOD: `GET` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:01:05.663270 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8378f97b-02e0-492d-b310-cdaacb4fb753 None admin] SCRIPT_NAME: `/identity` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:01:05.663712 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8378f97b-02e0-492d-b310-cdaacb4fb753 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:01:05.665034 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8378f97b-02e0-492d-b310-cdaacb4fb753 None admin] RBAC: Adding query filter params (role.id=444e768aa5fb44bbab6c20afad6a0218, scope.project.id=c1809c9e380b45ec803130fe38eeaf24, user.id=1c51a65536174654945f89334229c254) {{(pid=80377) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 02 10:01:05.665447 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8378f97b-02e0-492d-b310-cdaacb4fb753 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:01:05.667398 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8378f97b-02e0-492d-b310-cdaacb4fb753 None admin] RBAC: Authorization granted {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:01:05.669438 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application [None req-9b62e76b-6a64-4bb1-83e4-3f2c3f2c21a4 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Aug 02 10:01:05.669438 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 02 10:01:05.669438 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 02 10:01:05.669438 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 02 10:01:05.669438 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 02 10:01:05.669438 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 02 10:01:05.669438 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 02 10:01:05.669438 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 02 10:01:05.669438 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 02 10:01:05.669438 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 02 10:01:05.669438 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 02 10:01:05.669438 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 02 10:01:05.669438 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 02 10:01:05.669438 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 02 10:01:05.669438 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 02 10:01:05.669438 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 02 10:01:05.670330 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 02 10:01:05.670330 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 02 10:01:05.670330 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 02 10:01:05.670330 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 02 10:01:05.670330 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 02 10:01:05.670330 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 02 10:01:05.670330 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 02 10:01:05.670330 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 02 10:01:05.670330 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 02 10:01:05.670330 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application return self.get_or_create( Aug 02 10:01:05.670330 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 02 10:01:05.670330 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application with Lock( Aug 02 10:01:05.670330 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 02 10:01:05.670330 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application return self._enter() Aug 02 10:01:05.670330 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 02 10:01:05.670330 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 02 10:01:05.671119 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 02 10:01:05.671119 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application return self.creator() Aug 02 10:01:05.671119 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 02 10:01:05.671119 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application created_value = creator( Aug 02 10:01:05.671119 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 02 10:01:05.671119 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 02 10:01:05.671119 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 02 10:01:05.671119 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 02 10:01:05.671119 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 02 10:01:05.671119 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 02 10:01:05.671119 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Aug 02 10:01:05.671119 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application Aug 02 10:01:05.678646 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 63/128] 10.222.0.82 () {56 vars in 1099 bytes} [Tue Aug 2 10:01:05 2022] GET /identity/v3/roles/member => generated 84 bytes in 252 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 02 10:01:05.680558 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b62e76b-6a64-4bb1-83e4-3f2c3f2c21a4 None admin] Authenticating user token {{(pid=80378) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:01:05.686919 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: [pid: 80377|app: 0|req: 66/129] 10.222.0.82 () {56 vars in 1374 bytes} [Tue Aug 2 10:01:05 2022] GET /identity/v3/role_assignments?user.id=1c51a65536174654945f89334229c254&scope.project.id=c1809c9e380b45ec803130fe38eeaf24&role.id=444e768aa5fb44bbab6c20afad6a0218 => generated 259 bytes in 243 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 02 10:01:05.688056 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8378f97b-02e0-492d-b310-cdaacb4fb753 None admin] Authenticating user token {{(pid=80377) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:01:05.827261 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b62e76b-6a64-4bb1-83e4-3f2c3f2c21a4 None admin] Validating token access rules against request {{(pid=80378) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:01:05.884525 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8378f97b-02e0-492d-b310-cdaacb4fb753 None admin] Validating token access rules against request {{(pid=80377) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:01:05.889996 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6506bad-4987-40ff-803c-d4ff5e1ce21e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '891005fa5ff74041b75747d16cc66798', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80378) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:01:05.890755 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6506bad-4987-40ff-803c-d4ff5e1ce21e None admin] REQUEST_METHOD: `GET` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:01:05.891203 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6506bad-4987-40ff-803c-d4ff5e1ce21e None admin] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:01:05.891720 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6506bad-4987-40ff-803c-d4ff5e1ce21e None admin] PATH_INFO: `/v3/users/71a4a6ffabe74df39f0a7ca7ea7b33b8` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:01:05.895982 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6506bad-4987-40ff-803c-d4ff5e1ce21e None admin] RBAC: Authorizing `identity:get_user(user_id=71a4a6ffabe74df39f0a7ca7ea7b33b8)` {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:01:05.899136 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6506bad-4987-40ff-803c-d4ff5e1ce21e None admin] RBAC: Authorization granted {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:01:05.901327 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 64/130] 10.222.0.82 () {56 vars in 1177 bytes} [Tue Aug 2 10:01:05 2022] GET /identity/v3/users/71a4a6ffabe74df39f0a7ca7ea7b33b8 => generated 275 bytes in 223 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 02 10:01:05.902499 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6506bad-4987-40ff-803c-d4ff5e1ce21e None admin] Authenticating user token {{(pid=80378) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:01:05.940012 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG dbcounter [-] [80378] Writing DB stats keystone:SELECT=100 {{(pid=80378) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 10:01:06.011343 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ca9899a-bf04-4818-9627-060473bbcb95 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '891005fa5ff74041b75747d16cc66798', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80377) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:01:06.012098 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ca9899a-bf04-4818-9627-060473bbcb95 None admin] REQUEST_METHOD: `PUT` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:01:06.012385 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ca9899a-bf04-4818-9627-060473bbcb95 None admin] SCRIPT_NAME: `/identity` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:01:06.012672 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ca9899a-bf04-4818-9627-060473bbcb95 None admin] PATH_INFO: `/v3/projects/c1809c9e380b45ec803130fe38eeaf24/users/71a4a6ffabe74df39f0a7ca7ea7b33b8/roles/e1c04b19fc4543b8b54f58bbcffbc598` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:01:06.015939 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ca9899a-bf04-4818-9627-060473bbcb95 None admin] RBAC: Authorizing `identity:create_grant(project_id=c1809c9e380b45ec803130fe38eeaf24, user_id=71a4a6ffabe74df39f0a7ca7ea7b33b8, role_id=e1c04b19fc4543b8b54f58bbcffbc598)` {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:01:06.018076 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ca9899a-bf04-4818-9627-060473bbcb95 None admin] RBAC: Authorization granted {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:01:06.043611 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG dbcounter [-] [80377] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80377) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 10:01:06.049704 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6506bad-4987-40ff-803c-d4ff5e1ce21e None admin] Validating token access rules against request {{(pid=80378) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:01:06.064415 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: [pid: 80377|app: 0|req: 67/131] 10.222.0.82 () {58 vars in 1439 bytes} [Tue Aug 2 10:01:05 2022] PUT /identity/v3/projects/c1809c9e380b45ec803130fe38eeaf24/users/71a4a6ffabe74df39f0a7ca7ea7b33b8/roles/e1c04b19fc4543b8b54f58bbcffbc598 => generated 0 bytes in 377 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 02 10:01:06.065659 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ca9899a-bf04-4818-9627-060473bbcb95 None admin] Authenticating user token {{(pid=80377) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:01:06.095992 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab28a2af-c658-49a9-bb91-758d14320d62 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '891005fa5ff74041b75747d16cc66798', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80378) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:01:06.097134 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab28a2af-c658-49a9-bb91-758d14320d62 None admin] REQUEST_METHOD: `GET` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:01:06.097749 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab28a2af-c658-49a9-bb91-758d14320d62 None admin] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:01:06.098310 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab28a2af-c658-49a9-bb91-758d14320d62 None admin] PATH_INFO: `/v3/roles` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:01:06.099522 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab28a2af-c658-49a9-bb91-758d14320d62 None admin] RBAC: Adding query filter params (name=member) {{(pid=80378) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 02 10:01:06.100159 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab28a2af-c658-49a9-bb91-758d14320d62 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:01:06.102341 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab28a2af-c658-49a9-bb91-758d14320d62 None admin] RBAC: Authorization granted {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:01:06.115696 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 65/132] 10.222.0.82 () {56 vars in 1101 bytes} [Tue Aug 2 10:01:05 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 213 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 02 10:01:06.123696 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab28a2af-c658-49a9-bb91-758d14320d62 None admin] Authenticating user token {{(pid=80378) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:01:06.256019 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG dbcounter [-] [80378] Writing DB stats keystone:SELECT=100 {{(pid=80378) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 10:01:06.272111 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab28a2af-c658-49a9-bb91-758d14320d62 None admin] Validating token access rules against request {{(pid=80378) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:01:06.318677 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffa2b451-527f-485d-a6f5-7a9837238a0b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '891005fa5ff74041b75747d16cc66798', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80378) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:01:06.319738 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffa2b451-527f-485d-a6f5-7a9837238a0b None admin] REQUEST_METHOD: `GET` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:01:06.320251 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffa2b451-527f-485d-a6f5-7a9837238a0b None admin] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:01:06.320770 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffa2b451-527f-485d-a6f5-7a9837238a0b None admin] PATH_INFO: `/v3/users/71a4a6ffabe74df39f0a7ca7ea7b33b8` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:01:06.322399 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ffa2b451-527f-485d-a6f5-7a9837238a0b None admin] RBAC: Authorizing `identity:get_user(user_id=71a4a6ffabe74df39f0a7ca7ea7b33b8)` {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:01:06.324112 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ffa2b451-527f-485d-a6f5-7a9837238a0b None admin] RBAC: Authorization granted {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:01:06.326177 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 66/133] 10.222.0.82 () {56 vars in 1177 bytes} [Tue Aug 2 10:01:06 2022] GET /identity/v3/users/71a4a6ffabe74df39f0a7ca7ea7b33b8 => generated 275 bytes in 204 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 02 10:01:06.336995 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffa2b451-527f-485d-a6f5-7a9837238a0b None admin] Authenticating user token {{(pid=80378) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:01:06.453531 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ca9899a-bf04-4818-9627-060473bbcb95 None admin] Validating token access rules against request {{(pid=80377) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:01:06.579926 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89803a8b-18a3-47ad-9bc1-eae765e89fe1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '891005fa5ff74041b75747d16cc66798', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80377) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:01:06.581072 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89803a8b-18a3-47ad-9bc1-eae765e89fe1 None admin] REQUEST_METHOD: `GET` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:01:06.581576 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89803a8b-18a3-47ad-9bc1-eae765e89fe1 None admin] SCRIPT_NAME: `/identity` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:01:06.582088 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89803a8b-18a3-47ad-9bc1-eae765e89fe1 None admin] PATH_INFO: `/v3/projects/c1809c9e380b45ec803130fe38eeaf24` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:01:06.587178 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89803a8b-18a3-47ad-9bc1-eae765e89fe1 None admin] RBAC: Authorizing `identity:get_project(project_id=c1809c9e380b45ec803130fe38eeaf24)` {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:01:06.592893 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89803a8b-18a3-47ad-9bc1-eae765e89fe1 None admin] RBAC: Authorization granted {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:01:06.593278 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: [pid: 80377|app: 0|req: 68/134] 10.222.0.82 () {56 vars in 1186 bytes} [Tue Aug 2 10:01:06 2022] GET /identity/v3/projects/c1809c9e380b45ec803130fe38eeaf24 => generated 298 bytes in 528 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 02 10:01:06.600463 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89803a8b-18a3-47ad-9bc1-eae765e89fe1 None admin] Authenticating user token {{(pid=80377) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:01:06.622741 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffa2b451-527f-485d-a6f5-7a9837238a0b None admin] Validating token access rules against request {{(pid=80378) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:01:06.664946 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-866a820f-e21d-4248-a883-b53c52a413ff None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '891005fa5ff74041b75747d16cc66798', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80378) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:01:06.666058 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-866a820f-e21d-4248-a883-b53c52a413ff None admin] REQUEST_METHOD: `GET` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:01:06.666550 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-866a820f-e21d-4248-a883-b53c52a413ff None admin] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:01:06.667120 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-866a820f-e21d-4248-a883-b53c52a413ff None admin] PATH_INFO: `/v3/projects/354cc945e1c64e598aefaee72be3cffc` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:01:06.668714 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-866a820f-e21d-4248-a883-b53c52a413ff None admin] RBAC: Authorizing `identity:get_project(project_id=354cc945e1c64e598aefaee72be3cffc)` {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:01:06.670517 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-866a820f-e21d-4248-a883-b53c52a413ff None admin] RBAC: Authorization granted {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:01:06.672644 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 67/135] 10.222.0.82 () {56 vars in 1186 bytes} [Tue Aug 2 10:01:06 2022] GET /identity/v3/projects/354cc945e1c64e598aefaee72be3cffc => generated 312 bytes in 337 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 02 10:01:06.679821 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-866a820f-e21d-4248-a883-b53c52a413ff None admin] Authenticating user token {{(pid=80378) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:01:06.719606 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG dbcounter [-] [80378] Writing DB stats keystone:SELECT=100 {{(pid=80378) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 10:01:06.790640 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG dbcounter [-] [80377] Writing DB stats keystone:SELECT=100 {{(pid=80377) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 10:01:06.810011 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-866a820f-e21d-4248-a883-b53c52a413ff None admin] Validating token access rules against request {{(pid=80378) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:01:06.885479 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89803a8b-18a3-47ad-9bc1-eae765e89fe1 None admin] Validating token access rules against request {{(pid=80377) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:01:06.914818 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4cc35f1b-f4e9-43f4-9b80-9a01c4b8e4e8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '891005fa5ff74041b75747d16cc66798', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80378) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:01:06.918163 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cc35f1b-f4e9-43f4-9b80-9a01c4b8e4e8 None admin] REQUEST_METHOD: `PUT` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:01:06.919446 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cc35f1b-f4e9-43f4-9b80-9a01c4b8e4e8 None admin] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:01:06.919998 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cc35f1b-f4e9-43f4-9b80-9a01c4b8e4e8 None admin] PATH_INFO: `/v3/projects/354cc945e1c64e598aefaee72be3cffc/users/71a4a6ffabe74df39f0a7ca7ea7b33b8/roles/e1c04b19fc4543b8b54f58bbcffbc598` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:01:06.925534 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4cc35f1b-f4e9-43f4-9b80-9a01c4b8e4e8 None admin] RBAC: Authorizing `identity:create_grant(project_id=354cc945e1c64e598aefaee72be3cffc, user_id=71a4a6ffabe74df39f0a7ca7ea7b33b8, role_id=e1c04b19fc4543b8b54f58bbcffbc598)` {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:01:06.928129 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4cc35f1b-f4e9-43f4-9b80-9a01c4b8e4e8 None admin] RBAC: Authorization granted {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:01:06.952527 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 68/136] 10.222.0.82 () {58 vars in 1439 bytes} [Tue Aug 2 10:01:06 2022] PUT /identity/v3/projects/354cc945e1c64e598aefaee72be3cffc/users/71a4a6ffabe74df39f0a7ca7ea7b33b8/roles/e1c04b19fc4543b8b54f58bbcffbc598 => generated 0 bytes in 273 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 02 10:01:06.956548 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aba755ec-b06b-4e47-8469-0e3617209f33 None None] REQUEST_METHOD: `GET` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:01:06.957360 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aba755ec-b06b-4e47-8469-0e3617209f33 None None] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:01:06.958050 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aba755ec-b06b-4e47-8469-0e3617209f33 None None] PATH_INFO: `/` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:01:06.962519 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 69/137] 10.222.0.82 () {54 vars in 926 bytes} [Tue Aug 2 10:01:06 2022] GET /identity => generated 270 bytes in 9 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 02 10:01:06.969003 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3985d7a3-060a-4226-9388-819c6f298622 None None] REQUEST_METHOD: `POST` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:01:06.969748 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3985d7a3-060a-4226-9388-819c6f298622 None None] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:01:06.970342 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3985d7a3-060a-4226-9388-819c6f298622 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:01:06.974554 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-158b720c-b3e0-49f5-92bd-f995efc86513 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '891005fa5ff74041b75747d16cc66798', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80377) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:01:06.976197 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-158b720c-b3e0-49f5-92bd-f995efc86513 None admin] REQUEST_METHOD: `PUT` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:01:06.977648 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-158b720c-b3e0-49f5-92bd-f995efc86513 None admin] SCRIPT_NAME: `/identity` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:01:06.979336 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-158b720c-b3e0-49f5-92bd-f995efc86513 None admin] PATH_INFO: `/v3/projects/c1809c9e380b45ec803130fe38eeaf24/users/71a4a6ffabe74df39f0a7ca7ea7b33b8/roles/649993d933d44c50b268df7fdf336fbf` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:01:06.983943 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-158b720c-b3e0-49f5-92bd-f995efc86513 None admin] RBAC: Authorizing `identity:create_grant(project_id=c1809c9e380b45ec803130fe38eeaf24, user_id=71a4a6ffabe74df39f0a7ca7ea7b33b8, role_id=649993d933d44c50b268df7fdf336fbf)` {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:01:06.987549 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-158b720c-b3e0-49f5-92bd-f995efc86513 None admin] RBAC: Authorization granted {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:01:07.015746 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.auth.core [None req-3985d7a3-060a-4226-9388-819c6f298622 None None] MFA Rules not processed for user `891005fa5ff74041b75747d16cc66798`. Rule list: `[]` (Enabled: `True`). {{(pid=80378) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 02 10:01:07.017717 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.fernet_utils [None req-3985d7a3-060a-4226-9388-819c6f298622 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80378) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 10:01:07.018497 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: [pid: 80377|app: 0|req: 69/138] 10.222.0.82 () {58 vars in 1439 bytes} [Tue Aug 2 10:01:06 2022] PUT /identity/v3/projects/c1809c9e380b45ec803130fe38eeaf24/users/71a4a6ffabe74df39f0a7ca7ea7b33b8/roles/649993d933d44c50b268df7fdf336fbf => generated 0 bytes in 419 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 02 10:01:07.159844 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 70/139] 10.222.0.82 () {58 vars in 1023 bytes} [Tue Aug 2 10:01:06 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 192 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Aug 02 10:01:07.211833 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-158b720c-b3e0-49f5-92bd-f995efc86513 None admin] Authenticating user token {{(pid=80377) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:01:07.371613 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-158b720c-b3e0-49f5-92bd-f995efc86513 None admin] Validating token access rules against request {{(pid=80377) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:01:07.415551 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG dbcounter [-] [80377] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80377) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 10:01:07.432380 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4217f3ea-4de4-4c33-a4d6-f49591786d9b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '891005fa5ff74041b75747d16cc66798', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80377) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:01:07.433089 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4217f3ea-4de4-4c33-a4d6-f49591786d9b None admin] REQUEST_METHOD: `GET` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:01:07.433402 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4217f3ea-4de4-4c33-a4d6-f49591786d9b None admin] SCRIPT_NAME: `/identity` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:01:07.433826 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4217f3ea-4de4-4c33-a4d6-f49591786d9b None admin] PATH_INFO: `/v3/domains/default` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:01:07.435102 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4217f3ea-4de4-4c33-a4d6-f49591786d9b None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:01:07.436704 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4217f3ea-4de4-4c33-a4d6-f49591786d9b None admin] RBAC: Authorization granted {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:01:07.438704 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: [pid: 80377|app: 0|req: 70/140] 10.222.0.82 () {56 vars in 1108 bytes} [Tue Aug 2 10:01:07 2022] GET /identity/v3/domains/default => generated 199 bytes in 228 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 02 10:01:07.444607 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3985d7a3-060a-4226-9388-819c6f298622 None None] Authenticating user token {{(pid=80378) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:01:07.481335 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG dbcounter [-] [80378] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80378) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 10:01:07.664403 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3985d7a3-060a-4226-9388-819c6f298622 None None] Validating token access rules against request {{(pid=80378) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:01:07.732293 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8ec8d24-6712-4a76-871e-da4fa28c54a3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '891005fa5ff74041b75747d16cc66798', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80378) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:01:07.733906 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8ec8d24-6712-4a76-871e-da4fa28c54a3 None admin] REQUEST_METHOD: `POST` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:01:07.734716 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8ec8d24-6712-4a76-871e-da4fa28c54a3 None admin] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:01:07.736028 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8ec8d24-6712-4a76-871e-da4fa28c54a3 None admin] PATH_INFO: `/v3/users` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:01:07.737628 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8ec8d24-6712-4a76-871e-da4fa28c54a3 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:01:07.740832 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8ec8d24-6712-4a76-871e-da4fa28c54a3 None admin] RBAC: Authorization granted {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:01:07.783071 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 71/141] 10.222.0.82 () {60 vars in 1132 bytes} [Tue Aug 2 10:01:07 2022] POST /identity/v3/users => generated 283 bytes in 339 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 02 10:01:09.166732 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54868b45-8a18-4a70-b6c2-dca745d35550 None None] REQUEST_METHOD: `GET` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:01:09.167467 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54868b45-8a18-4a70-b6c2-dca745d35550 None None] SCRIPT_NAME: `/identity` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:01:09.167467 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54868b45-8a18-4a70-b6c2-dca745d35550 None None] PATH_INFO: `/` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:01:09.168586 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: [pid: 80377|app: 0|req: 71/142] 10.222.0.82 () {54 vars in 926 bytes} [Tue Aug 2 10:01:09 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 02 10:01:09.178521 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62baa18e-16c1-4650-88fe-76ee0a63c871 None None] REQUEST_METHOD: `POST` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:01:09.180132 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62baa18e-16c1-4650-88fe-76ee0a63c871 None None] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:01:09.180918 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62baa18e-16c1-4650-88fe-76ee0a63c871 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:01:09.223327 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.auth.core [None req-62baa18e-16c1-4650-88fe-76ee0a63c871 None None] MFA Rules not processed for user `891005fa5ff74041b75747d16cc66798`. Rule list: `[]` (Enabled: `True`). {{(pid=80378) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 02 10:01:09.225810 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.fernet_utils [None req-62baa18e-16c1-4650-88fe-76ee0a63c871 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80378) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 10:01:09.370489 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 72/143] 10.222.0.82 () {58 vars in 1023 bytes} [Tue Aug 2 10:01:09 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 194 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 02 10:01:09.420272 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54868b45-8a18-4a70-b6c2-dca745d35550 None None] Authenticating user token {{(pid=80377) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:01:09.611783 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54868b45-8a18-4a70-b6c2-dca745d35550 None None] Validating token access rules against request {{(pid=80377) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:01:09.698137 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09fdf16e-1871-424c-b379-beabffc29770 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '891005fa5ff74041b75747d16cc66798', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80377) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:01:09.699115 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09fdf16e-1871-424c-b379-beabffc29770 None admin] REQUEST_METHOD: `GET` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:01:09.699603 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09fdf16e-1871-424c-b379-beabffc29770 None admin] SCRIPT_NAME: `/identity` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:01:09.700059 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09fdf16e-1871-424c-b379-beabffc29770 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:01:09.731546 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09fdf16e-1871-424c-b379-beabffc29770 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 02 10:01:09.742745 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application [None req-09fdf16e-1871-424c-b379-beabffc29770 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 02 10:01:09.742745 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 02 10:01:09.742745 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 02 10:01:09.742745 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 02 10:01:09.742745 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 02 10:01:09.742745 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 02 10:01:09.742745 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 02 10:01:09.742745 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 02 10:01:09.742745 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 02 10:01:09.742745 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 02 10:01:09.742745 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 02 10:01:09.742745 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 02 10:01:09.742745 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 02 10:01:09.742745 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 02 10:01:09.742745 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 02 10:01:09.742745 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 02 10:01:09.744533 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 02 10:01:09.744533 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 02 10:01:09.744533 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 02 10:01:09.744533 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 02 10:01:09.744533 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 02 10:01:09.744533 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 02 10:01:09.744533 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 02 10:01:09.744533 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 02 10:01:09.744533 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 02 10:01:09.744533 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application return self.get_or_create( Aug 02 10:01:09.744533 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 02 10:01:09.744533 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application with Lock( Aug 02 10:01:09.744533 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 02 10:01:09.744533 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application return self._enter() Aug 02 10:01:09.744533 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 02 10:01:09.744533 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 02 10:01:09.745990 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 02 10:01:09.745990 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application return self.creator() Aug 02 10:01:09.745990 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 02 10:01:09.745990 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application created_value = creator( Aug 02 10:01:09.745990 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 02 10:01:09.745990 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 02 10:01:09.745990 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 02 10:01:09.745990 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 02 10:01:09.745990 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 02 10:01:09.745990 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 02 10:01:09.745990 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 02 10:01:09.745990 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application Aug 02 10:01:09.745990 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: [pid: 80377|app: 0|req: 72/144] 10.222.0.82 () {56 vars in 1096 bytes} [Tue Aug 2 10:01:09 2022] GET /identity/v3/roles/admin => generated 83 bytes in 325 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 02 10:01:09.762994 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62baa18e-16c1-4650-88fe-76ee0a63c871 None None] Authenticating user token {{(pid=80378) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:01:09.765872 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG dbcounter [-] [80378] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80378) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 10:01:09.866651 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3777bca1-0c43-4cfc-8d87-b7f60bcfecb8 None None] REQUEST_METHOD: `GET` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:01:09.867126 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3777bca1-0c43-4cfc-8d87-b7f60bcfecb8 None None] SCRIPT_NAME: `/identity` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:01:09.867624 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3777bca1-0c43-4cfc-8d87-b7f60bcfecb8 None None] PATH_INFO: `/` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:01:09.870371 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: [pid: 80377|app: 0|req: 73/145] 10.222.0.82 () {54 vars in 926 bytes} [Tue Aug 2 10:01:09 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 02 10:01:09.892838 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1806a93-38ca-4d86-9ad2-ced7c65355df None None] REQUEST_METHOD: `POST` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:01:09.893372 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1806a93-38ca-4d86-9ad2-ced7c65355df None None] SCRIPT_NAME: `/identity` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:01:09.893930 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1806a93-38ca-4d86-9ad2-ced7c65355df None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:01:09.935527 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.auth.core [None req-f1806a93-38ca-4d86-9ad2-ced7c65355df None None] MFA Rules not processed for user `891005fa5ff74041b75747d16cc66798`. Rule list: `[]` (Enabled: `True`). {{(pid=80377) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 02 10:01:09.936825 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.fernet_utils [None req-f1806a93-38ca-4d86-9ad2-ced7c65355df None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80377) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 10:01:09.972827 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62baa18e-16c1-4650-88fe-76ee0a63c871 None None] Validating token access rules against request {{(pid=80378) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:01:10.076803 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3941420-fa90-48c2-94ac-b15ce620d8ea None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '891005fa5ff74041b75747d16cc66798', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80378) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:01:10.079022 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3941420-fa90-48c2-94ac-b15ce620d8ea None admin] REQUEST_METHOD: `GET` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:01:10.080364 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3941420-fa90-48c2-94ac-b15ce620d8ea None admin] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:01:10.081601 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3941420-fa90-48c2-94ac-b15ce620d8ea None admin] PATH_INFO: `/v3/roles` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:01:10.083378 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3941420-fa90-48c2-94ac-b15ce620d8ea None admin] RBAC: Adding query filter params (name=admin) {{(pid=80378) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 02 10:01:10.084218 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3941420-fa90-48c2-94ac-b15ce620d8ea None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:01:10.091935 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3941420-fa90-48c2-94ac-b15ce620d8ea None admin] RBAC: Authorization granted {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:01:10.119105 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 73/146] 10.222.0.82 () {56 vars in 1099 bytes} [Tue Aug 2 10:01:09 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 357 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 02 10:01:10.121096 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG dbcounter [-] [80377] Writing DB stats keystone:SELECT=100 {{(pid=80377) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 10:01:10.131210 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3941420-fa90-48c2-94ac-b15ce620d8ea None admin] Authenticating user token {{(pid=80378) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:01:10.173714 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: [pid: 80377|app: 0|req: 74/147] 10.222.0.82 () {58 vars in 1023 bytes} [Tue Aug 2 10:01:09 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 283 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 02 10:01:10.245568 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1806a93-38ca-4d86-9ad2-ced7c65355df None None] Authenticating user token {{(pid=80377) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:01:10.303375 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG dbcounter [-] [80378] Writing DB stats keystone:SELECT=100 {{(pid=80378) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 10:01:10.425651 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3941420-fa90-48c2-94ac-b15ce620d8ea None admin] Validating token access rules against request {{(pid=80378) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:01:10.523915 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-782b129f-8e2d-4e24-a589-b57284f359c2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '891005fa5ff74041b75747d16cc66798', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80378) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:01:10.525778 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-782b129f-8e2d-4e24-a589-b57284f359c2 None admin] REQUEST_METHOD: `GET` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:01:10.526775 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-782b129f-8e2d-4e24-a589-b57284f359c2 None admin] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:01:10.527599 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-782b129f-8e2d-4e24-a589-b57284f359c2 None admin] PATH_INFO: `/v3/users/891005fa5ff74041b75747d16cc66798` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:01:10.530591 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-782b129f-8e2d-4e24-a589-b57284f359c2 None admin] RBAC: Authorizing `identity:get_user(user_id=891005fa5ff74041b75747d16cc66798)` {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:01:10.534329 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-782b129f-8e2d-4e24-a589-b57284f359c2 None admin] RBAC: Authorization granted {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:01:10.543845 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 74/148] 10.222.0.82 () {56 vars in 1177 bytes} [Tue Aug 2 10:01:10 2022] GET /identity/v3/users/891005fa5ff74041b75747d16cc66798 => generated 247 bytes in 414 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 02 10:01:10.557148 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-782b129f-8e2d-4e24-a589-b57284f359c2 None admin] Authenticating user token {{(pid=80378) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:01:10.621574 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1806a93-38ca-4d86-9ad2-ced7c65355df None None] Validating token access rules against request {{(pid=80377) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:01:10.748098 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e35f7a4-9707-4891-82e7-9dcf0f79bc36 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '891005fa5ff74041b75747d16cc66798', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80377) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:01:10.752153 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e35f7a4-9707-4891-82e7-9dcf0f79bc36 None admin] REQUEST_METHOD: `GET` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:01:10.752153 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e35f7a4-9707-4891-82e7-9dcf0f79bc36 None admin] SCRIPT_NAME: `/identity` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:01:10.752153 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e35f7a4-9707-4891-82e7-9dcf0f79bc36 None admin] PATH_INFO: `/v3/roles/reader` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:01:10.818916 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e35f7a4-9707-4891-82e7-9dcf0f79bc36 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 02 10:01:10.827536 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application [None req-7e35f7a4-9707-4891-82e7-9dcf0f79bc36 None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Aug 02 10:01:10.827536 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 02 10:01:10.827536 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 02 10:01:10.827536 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 02 10:01:10.827536 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 02 10:01:10.827536 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 02 10:01:10.827536 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 02 10:01:10.827536 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 02 10:01:10.827536 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 02 10:01:10.827536 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 02 10:01:10.827536 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 02 10:01:10.827536 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 02 10:01:10.827536 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 02 10:01:10.827536 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 02 10:01:10.827536 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 02 10:01:10.827536 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 02 10:01:10.829264 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 02 10:01:10.829264 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 02 10:01:10.829264 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 02 10:01:10.829264 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 02 10:01:10.829264 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 02 10:01:10.829264 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 02 10:01:10.829264 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 02 10:01:10.829264 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 02 10:01:10.829264 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 02 10:01:10.829264 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application return self.get_or_create( Aug 02 10:01:10.829264 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 02 10:01:10.829264 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application with Lock( Aug 02 10:01:10.829264 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 02 10:01:10.829264 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application return self._enter() Aug 02 10:01:10.829264 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 02 10:01:10.829264 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 02 10:01:10.830754 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 02 10:01:10.830754 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application return self.creator() Aug 02 10:01:10.830754 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 02 10:01:10.830754 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application created_value = creator( Aug 02 10:01:10.830754 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 02 10:01:10.830754 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 02 10:01:10.830754 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 02 10:01:10.830754 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 02 10:01:10.830754 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 02 10:01:10.830754 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 02 10:01:10.830754 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Aug 02 10:01:10.830754 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application Aug 02 10:01:10.830754 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: [pid: 80377|app: 0|req: 75/149] 10.222.0.82 () {56 vars in 1099 bytes} [Tue Aug 2 10:01:10 2022] GET /identity/v3/roles/reader => generated 84 bytes in 589 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 02 10:01:10.845852 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e35f7a4-9707-4891-82e7-9dcf0f79bc36 None admin] Authenticating user token {{(pid=80377) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:01:10.969245 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-782b129f-8e2d-4e24-a589-b57284f359c2 None admin] Validating token access rules against request {{(pid=80378) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:01:11.087712 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG dbcounter [-] [80378] Writing DB stats keystone:SELECT=100 {{(pid=80378) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 10:01:11.096640 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG dbcounter [-] [80377] Writing DB stats keystone:SELECT=100 {{(pid=80377) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 10:01:11.105910 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1aaac157-4b22-4c44-a9ea-751848645808 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '891005fa5ff74041b75747d16cc66798', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80378) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:01:11.107217 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1aaac157-4b22-4c44-a9ea-751848645808 None admin] REQUEST_METHOD: `GET` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:01:11.107731 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1aaac157-4b22-4c44-a9ea-751848645808 None admin] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:01:11.108342 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1aaac157-4b22-4c44-a9ea-751848645808 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:01:11.298168 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e35f7a4-9707-4891-82e7-9dcf0f79bc36 None admin] Validating token access rules against request {{(pid=80377) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:01:11.380378 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0bce4138-2ce9-49fd-9d95-d3eed1e9e0cf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '891005fa5ff74041b75747d16cc66798', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80377) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:01:11.381177 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bce4138-2ce9-49fd-9d95-d3eed1e9e0cf None admin] REQUEST_METHOD: `GET` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:01:11.381473 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bce4138-2ce9-49fd-9d95-d3eed1e9e0cf None admin] SCRIPT_NAME: `/identity` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:01:11.381869 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bce4138-2ce9-49fd-9d95-d3eed1e9e0cf None admin] PATH_INFO: `/v3/roles` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:01:11.382653 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0bce4138-2ce9-49fd-9d95-d3eed1e9e0cf None admin] RBAC: Adding query filter params (name=reader) {{(pid=80377) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 02 10:01:11.382968 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0bce4138-2ce9-49fd-9d95-d3eed1e9e0cf None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:01:11.384416 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0bce4138-2ce9-49fd-9d95-d3eed1e9e0cf None admin] RBAC: Authorization granted {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:01:11.394762 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1aaac157-4b22-4c44-a9ea-751848645808 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:01:11.396334 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1aaac157-4b22-4c44-a9ea-751848645808 None admin] RBAC: Authorization granted {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:01:11.419945 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: [pid: 80377|app: 0|req: 76/150] 10.222.0.82 () {56 vars in 1101 bytes} [Tue Aug 2 10:01:10 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 578 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 02 10:01:11.439848 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0bce4138-2ce9-49fd-9d95-d3eed1e9e0cf None admin] Authenticating user token {{(pid=80377) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:01:11.711597 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 75/151] 10.222.0.82 () {58 vars in 1282 bytes} [Tue Aug 2 10:01:10 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 1155 msecs (HTTP/1.1 200) 6 headers in 358 bytes (2 switches on core 0) Aug 02 10:01:11.713314 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9e7f716-2773-48d9-a2e7-5810b914bbda None None] REQUEST_METHOD: `GET` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:01:11.718513 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9e7f716-2773-48d9-a2e7-5810b914bbda None None] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:01:11.718991 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9e7f716-2773-48d9-a2e7-5810b914bbda None None] PATH_INFO: `/` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:01:11.721647 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 76/152] 10.222.0.82 () {54 vars in 926 bytes} [Tue Aug 2 10:01:11 2022] GET /identity => generated 270 bytes in 10 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 02 10:01:11.723022 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df5e84bc-d95a-4ee4-a187-b9838d48c220 None None] REQUEST_METHOD: `GET` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:01:11.723323 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df5e84bc-d95a-4ee4-a187-b9838d48c220 None None] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:01:11.723605 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df5e84bc-d95a-4ee4-a187-b9838d48c220 None None] PATH_INFO: `/` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:01:11.725460 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 77/153] 10.222.0.82 () {54 vars in 926 bytes} [Tue Aug 2 10:01:11 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 02 10:01:11.732690 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3af0b4bd-9687-4413-9087-3042578ec6c7 None None] REQUEST_METHOD: `POST` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:01:11.733355 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3af0b4bd-9687-4413-9087-3042578ec6c7 None None] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:01:11.733651 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3af0b4bd-9687-4413-9087-3042578ec6c7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:01:11.762812 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.auth.core [None req-3af0b4bd-9687-4413-9087-3042578ec6c7 None None] MFA Rules not processed for user `891005fa5ff74041b75747d16cc66798`. Rule list: `[]` (Enabled: `True`). {{(pid=80378) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 02 10:01:11.764568 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.fernet_utils [None req-3af0b4bd-9687-4413-9087-3042578ec6c7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80378) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 10:01:11.819243 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0bce4138-2ce9-49fd-9d95-d3eed1e9e0cf None admin] Validating token access rules against request {{(pid=80377) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:01:11.834649 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG dbcounter [-] [80377] Writing DB stats keystone:SELECT=100 {{(pid=80377) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 10:01:11.837155 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG dbcounter [-] [80378] Writing DB stats keystone:SELECT=100 {{(pid=80378) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 10:01:11.903812 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f6a9188-0258-4097-a811-84f376b70c81 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '891005fa5ff74041b75747d16cc66798', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80377) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:01:11.904917 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f6a9188-0258-4097-a811-84f376b70c81 None admin] REQUEST_METHOD: `GET` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:01:11.905679 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f6a9188-0258-4097-a811-84f376b70c81 None admin] SCRIPT_NAME: `/identity` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:01:11.906006 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f6a9188-0258-4097-a811-84f376b70c81 None admin] PATH_INFO: `/v3/users/1c51a65536174654945f89334229c254` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:01:11.909973 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f6a9188-0258-4097-a811-84f376b70c81 None admin] RBAC: Authorizing `identity:get_user(user_id=1c51a65536174654945f89334229c254)` {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:01:11.911714 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f6a9188-0258-4097-a811-84f376b70c81 None admin] RBAC: Authorization granted {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:01:11.913692 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: [pid: 80377|app: 0|req: 77/154] 10.222.0.82 () {56 vars in 1177 bytes} [Tue Aug 2 10:01:11 2022] GET /identity/v3/users/1c51a65536174654945f89334229c254 => generated 289 bytes in 475 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 02 10:01:11.914668 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f6a9188-0258-4097-a811-84f376b70c81 None admin] Authenticating user token {{(pid=80377) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:01:11.950729 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 78/155] 10.222.0.82 () {58 vars in 1023 bytes} [Tue Aug 2 10:01:11 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 219 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 02 10:01:11.951774 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9464b3ab-a86f-4528-b657-ba9db7163a8b None None] REQUEST_METHOD: `POST` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:01:11.952062 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9464b3ab-a86f-4528-b657-ba9db7163a8b None None] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:01:11.952342 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9464b3ab-a86f-4528-b657-ba9db7163a8b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:01:11.992713 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.auth.core [None req-9464b3ab-a86f-4528-b657-ba9db7163a8b None None] MFA Rules not processed for user `891005fa5ff74041b75747d16cc66798`. Rule list: `[]` (Enabled: `True`). {{(pid=80378) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 02 10:01:11.994208 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.fernet_utils [None req-9464b3ab-a86f-4528-b657-ba9db7163a8b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80378) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 10:01:12.192619 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 79/156] 10.222.0.82 () {58 vars in 1023 bytes} [Tue Aug 2 10:01:11 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 241 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 02 10:01:12.193266 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9464b3ab-a86f-4528-b657-ba9db7163a8b None None] Authenticating user token {{(pid=80378) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:01:12.238106 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f6a9188-0258-4097-a811-84f376b70c81 None admin] Validating token access rules against request {{(pid=80377) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:01:12.332499 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ff893c8-22fd-43b5-a760-f2fd6d4d1ec0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '891005fa5ff74041b75747d16cc66798', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80377) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:01:12.333278 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ff893c8-22fd-43b5-a760-f2fd6d4d1ec0 None admin] REQUEST_METHOD: `GET` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:01:12.333364 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ff893c8-22fd-43b5-a760-f2fd6d4d1ec0 None admin] SCRIPT_NAME: `/identity` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:01:12.333618 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ff893c8-22fd-43b5-a760-f2fd6d4d1ec0 None admin] PATH_INFO: `/v3/projects/c1809c9e380b45ec803130fe38eeaf24` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:01:12.334934 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ff893c8-22fd-43b5-a760-f2fd6d4d1ec0 None admin] RBAC: Authorizing `identity:get_project(project_id=c1809c9e380b45ec803130fe38eeaf24)` {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:01:12.336405 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ff893c8-22fd-43b5-a760-f2fd6d4d1ec0 None admin] RBAC: Authorization granted {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:01:12.346722 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: [pid: 80377|app: 0|req: 78/157] 10.222.0.82 () {56 vars in 1186 bytes} [Tue Aug 2 10:01:11 2022] GET /identity/v3/projects/c1809c9e380b45ec803130fe38eeaf24 => generated 298 bytes in 432 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 02 10:01:12.347746 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ff893c8-22fd-43b5-a760-f2fd6d4d1ec0 None admin] Authenticating user token {{(pid=80377) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:01:12.479620 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG dbcounter [-] [80377] Writing DB stats keystone:SELECT=100 {{(pid=80377) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 10:01:12.499526 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG dbcounter [-] [80378] Writing DB stats keystone:SELECT=100 {{(pid=80378) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 10:01:12.545762 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9464b3ab-a86f-4528-b657-ba9db7163a8b None None] Validating token access rules against request {{(pid=80378) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:01:12.616612 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ff893c8-22fd-43b5-a760-f2fd6d4d1ec0 None admin] Validating token access rules against request {{(pid=80377) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:01:12.666791 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72b81b07-2282-4946-8bf9-51610015a4ee None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '891005fa5ff74041b75747d16cc66798', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80378) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:01:12.667821 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72b81b07-2282-4946-8bf9-51610015a4ee None admin] REQUEST_METHOD: `GET` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:01:12.668555 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72b81b07-2282-4946-8bf9-51610015a4ee None admin] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:01:12.669273 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72b81b07-2282-4946-8bf9-51610015a4ee None admin] PATH_INFO: `/v3/projects/c1809c9e380b45ec803130fe38eeaf24` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:01:12.672321 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72b81b07-2282-4946-8bf9-51610015a4ee None admin] RBAC: Authorizing `identity:get_project(project_id=c1809c9e380b45ec803130fe38eeaf24)` {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:01:12.675040 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72b81b07-2282-4946-8bf9-51610015a4ee None admin] RBAC: Authorization granted {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:01:12.678492 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 80/158] 10.222.0.82 () {56 vars in 1186 bytes} [Tue Aug 2 10:01:12 2022] GET /identity/v3/projects/c1809c9e380b45ec803130fe38eeaf24 => generated 298 bytes in 486 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 02 10:01:12.679786 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72b81b07-2282-4946-8bf9-51610015a4ee None admin] Authenticating user token {{(pid=80378) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:01:12.703136 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9204bf7c-7c3d-4020-9bf7-0318e6231c4d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '891005fa5ff74041b75747d16cc66798', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80377) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:01:12.704890 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9204bf7c-7c3d-4020-9bf7-0318e6231c4d None admin] REQUEST_METHOD: `GET` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:01:12.705433 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9204bf7c-7c3d-4020-9bf7-0318e6231c4d None admin] SCRIPT_NAME: `/identity` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:01:12.705952 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9204bf7c-7c3d-4020-9bf7-0318e6231c4d None admin] PATH_INFO: `/v3/roles/member` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:01:12.735008 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9204bf7c-7c3d-4020-9bf7-0318e6231c4d None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 02 10:01:12.744638 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application [None req-9204bf7c-7c3d-4020-9bf7-0318e6231c4d None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Aug 02 10:01:12.744638 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 02 10:01:12.744638 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 02 10:01:12.744638 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 02 10:01:12.744638 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 02 10:01:12.744638 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 02 10:01:12.744638 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 02 10:01:12.744638 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 02 10:01:12.744638 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 02 10:01:12.744638 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 02 10:01:12.744638 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 02 10:01:12.744638 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 02 10:01:12.744638 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 02 10:01:12.744638 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 02 10:01:12.744638 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 02 10:01:12.744638 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 02 10:01:12.745623 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 02 10:01:12.745623 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 02 10:01:12.745623 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 02 10:01:12.745623 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 02 10:01:12.745623 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 02 10:01:12.745623 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 02 10:01:12.745623 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 02 10:01:12.745623 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 02 10:01:12.745623 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 02 10:01:12.745623 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application return self.get_or_create( Aug 02 10:01:12.745623 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 02 10:01:12.745623 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application with Lock( Aug 02 10:01:12.745623 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 02 10:01:12.745623 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application return self._enter() Aug 02 10:01:12.745623 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 02 10:01:12.745623 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 02 10:01:12.746464 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 02 10:01:12.746464 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application return self.creator() Aug 02 10:01:12.746464 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 02 10:01:12.746464 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application created_value = creator( Aug 02 10:01:12.746464 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 02 10:01:12.746464 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 02 10:01:12.746464 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 02 10:01:12.746464 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 02 10:01:12.746464 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 02 10:01:12.746464 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 02 10:01:12.746464 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Aug 02 10:01:12.746464 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application Aug 02 10:01:12.759364 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: [pid: 80377|app: 0|req: 79/159] 10.222.0.82 () {56 vars in 1099 bytes} [Tue Aug 2 10:01:12 2022] GET /identity/v3/roles/member => generated 84 bytes in 412 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 02 10:01:12.760818 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9204bf7c-7c3d-4020-9bf7-0318e6231c4d None admin] Authenticating user token {{(pid=80377) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:01:12.884493 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72b81b07-2282-4946-8bf9-51610015a4ee None admin] Validating token access rules against request {{(pid=80378) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:01:12.935812 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9dffc647-50bb-4ddd-a434-23ff23541db2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '891005fa5ff74041b75747d16cc66798', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80378) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:01:12.937085 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9dffc647-50bb-4ddd-a434-23ff23541db2 None admin] REQUEST_METHOD: `GET` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:01:12.937653 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9dffc647-50bb-4ddd-a434-23ff23541db2 None admin] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:01:12.939335 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9dffc647-50bb-4ddd-a434-23ff23541db2 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:01:12.957728 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9dffc647-50bb-4ddd-a434-23ff23541db2 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 02 10:01:12.968186 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application [None req-9dffc647-50bb-4ddd-a434-23ff23541db2 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Aug 02 10:01:12.968186 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 02 10:01:12.968186 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 02 10:01:12.968186 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 02 10:01:12.968186 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 02 10:01:12.968186 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 02 10:01:12.968186 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 02 10:01:12.968186 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 02 10:01:12.968186 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 02 10:01:12.968186 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 02 10:01:12.968186 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 02 10:01:12.968186 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 02 10:01:12.968186 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 02 10:01:12.968186 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 02 10:01:12.968186 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 02 10:01:12.968186 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 02 10:01:12.969646 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 02 10:01:12.969646 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 02 10:01:12.969646 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 02 10:01:12.969646 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 02 10:01:12.969646 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 02 10:01:12.969646 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 02 10:01:12.969646 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 02 10:01:12.969646 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 02 10:01:12.969646 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 02 10:01:12.969646 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application return self.get_or_create( Aug 02 10:01:12.969646 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 02 10:01:12.969646 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application with Lock( Aug 02 10:01:12.969646 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 02 10:01:12.969646 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application return self._enter() Aug 02 10:01:12.969646 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 02 10:01:12.969646 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 02 10:01:12.972075 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 02 10:01:12.972075 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application return self.creator() Aug 02 10:01:12.972075 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 02 10:01:12.972075 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application created_value = creator( Aug 02 10:01:12.972075 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 02 10:01:12.972075 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 02 10:01:12.972075 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 02 10:01:12.972075 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 02 10:01:12.972075 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 02 10:01:12.972075 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 02 10:01:12.972075 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Aug 02 10:01:12.972075 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application Aug 02 10:01:12.976043 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 81/160] 10.222.0.82 () {56 vars in 1114 bytes} [Tue Aug 2 10:01:12 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 297 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 02 10:01:12.978400 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a063e43-2a45-4594-aa3e-928e2f348bc8 None None] REQUEST_METHOD: `GET` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:01:12.979004 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a063e43-2a45-4594-aa3e-928e2f348bc8 None None] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:01:12.979548 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a063e43-2a45-4594-aa3e-928e2f348bc8 None None] PATH_INFO: `/` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:01:12.981705 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 82/161] 10.222.0.82 () {54 vars in 926 bytes} [Tue Aug 2 10:01:12 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 02 10:01:12.982933 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a063e43-2a45-4594-aa3e-928e2f348bc8 None None] Authenticating user token {{(pid=80378) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:01:13.004532 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9204bf7c-7c3d-4020-9bf7-0318e6231c4d None admin] Validating token access rules against request {{(pid=80377) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:01:13.008410 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG dbcounter [-] [80377] Writing DB stats keystone:SELECT=100 {{(pid=80377) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 10:01:13.026512 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG dbcounter [-] [80378] Writing DB stats keystone:SELECT=100 {{(pid=80378) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 10:01:13.069480 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37bfd576-1cae-41f0-ae2b-187380631700 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '891005fa5ff74041b75747d16cc66798', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80377) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:01:13.071127 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37bfd576-1cae-41f0-ae2b-187380631700 None admin] REQUEST_METHOD: `GET` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:01:13.071946 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37bfd576-1cae-41f0-ae2b-187380631700 None admin] SCRIPT_NAME: `/identity` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:01:13.072799 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37bfd576-1cae-41f0-ae2b-187380631700 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:01:13.075027 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37bfd576-1cae-41f0-ae2b-187380631700 None admin] RBAC: Adding query filter params (role.id=237966442f05411191105bf336bc14f3, scope.project.id=c1809c9e380b45ec803130fe38eeaf24, user.id=891005fa5ff74041b75747d16cc66798) {{(pid=80377) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 02 10:01:13.075359 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37bfd576-1cae-41f0-ae2b-187380631700 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:01:13.077153 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37bfd576-1cae-41f0-ae2b-187380631700 None admin] RBAC: Authorization granted {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:01:13.086386 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: [pid: 80377|app: 0|req: 80/162] 10.222.0.82 () {56 vars in 1374 bytes} [Tue Aug 2 10:01:12 2022] GET /identity/v3/role_assignments?user.id=891005fa5ff74041b75747d16cc66798&scope.project.id=c1809c9e380b45ec803130fe38eeaf24&role.id=237966442f05411191105bf336bc14f3 => generated 608 bytes in 326 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 02 10:01:13.087307 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37bfd576-1cae-41f0-ae2b-187380631700 None admin] Authenticating user token {{(pid=80377) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:01:13.176044 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a063e43-2a45-4594-aa3e-928e2f348bc8 None None] Validating token access rules against request {{(pid=80378) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:01:13.236923 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf95b1dd-24b0-47d3-8558-cb5aba01719b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '891005fa5ff74041b75747d16cc66798', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80378) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:01:13.238963 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf95b1dd-24b0-47d3-8558-cb5aba01719b None admin] REQUEST_METHOD: `PUT` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:01:13.239554 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf95b1dd-24b0-47d3-8558-cb5aba01719b None admin] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:01:13.240043 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf95b1dd-24b0-47d3-8558-cb5aba01719b None admin] PATH_INFO: `/v3/projects/c1809c9e380b45ec803130fe38eeaf24/users/1c51a65536174654945f89334229c254/roles/444e768aa5fb44bbab6c20afad6a0218` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:01:13.242936 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf95b1dd-24b0-47d3-8558-cb5aba01719b None admin] RBAC: Authorizing `identity:create_grant(project_id=c1809c9e380b45ec803130fe38eeaf24, user_id=1c51a65536174654945f89334229c254, role_id=444e768aa5fb44bbab6c20afad6a0218)` {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:01:13.245985 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf95b1dd-24b0-47d3-8558-cb5aba01719b None admin] RBAC: Authorization granted {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:01:13.270652 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 83/163] 10.222.0.82 () {58 vars in 1439 bytes} [Tue Aug 2 10:01:12 2022] PUT /identity/v3/projects/c1809c9e380b45ec803130fe38eeaf24/users/1c51a65536174654945f89334229c254/roles/444e768aa5fb44bbab6c20afad6a0218 => generated 0 bytes in 288 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 02 10:01:13.274329 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56497888-3752-482b-974e-ef275a689033 None None] REQUEST_METHOD: `GET` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:01:13.276875 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56497888-3752-482b-974e-ef275a689033 None None] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:01:13.277883 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56497888-3752-482b-974e-ef275a689033 None None] PATH_INFO: `/` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:01:13.281262 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 84/164] 10.222.0.82 () {54 vars in 926 bytes} [Tue Aug 2 10:01:13 2022] GET /identity => generated 270 bytes in 10 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 02 10:01:13.284829 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56497888-3752-482b-974e-ef275a689033 None None] Authenticating user token {{(pid=80378) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:01:13.426627 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG dbcounter [-] [80378] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80378) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 10:01:13.465162 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56497888-3752-482b-974e-ef275a689033 None None] Validating token access rules against request {{(pid=80378) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:01:13.478738 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37bfd576-1cae-41f0-ae2b-187380631700 None admin] Validating token access rules against request {{(pid=80377) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:01:13.512976 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa8c30da-8b2f-4c51-8c96-86383d7cda5d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '891005fa5ff74041b75747d16cc66798', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80378) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:01:13.514515 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa8c30da-8b2f-4c51-8c96-86383d7cda5d None admin] REQUEST_METHOD: `GET` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:01:13.515002 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa8c30da-8b2f-4c51-8c96-86383d7cda5d None admin] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:01:13.515471 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa8c30da-8b2f-4c51-8c96-86383d7cda5d None admin] PATH_INFO: `/v3/roles` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:01:13.516433 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa8c30da-8b2f-4c51-8c96-86383d7cda5d None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=80378) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 02 10:01:13.517503 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa8c30da-8b2f-4c51-8c96-86383d7cda5d None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:01:13.519297 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa8c30da-8b2f-4c51-8c96-86383d7cda5d None admin] RBAC: Authorization granted {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:01:13.531695 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 85/165] 10.222.0.82 () {56 vars in 1111 bytes} [Tue Aug 2 10:01:13 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 249 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 02 10:01:13.534291 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-732e180d-67fd-4225-ae60-1d760d3ab2d4 None None] REQUEST_METHOD: `POST` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:01:13.534801 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-732e180d-67fd-4225-ae60-1d760d3ab2d4 None None] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:01:13.535440 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-732e180d-67fd-4225-ae60-1d760d3ab2d4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:01:13.557235 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-675711e6-902b-4f3e-8041-9b5b6b05ac8e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '891005fa5ff74041b75747d16cc66798', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80377) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:01:13.558083 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-675711e6-902b-4f3e-8041-9b5b6b05ac8e None admin] REQUEST_METHOD: `GET` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:01:13.558510 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-675711e6-902b-4f3e-8041-9b5b6b05ac8e None admin] SCRIPT_NAME: `/identity` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:01:13.558971 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-675711e6-902b-4f3e-8041-9b5b6b05ac8e None admin] PATH_INFO: `/v3/roles` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:01:13.559930 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-675711e6-902b-4f3e-8041-9b5b6b05ac8e None admin] RBAC: Adding query filter params (name=member) {{(pid=80377) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 02 10:01:13.560254 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-675711e6-902b-4f3e-8041-9b5b6b05ac8e None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:01:13.561922 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.auth.core [None req-732e180d-67fd-4225-ae60-1d760d3ab2d4 None None] MFA Rules not processed for user `891005fa5ff74041b75747d16cc66798`. Rule list: `[]` (Enabled: `True`). {{(pid=80378) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 02 10:01:13.563038 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-675711e6-902b-4f3e-8041-9b5b6b05ac8e None admin] RBAC: Authorization granted {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:01:13.564490 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.fernet_utils [None req-732e180d-67fd-4225-ae60-1d760d3ab2d4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80378) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 10:01:13.576872 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: [pid: 80377|app: 0|req: 81/166] 10.222.0.82 () {56 vars in 1101 bytes} [Tue Aug 2 10:01:13 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 490 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 02 10:01:13.580986 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a7e249c-627b-4ada-af57-b96901a77516 None None] REQUEST_METHOD: `POST` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:01:13.581516 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a7e249c-627b-4ada-af57-b96901a77516 None None] SCRIPT_NAME: `/identity` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:01:13.581977 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a7e249c-627b-4ada-af57-b96901a77516 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:01:13.610001 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.auth.core [None req-0a7e249c-627b-4ada-af57-b96901a77516 None None] MFA Rules not processed for user `891005fa5ff74041b75747d16cc66798`. Rule list: `[]` (Enabled: `True`). {{(pid=80377) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 02 10:01:13.611617 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.fernet_utils [None req-0a7e249c-627b-4ada-af57-b96901a77516 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80377) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 10:01:13.644304 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG dbcounter [-] [80377] Writing DB stats keystone:SELECT=100 {{(pid=80377) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 10:01:13.677243 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 86/167] 10.222.0.82 () {58 vars in 1023 bytes} [Tue Aug 2 10:01:13 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 145 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 02 10:01:13.678549 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-732e180d-67fd-4225-ae60-1d760d3ab2d4 None None] Authenticating user token {{(pid=80378) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:01:13.720512 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: [pid: 80377|app: 0|req: 82/168] 10.222.0.82 () {58 vars in 1023 bytes} [Tue Aug 2 10:01:13 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 143 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 02 10:01:13.721406 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a7e249c-627b-4ada-af57-b96901a77516 None None] Authenticating user token {{(pid=80377) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:01:13.773279 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG dbcounter [-] [80378] Writing DB stats keystone:SELECT=100 {{(pid=80378) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 10:01:13.822776 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-732e180d-67fd-4225-ae60-1d760d3ab2d4 None None] Validating token access rules against request {{(pid=80378) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:01:13.865451 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a7e249c-627b-4ada-af57-b96901a77516 None None] Validating token access rules against request {{(pid=80377) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:01:13.882258 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49032c20-4980-4835-abfa-985780e58135 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '891005fa5ff74041b75747d16cc66798', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80378) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:01:13.883952 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49032c20-4980-4835-abfa-985780e58135 None admin] REQUEST_METHOD: `GET` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:01:13.884857 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49032c20-4980-4835-abfa-985780e58135 None admin] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:01:13.885626 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49032c20-4980-4835-abfa-985780e58135 None admin] PATH_INFO: `/v3/users/71a4a6ffabe74df39f0a7ca7ea7b33b8` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:01:13.890561 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49032c20-4980-4835-abfa-985780e58135 None admin] RBAC: Authorizing `identity:get_user(user_id=71a4a6ffabe74df39f0a7ca7ea7b33b8)` {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:01:13.893551 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49032c20-4980-4835-abfa-985780e58135 None admin] RBAC: Authorization granted {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:01:13.897529 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 87/169] 10.222.0.82 () {56 vars in 1177 bytes} [Tue Aug 2 10:01:13 2022] GET /identity/v3/users/71a4a6ffabe74df39f0a7ca7ea7b33b8 => generated 275 bytes in 219 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 02 10:01:13.898824 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49032c20-4980-4835-abfa-985780e58135 None admin] Authenticating user token {{(pid=80378) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:01:13.911729 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c43b246-2e80-43af-9ef3-82f88229d1fe None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '891005fa5ff74041b75747d16cc66798', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80377) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:01:13.912425 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c43b246-2e80-43af-9ef3-82f88229d1fe None admin] REQUEST_METHOD: `GET` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:01:13.912712 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c43b246-2e80-43af-9ef3-82f88229d1fe None admin] SCRIPT_NAME: `/identity` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:01:13.912991 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c43b246-2e80-43af-9ef3-82f88229d1fe None admin] PATH_INFO: `/v3/users/71a4a6ffabe74df39f0a7ca7ea7b33b8` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:01:13.916696 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c43b246-2e80-43af-9ef3-82f88229d1fe None admin] RBAC: Authorizing `identity:get_user(user_id=71a4a6ffabe74df39f0a7ca7ea7b33b8)` {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:01:13.918322 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c43b246-2e80-43af-9ef3-82f88229d1fe None admin] RBAC: Authorization granted {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:01:13.921101 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: [pid: 80377|app: 0|req: 83/170] 10.222.0.82 () {56 vars in 1177 bytes} [Tue Aug 2 10:01:13 2022] GET /identity/v3/users/71a4a6ffabe74df39f0a7ca7ea7b33b8 => generated 275 bytes in 200 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 02 10:01:13.922318 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c43b246-2e80-43af-9ef3-82f88229d1fe None admin] Authenticating user token {{(pid=80377) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:01:13.956668 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG dbcounter [-] [80377] Writing DB stats keystone:SELECT=100 {{(pid=80377) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 10:01:14.055359 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c43b246-2e80-43af-9ef3-82f88229d1fe None admin] Validating token access rules against request {{(pid=80377) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:01:14.080852 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49032c20-4980-4835-abfa-985780e58135 None admin] Validating token access rules against request {{(pid=80378) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:01:14.110583 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06012c14-5898-4d76-9e22-7c708f0cea5b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '891005fa5ff74041b75747d16cc66798', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80377) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:01:14.111274 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06012c14-5898-4d76-9e22-7c708f0cea5b None admin] REQUEST_METHOD: `GET` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:01:14.111557 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06012c14-5898-4d76-9e22-7c708f0cea5b None admin] SCRIPT_NAME: `/identity` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:01:14.111835 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06012c14-5898-4d76-9e22-7c708f0cea5b None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:01:14.129025 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06012c14-5898-4d76-9e22-7c708f0cea5b None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 02 10:01:14.134400 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application [None req-06012c14-5898-4d76-9e22-7c708f0cea5b None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 02 10:01:14.134400 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 02 10:01:14.134400 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 02 10:01:14.134400 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 02 10:01:14.134400 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 02 10:01:14.134400 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 02 10:01:14.134400 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 02 10:01:14.134400 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 02 10:01:14.134400 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 02 10:01:14.134400 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 02 10:01:14.134400 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 02 10:01:14.134400 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 02 10:01:14.134400 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 02 10:01:14.134400 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 02 10:01:14.134400 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 02 10:01:14.134400 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 02 10:01:14.135679 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 02 10:01:14.135679 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 02 10:01:14.135679 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 02 10:01:14.135679 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 02 10:01:14.135679 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 02 10:01:14.135679 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 02 10:01:14.135679 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 02 10:01:14.135679 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 02 10:01:14.135679 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 02 10:01:14.135679 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application return self.get_or_create( Aug 02 10:01:14.135679 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 02 10:01:14.135679 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application with Lock( Aug 02 10:01:14.135679 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 02 10:01:14.135679 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application return self._enter() Aug 02 10:01:14.135679 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 02 10:01:14.135679 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 02 10:01:14.137285 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 02 10:01:14.137285 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application return self.creator() Aug 02 10:01:14.137285 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 02 10:01:14.137285 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application created_value = creator( Aug 02 10:01:14.137285 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 02 10:01:14.137285 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 02 10:01:14.137285 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 02 10:01:14.137285 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 02 10:01:14.137285 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 02 10:01:14.137285 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 02 10:01:14.137285 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 02 10:01:14.137285 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application Aug 02 10:01:14.137285 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: [pid: 80377|app: 0|req: 84/171] 10.222.0.82 () {56 vars in 1096 bytes} [Tue Aug 2 10:01:13 2022] GET /identity/v3/roles/admin => generated 83 bytes in 214 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 02 10:01:14.137285 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06012c14-5898-4d76-9e22-7c708f0cea5b None admin] Authenticating user token {{(pid=80377) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:01:14.162308 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-596d7ac6-1efa-4f0f-8258-7d8d4bbe061e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '891005fa5ff74041b75747d16cc66798', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80378) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:01:14.162582 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-596d7ac6-1efa-4f0f-8258-7d8d4bbe061e None admin] REQUEST_METHOD: `GET` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:01:14.163077 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-596d7ac6-1efa-4f0f-8258-7d8d4bbe061e None admin] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:01:14.163470 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-596d7ac6-1efa-4f0f-8258-7d8d4bbe061e None admin] PATH_INFO: `/v3/roles/member` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:01:14.179197 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG dbcounter [-] [80378] Writing DB stats keystone:SELECT=100 {{(pid=80378) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 10:01:14.186997 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-596d7ac6-1efa-4f0f-8258-7d8d4bbe061e None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 02 10:01:14.195965 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application [None req-596d7ac6-1efa-4f0f-8258-7d8d4bbe061e None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Aug 02 10:01:14.195965 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 02 10:01:14.195965 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 02 10:01:14.195965 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 02 10:01:14.195965 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 02 10:01:14.195965 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 02 10:01:14.195965 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 02 10:01:14.195965 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 02 10:01:14.195965 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 02 10:01:14.195965 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 02 10:01:14.195965 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 02 10:01:14.195965 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 02 10:01:14.195965 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 02 10:01:14.195965 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 02 10:01:14.195965 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 02 10:01:14.195965 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 02 10:01:14.197731 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 02 10:01:14.197731 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 02 10:01:14.197731 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 02 10:01:14.197731 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 02 10:01:14.197731 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 02 10:01:14.197731 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 02 10:01:14.197731 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 02 10:01:14.197731 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 02 10:01:14.197731 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 02 10:01:14.197731 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application return self.get_or_create( Aug 02 10:01:14.197731 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 02 10:01:14.197731 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application with Lock( Aug 02 10:01:14.197731 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 02 10:01:14.197731 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application return self._enter() Aug 02 10:01:14.197731 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 02 10:01:14.197731 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 02 10:01:14.206496 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 02 10:01:14.206496 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application return self.creator() Aug 02 10:01:14.206496 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 02 10:01:14.206496 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application created_value = creator( Aug 02 10:01:14.206496 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 02 10:01:14.206496 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 02 10:01:14.206496 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 02 10:01:14.206496 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 02 10:01:14.206496 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 02 10:01:14.206496 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 02 10:01:14.206496 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Aug 02 10:01:14.206496 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application Aug 02 10:01:14.206496 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 88/172] 10.222.0.82 () {56 vars in 1099 bytes} [Tue Aug 2 10:01:13 2022] GET /identity/v3/roles/member => generated 84 bytes in 299 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 02 10:01:14.206496 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-596d7ac6-1efa-4f0f-8258-7d8d4bbe061e None admin] Authenticating user token {{(pid=80378) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:01:14.316193 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG dbcounter [-] [80377] Writing DB stats keystone:SELECT=100 {{(pid=80377) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 10:01:14.340204 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06012c14-5898-4d76-9e22-7c708f0cea5b None admin] Validating token access rules against request {{(pid=80377) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:01:14.396808 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-596d7ac6-1efa-4f0f-8258-7d8d4bbe061e None admin] Validating token access rules against request {{(pid=80378) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:01:14.415402 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76e3eb17-be5b-4420-94b1-b21e3a69c7f1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '891005fa5ff74041b75747d16cc66798', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80377) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:01:14.416395 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76e3eb17-be5b-4420-94b1-b21e3a69c7f1 None admin] REQUEST_METHOD: `GET` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:01:14.416821 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76e3eb17-be5b-4420-94b1-b21e3a69c7f1 None admin] SCRIPT_NAME: `/identity` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:01:14.417265 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76e3eb17-be5b-4420-94b1-b21e3a69c7f1 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:01:14.461942 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff645329-452f-4b88-8ad4-4d33c37edee7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '891005fa5ff74041b75747d16cc66798', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80378) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:01:14.462906 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff645329-452f-4b88-8ad4-4d33c37edee7 None admin] REQUEST_METHOD: `GET` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:01:14.463331 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff645329-452f-4b88-8ad4-4d33c37edee7 None admin] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:01:14.463742 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff645329-452f-4b88-8ad4-4d33c37edee7 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:01:14.547623 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76e3eb17-be5b-4420-94b1-b21e3a69c7f1 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:01:14.550917 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76e3eb17-be5b-4420-94b1-b21e3a69c7f1 None admin] RBAC: Authorization granted {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:01:14.581187 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG dbcounter [-] [80378] Writing DB stats keystone:SELECT=100 {{(pid=80378) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 10:01:14.585572 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff645329-452f-4b88-8ad4-4d33c37edee7 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:01:14.588437 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff645329-452f-4b88-8ad4-4d33c37edee7 None admin] RBAC: Authorization granted {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:01:14.704967 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG dbcounter [-] [80377] Writing DB stats keystone:SELECT=100 {{(pid=80377) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 10:01:14.735309 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: [pid: 80377|app: 0|req: 85/173] 10.222.0.82 () {58 vars in 1282 bytes} [Tue Aug 2 10:01:14 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 599 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 02 10:01:14.735309 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e73fca48-3505-4f1b-ab18-cdd985b598ec None None] REQUEST_METHOD: `GET` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:01:14.735309 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e73fca48-3505-4f1b-ab18-cdd985b598ec None None] SCRIPT_NAME: `/identity` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:01:14.735309 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e73fca48-3505-4f1b-ab18-cdd985b598ec None None] PATH_INFO: `/` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:01:14.741769 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: [pid: 80377|app: 0|req: 86/174] 10.222.0.82 () {54 vars in 926 bytes} [Tue Aug 2 10:01:14 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 02 10:01:14.745567 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06f6e9e0-bf3a-4d14-90d4-146a2aff3be8 None None] REQUEST_METHOD: `GET` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:01:14.746311 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06f6e9e0-bf3a-4d14-90d4-146a2aff3be8 None None] SCRIPT_NAME: `/identity` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:01:14.746996 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06f6e9e0-bf3a-4d14-90d4-146a2aff3be8 None None] PATH_INFO: `/` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:01:14.750106 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: [pid: 80377|app: 0|req: 87/175] 10.222.0.82 () {54 vars in 926 bytes} [Tue Aug 2 10:01:14 2022] GET /identity => generated 270 bytes in 8 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 02 10:01:14.751968 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06f6e9e0-bf3a-4d14-90d4-146a2aff3be8 None None] Authenticating user token {{(pid=80377) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:01:14.772773 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 89/176] 10.222.0.82 () {58 vars in 1282 bytes} [Tue Aug 2 10:01:14 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 574 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 02 10:01:14.775062 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff645329-452f-4b88-8ad4-4d33c37edee7 None admin] Authenticating user token {{(pid=80378) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:01:14.970506 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG dbcounter [-] [80378] Writing DB stats keystone:SELECT=100 {{(pid=80378) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 10:01:14.974783 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff645329-452f-4b88-8ad4-4d33c37edee7 None admin] Validating token access rules against request {{(pid=80378) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:01:14.976262 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06f6e9e0-bf3a-4d14-90d4-146a2aff3be8 None None] Validating token access rules against request {{(pid=80377) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:01:15.045436 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6208ade2-fbbe-4c1b-8c50-28e99bac12d3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '891005fa5ff74041b75747d16cc66798', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80378) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:01:15.046562 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6208ade2-fbbe-4c1b-8c50-28e99bac12d3 None admin] REQUEST_METHOD: `GET` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:01:15.047024 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6208ade2-fbbe-4c1b-8c50-28e99bac12d3 None admin] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:01:15.047480 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6208ade2-fbbe-4c1b-8c50-28e99bac12d3 None admin] PATH_INFO: `/v3/roles` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:01:15.049266 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6208ade2-fbbe-4c1b-8c50-28e99bac12d3 None admin] RBAC: Adding query filter params (name=member) {{(pid=80378) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 02 10:01:15.049849 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6208ade2-fbbe-4c1b-8c50-28e99bac12d3 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:01:15.052173 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-533e3733-62e4-4f2d-b5eb-cd0ddc49c817 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '891005fa5ff74041b75747d16cc66798', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80377) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:01:15.052393 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6208ade2-fbbe-4c1b-8c50-28e99bac12d3 None admin] RBAC: Authorization granted {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:01:15.052806 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-533e3733-62e4-4f2d-b5eb-cd0ddc49c817 None admin] REQUEST_METHOD: `GET` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:01:15.053095 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-533e3733-62e4-4f2d-b5eb-cd0ddc49c817 None admin] SCRIPT_NAME: `/identity` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:01:15.053364 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-533e3733-62e4-4f2d-b5eb-cd0ddc49c817 None admin] PATH_INFO: `/v3/roles` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:01:15.054142 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-533e3733-62e4-4f2d-b5eb-cd0ddc49c817 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80377) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 02 10:01:15.054477 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-533e3733-62e4-4f2d-b5eb-cd0ddc49c817 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:01:15.056144 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-533e3733-62e4-4f2d-b5eb-cd0ddc49c817 None admin] RBAC: Authorization granted {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:01:15.069472 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: [pid: 80377|app: 0|req: 88/177] 10.222.0.82 () {56 vars in 1099 bytes} [Tue Aug 2 10:01:14 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 318 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 02 10:01:15.070416 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-533e3733-62e4-4f2d-b5eb-cd0ddc49c817 None admin] Authenticating user token {{(pid=80377) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:01:15.078100 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 90/178] 10.222.0.82 () {56 vars in 1101 bytes} [Tue Aug 2 10:01:14 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 300 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 02 10:01:15.078100 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36d75e0f-2358-4d06-b017-c1c57325adc3 None None] REQUEST_METHOD: `POST` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:01:15.078100 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36d75e0f-2358-4d06-b017-c1c57325adc3 None None] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:01:15.078100 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36d75e0f-2358-4d06-b017-c1c57325adc3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:01:15.105089 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.auth.core [None req-36d75e0f-2358-4d06-b017-c1c57325adc3 None None] MFA Rules not processed for user `891005fa5ff74041b75747d16cc66798`. Rule list: `[]` (Enabled: `True`). {{(pid=80378) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 02 10:01:15.105556 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.fernet_utils [None req-36d75e0f-2358-4d06-b017-c1c57325adc3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80378) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 10:01:15.178195 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG dbcounter [-] [80377] Writing DB stats keystone:SELECT=100 {{(pid=80377) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 10:01:15.231315 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 91/179] 10.222.0.82 () {58 vars in 1023 bytes} [Tue Aug 2 10:01:15 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 157 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 02 10:01:15.232779 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72551145-6660-4d17-a0f1-cb7c12044576 None None] REQUEST_METHOD: `POST` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:01:15.233198 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72551145-6660-4d17-a0f1-cb7c12044576 None None] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:01:15.233536 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72551145-6660-4d17-a0f1-cb7c12044576 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:01:15.275439 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.auth.core [None req-72551145-6660-4d17-a0f1-cb7c12044576 None None] MFA Rules not processed for user `891005fa5ff74041b75747d16cc66798`. Rule list: `[]` (Enabled: `True`). {{(pid=80378) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 02 10:01:15.277329 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.fernet_utils [None req-72551145-6660-4d17-a0f1-cb7c12044576 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80378) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 10:01:15.291595 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-533e3733-62e4-4f2d-b5eb-cd0ddc49c817 None admin] Validating token access rules against request {{(pid=80377) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:01:15.350343 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-892b28f4-b684-482d-9eaa-37d9ebff0eb7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '891005fa5ff74041b75747d16cc66798', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80377) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:01:15.351258 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-892b28f4-b684-482d-9eaa-37d9ebff0eb7 None admin] REQUEST_METHOD: `GET` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:01:15.351689 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-892b28f4-b684-482d-9eaa-37d9ebff0eb7 None admin] SCRIPT_NAME: `/identity` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:01:15.352094 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-892b28f4-b684-482d-9eaa-37d9ebff0eb7 None admin] PATH_INFO: `/v3/projects/c1809c9e380b45ec803130fe38eeaf24` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:01:15.353816 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-892b28f4-b684-482d-9eaa-37d9ebff0eb7 None admin] RBAC: Authorizing `identity:get_project(project_id=c1809c9e380b45ec803130fe38eeaf24)` {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:01:15.356013 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-892b28f4-b684-482d-9eaa-37d9ebff0eb7 None admin] RBAC: Authorization granted {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:01:15.367313 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: [pid: 80377|app: 0|req: 89/180] 10.222.0.82 () {56 vars in 1186 bytes} [Tue Aug 2 10:01:15 2022] GET /identity/v3/projects/c1809c9e380b45ec803130fe38eeaf24 => generated 298 bytes in 290 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 02 10:01:15.367313 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-892b28f4-b684-482d-9eaa-37d9ebff0eb7 None admin] Authenticating user token {{(pid=80377) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:01:15.421820 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 92/181] 10.222.0.82 () {58 vars in 1023 bytes} [Tue Aug 2 10:01:15 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 183 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 02 10:01:15.421820 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72551145-6660-4d17-a0f1-cb7c12044576 None None] Authenticating user token {{(pid=80378) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:01:15.459405 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG dbcounter [-] [80378] Writing DB stats keystone:SELECT=100 {{(pid=80378) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 10:01:15.576247 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-892b28f4-b684-482d-9eaa-37d9ebff0eb7 None admin] Validating token access rules against request {{(pid=80377) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:01:15.619031 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72551145-6660-4d17-a0f1-cb7c12044576 None None] Validating token access rules against request {{(pid=80378) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:01:15.623916 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG dbcounter [-] [80377] Writing DB stats keystone:SELECT=100 {{(pid=80377) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 10:01:15.656416 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5d7360c-94cf-4648-86e8-6cb23a118f94 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '891005fa5ff74041b75747d16cc66798', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80377) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:01:15.658313 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5d7360c-94cf-4648-86e8-6cb23a118f94 None admin] REQUEST_METHOD: `GET` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:01:15.658890 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5d7360c-94cf-4648-86e8-6cb23a118f94 None admin] SCRIPT_NAME: `/identity` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:01:15.659502 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5d7360c-94cf-4648-86e8-6cb23a118f94 None admin] PATH_INFO: `/v3/projects/c1809c9e380b45ec803130fe38eeaf24` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:01:15.662298 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5d7360c-94cf-4648-86e8-6cb23a118f94 None admin] RBAC: Authorizing `identity:get_project(project_id=c1809c9e380b45ec803130fe38eeaf24)` {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:01:15.665136 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5d7360c-94cf-4648-86e8-6cb23a118f94 None admin] RBAC: Authorization granted {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:01:15.676761 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: [pid: 80377|app: 0|req: 90/182] 10.222.0.82 () {56 vars in 1186 bytes} [Tue Aug 2 10:01:15 2022] GET /identity/v3/projects/c1809c9e380b45ec803130fe38eeaf24 => generated 298 bytes in 309 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 02 10:01:15.676761 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5d7360c-94cf-4648-86e8-6cb23a118f94 None admin] Authenticating user token {{(pid=80377) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:01:15.695115 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0881a6f-3633-4705-903c-31d86206927e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '891005fa5ff74041b75747d16cc66798', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80378) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:01:15.697001 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0881a6f-3633-4705-903c-31d86206927e None admin] REQUEST_METHOD: `GET` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:01:15.697645 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0881a6f-3633-4705-903c-31d86206927e None admin] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:01:15.698263 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0881a6f-3633-4705-903c-31d86206927e None admin] PATH_INFO: `/v3/users/4b50324ba62a45b892d69d01fa699f73` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:01:15.722837 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0881a6f-3633-4705-903c-31d86206927e None admin] RBAC: Authorizing `identity:get_user(user_id=4b50324ba62a45b892d69d01fa699f73)` {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:01:15.725267 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0881a6f-3633-4705-903c-31d86206927e None admin] RBAC: Authorization granted {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:01:15.733837 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 93/183] 10.222.0.82 () {56 vars in 1177 bytes} [Tue Aug 2 10:01:15 2022] GET /identity/v3/users/4b50324ba62a45b892d69d01fa699f73 => generated 283 bytes in 313 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 02 10:01:15.733837 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0881a6f-3633-4705-903c-31d86206927e None admin] Authenticating user token {{(pid=80378) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:01:15.877704 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5d7360c-94cf-4648-86e8-6cb23a118f94 None admin] Validating token access rules against request {{(pid=80377) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:01:15.908945 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG dbcounter [-] [80378] Writing DB stats keystone:SELECT=100 {{(pid=80378) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 10:01:15.941340 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f75e4e2-a24e-4c72-b144-37c9b602bafd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '891005fa5ff74041b75747d16cc66798', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80377) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:01:15.942425 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f75e4e2-a24e-4c72-b144-37c9b602bafd None admin] REQUEST_METHOD: `GET` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:01:15.942893 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f75e4e2-a24e-4c72-b144-37c9b602bafd None admin] SCRIPT_NAME: `/identity` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:01:15.943353 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f75e4e2-a24e-4c72-b144-37c9b602bafd None admin] PATH_INFO: `/v3/users/71a4a6ffabe74df39f0a7ca7ea7b33b8` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:01:15.944059 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0881a6f-3633-4705-903c-31d86206927e None admin] Validating token access rules against request {{(pid=80378) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:01:15.945377 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f75e4e2-a24e-4c72-b144-37c9b602bafd None admin] RBAC: Authorizing `identity:get_user(user_id=71a4a6ffabe74df39f0a7ca7ea7b33b8)` {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:01:15.948203 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f75e4e2-a24e-4c72-b144-37c9b602bafd None admin] RBAC: Authorization granted {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:01:15.951588 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: [pid: 80377|app: 0|req: 91/184] 10.222.0.82 () {56 vars in 1177 bytes} [Tue Aug 2 10:01:15 2022] GET /identity/v3/users/71a4a6ffabe74df39f0a7ca7ea7b33b8 => generated 275 bytes in 282 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 02 10:01:15.953241 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f75e4e2-a24e-4c72-b144-37c9b602bafd None admin] Authenticating user token {{(pid=80377) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:01:16.017415 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6da419d0-e8de-4e63-8e3f-1dbbcadf8ae7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '891005fa5ff74041b75747d16cc66798', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80378) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:01:16.018129 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6da419d0-e8de-4e63-8e3f-1dbbcadf8ae7 None admin] REQUEST_METHOD: `GET` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:01:16.018412 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6da419d0-e8de-4e63-8e3f-1dbbcadf8ae7 None admin] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:01:16.018730 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6da419d0-e8de-4e63-8e3f-1dbbcadf8ae7 None admin] PATH_INFO: `/v3/domains/default` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:01:16.020018 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6da419d0-e8de-4e63-8e3f-1dbbcadf8ae7 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:01:16.021488 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6da419d0-e8de-4e63-8e3f-1dbbcadf8ae7 None admin] RBAC: Authorization granted {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:01:16.024168 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 94/185] 10.222.0.82 () {56 vars in 1108 bytes} [Tue Aug 2 10:01:15 2022] GET /identity/v3/domains/default => generated 199 bytes in 295 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 02 10:01:16.026592 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6da419d0-e8de-4e63-8e3f-1dbbcadf8ae7 None admin] Authenticating user token {{(pid=80378) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:01:16.063164 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG dbcounter [-] [80377] Writing DB stats keystone:SELECT=100 {{(pid=80377) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 10:01:16.166177 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f75e4e2-a24e-4c72-b144-37c9b602bafd None admin] Validating token access rules against request {{(pid=80377) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:01:16.234494 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6da419d0-e8de-4e63-8e3f-1dbbcadf8ae7 None admin] Validating token access rules against request {{(pid=80378) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:01:16.238861 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04c42de7-137c-4fef-8676-58a73ab6cb3c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '891005fa5ff74041b75747d16cc66798', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80377) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:01:16.239600 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04c42de7-137c-4fef-8676-58a73ab6cb3c None admin] REQUEST_METHOD: `GET` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:01:16.239931 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04c42de7-137c-4fef-8676-58a73ab6cb3c None admin] SCRIPT_NAME: `/identity` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:01:16.240272 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04c42de7-137c-4fef-8676-58a73ab6cb3c None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:01:16.241291 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04c42de7-137c-4fef-8676-58a73ab6cb3c None admin] RBAC: Adding query filter params (role.id=649993d933d44c50b268df7fdf336fbf, scope.project.id=c1809c9e380b45ec803130fe38eeaf24, user.id=71a4a6ffabe74df39f0a7ca7ea7b33b8) {{(pid=80377) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 02 10:01:16.241764 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04c42de7-137c-4fef-8676-58a73ab6cb3c None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:01:16.243810 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04c42de7-137c-4fef-8676-58a73ab6cb3c None admin] RBAC: Authorization granted {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:01:16.257888 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: [pid: 80377|app: 0|req: 92/186] 10.222.0.82 () {56 vars in 1374 bytes} [Tue Aug 2 10:01:15 2022] GET /identity/v3/role_assignments?user.id=71a4a6ffabe74df39f0a7ca7ea7b33b8&scope.project.id=c1809c9e380b45ec803130fe38eeaf24&role.id=649993d933d44c50b268df7fdf336fbf => generated 608 bytes in 301 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 02 10:01:16.257888 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04c42de7-137c-4fef-8676-58a73ab6cb3c None admin] Authenticating user token {{(pid=80377) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:01:16.305081 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cec55c4d-25f0-4462-83b2-e2ebd9ee92a9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '891005fa5ff74041b75747d16cc66798', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80378) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:01:16.305354 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cec55c4d-25f0-4462-83b2-e2ebd9ee92a9 None admin] REQUEST_METHOD: `GET` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:01:16.305885 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cec55c4d-25f0-4462-83b2-e2ebd9ee92a9 None admin] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:01:16.306456 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cec55c4d-25f0-4462-83b2-e2ebd9ee92a9 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:01:16.331341 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cec55c4d-25f0-4462-83b2-e2ebd9ee92a9 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 02 10:01:16.341392 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application [None req-cec55c4d-25f0-4462-83b2-e2ebd9ee92a9 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Aug 02 10:01:16.341392 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 02 10:01:16.341392 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 02 10:01:16.341392 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 02 10:01:16.341392 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 02 10:01:16.341392 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 02 10:01:16.341392 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 02 10:01:16.341392 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 02 10:01:16.341392 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 02 10:01:16.341392 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 02 10:01:16.341392 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 02 10:01:16.341392 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 02 10:01:16.341392 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 02 10:01:16.341392 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 02 10:01:16.341392 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 02 10:01:16.341392 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 02 10:01:16.342784 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 02 10:01:16.342784 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 02 10:01:16.342784 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 02 10:01:16.342784 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 02 10:01:16.342784 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 02 10:01:16.342784 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 02 10:01:16.342784 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 02 10:01:16.342784 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 02 10:01:16.342784 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 02 10:01:16.342784 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application return self.get_or_create( Aug 02 10:01:16.342784 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 02 10:01:16.342784 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application with Lock( Aug 02 10:01:16.342784 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 02 10:01:16.342784 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application return self._enter() Aug 02 10:01:16.342784 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 02 10:01:16.342784 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 02 10:01:16.344338 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 02 10:01:16.344338 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application return self.creator() Aug 02 10:01:16.344338 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 02 10:01:16.344338 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application created_value = creator( Aug 02 10:01:16.344338 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 02 10:01:16.344338 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 02 10:01:16.344338 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 02 10:01:16.344338 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 02 10:01:16.344338 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 02 10:01:16.344338 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 02 10:01:16.344338 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Aug 02 10:01:16.344338 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application Aug 02 10:01:16.344338 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 95/187] 10.222.0.82 () {56 vars in 1114 bytes} [Tue Aug 2 10:01:16 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 319 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 02 10:01:16.351326 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cec55c4d-25f0-4462-83b2-e2ebd9ee92a9 None admin] Authenticating user token {{(pid=80378) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:01:16.375892 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG dbcounter [-] [80378] Writing DB stats keystone:SELECT=100 {{(pid=80378) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 10:01:16.487914 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04c42de7-137c-4fef-8676-58a73ab6cb3c None admin] Validating token access rules against request {{(pid=80377) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:01:16.535520 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG dbcounter [-] [80377] Writing DB stats keystone:SELECT=100 {{(pid=80377) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 10:01:16.577145 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-803e165f-4f02-4e82-9c78-9d3ebd550e45 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '891005fa5ff74041b75747d16cc66798', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80377) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:01:16.583836 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-803e165f-4f02-4e82-9c78-9d3ebd550e45 None admin] REQUEST_METHOD: `GET` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:01:16.583836 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-803e165f-4f02-4e82-9c78-9d3ebd550e45 None admin] SCRIPT_NAME: `/identity` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:01:16.583836 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-803e165f-4f02-4e82-9c78-9d3ebd550e45 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:01:16.584123 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cec55c4d-25f0-4462-83b2-e2ebd9ee92a9 None admin] Validating token access rules against request {{(pid=80378) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:01:16.584202 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-803e165f-4f02-4e82-9c78-9d3ebd550e45 None admin] RBAC: Adding query filter params (role.id=e1c04b19fc4543b8b54f58bbcffbc598, scope.project.id=c1809c9e380b45ec803130fe38eeaf24, user.id=71a4a6ffabe74df39f0a7ca7ea7b33b8) {{(pid=80377) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 02 10:01:16.584202 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-803e165f-4f02-4e82-9c78-9d3ebd550e45 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:01:16.588027 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-803e165f-4f02-4e82-9c78-9d3ebd550e45 None admin] RBAC: Authorization granted {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:01:16.596365 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: [pid: 80377|app: 0|req: 93/188] 10.222.0.82 () {56 vars in 1374 bytes} [Tue Aug 2 10:01:16 2022] GET /identity/v3/role_assignments?user.id=71a4a6ffabe74df39f0a7ca7ea7b33b8&scope.project.id=c1809c9e380b45ec803130fe38eeaf24&role.id=e1c04b19fc4543b8b54f58bbcffbc598 => generated 608 bytes in 343 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 02 10:01:16.597846 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-803e165f-4f02-4e82-9c78-9d3ebd550e45 None admin] Authenticating user token {{(pid=80377) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:01:16.661817 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ded385f-a4ad-4156-ae07-6d45f25ef69a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '891005fa5ff74041b75747d16cc66798', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80378) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:01:16.662830 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ded385f-a4ad-4156-ae07-6d45f25ef69a None admin] REQUEST_METHOD: `GET` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:01:16.663134 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ded385f-a4ad-4156-ae07-6d45f25ef69a None admin] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:01:16.663457 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ded385f-a4ad-4156-ae07-6d45f25ef69a None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:01:16.772529 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ded385f-a4ad-4156-ae07-6d45f25ef69a None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:01:16.775265 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ded385f-a4ad-4156-ae07-6d45f25ef69a None admin] RBAC: Authorization granted {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:01:16.780056 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-803e165f-4f02-4e82-9c78-9d3ebd550e45 None admin] Validating token access rules against request {{(pid=80377) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:01:16.797652 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG dbcounter [-] [80378] Writing DB stats keystone:SELECT=100 {{(pid=80378) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 10:01:16.843119 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58b89943-6c1c-4ea8-aaa2-7c032d57ee8c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '891005fa5ff74041b75747d16cc66798', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80377) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:01:16.844005 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58b89943-6c1c-4ea8-aaa2-7c032d57ee8c None admin] REQUEST_METHOD: `GET` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:01:16.844427 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58b89943-6c1c-4ea8-aaa2-7c032d57ee8c None admin] SCRIPT_NAME: `/identity` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:01:16.845397 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58b89943-6c1c-4ea8-aaa2-7c032d57ee8c None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:01:16.934468 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG dbcounter [-] [80377] Writing DB stats keystone:SELECT=100 {{(pid=80377) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 10:01:16.958300 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58b89943-6c1c-4ea8-aaa2-7c032d57ee8c None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:01:16.959794 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58b89943-6c1c-4ea8-aaa2-7c032d57ee8c None admin] RBAC: Authorization granted {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:01:16.971166 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 96/189] 10.222.0.82 () {58 vars in 1282 bytes} [Tue Aug 2 10:01:16 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 627 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 02 10:01:16.971998 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ded385f-a4ad-4156-ae07-6d45f25ef69a None admin] Authenticating user token {{(pid=80378) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:01:17.113175 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: [pid: 80377|app: 0|req: 94/190] 10.222.0.82 () {58 vars in 1282 bytes} [Tue Aug 2 10:01:16 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 516 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 02 10:01:17.114375 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58b89943-6c1c-4ea8-aaa2-7c032d57ee8c None admin] Authenticating user token {{(pid=80377) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:01:17.115109 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ded385f-a4ad-4156-ae07-6d45f25ef69a None admin] Validating token access rules against request {{(pid=80378) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:01:17.145956 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG dbcounter [-] [80378] Writing DB stats keystone:SELECT=100 {{(pid=80378) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 10:01:17.179483 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36d64745-61e6-4ed5-a4f7-83fc05333976 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '891005fa5ff74041b75747d16cc66798', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80378) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:01:17.180426 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36d64745-61e6-4ed5-a4f7-83fc05333976 None admin] REQUEST_METHOD: `POST` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:01:17.180990 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36d64745-61e6-4ed5-a4f7-83fc05333976 None admin] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:01:17.181339 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36d64745-61e6-4ed5-a4f7-83fc05333976 None admin] PATH_INFO: `/v3/users` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:01:17.182152 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36d64745-61e6-4ed5-a4f7-83fc05333976 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:01:17.189419 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36d64745-61e6-4ed5-a4f7-83fc05333976 None admin] RBAC: Authorization granted {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:01:17.214724 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 97/191] 10.222.0.82 () {60 vars in 1132 bytes} [Tue Aug 2 10:01:16 2022] POST /identity/v3/users => generated 297 bytes in 243 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 02 10:01:17.216697 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fa05451-886a-42f3-b6dd-918b87917925 None None] REQUEST_METHOD: `GET` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:01:17.217190 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fa05451-886a-42f3-b6dd-918b87917925 None None] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:01:17.217682 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fa05451-886a-42f3-b6dd-918b87917925 None None] PATH_INFO: `/` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:01:17.224119 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 98/192] 10.222.0.82 () {54 vars in 926 bytes} [Tue Aug 2 10:01:17 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 02 10:01:17.224119 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fa05451-886a-42f3-b6dd-918b87917925 None None] Authenticating user token {{(pid=80378) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:01:17.289540 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG dbcounter [-] [80377] Writing DB stats keystone:SELECT=100 {{(pid=80377) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 10:01:17.336661 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58b89943-6c1c-4ea8-aaa2-7c032d57ee8c None admin] Validating token access rules against request {{(pid=80377) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:01:17.406822 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fa05451-886a-42f3-b6dd-918b87917925 None None] Validating token access rules against request {{(pid=80378) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:01:17.407026 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf948619-cb34-4e69-a57b-bc8a29b634e7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '891005fa5ff74041b75747d16cc66798', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80377) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:01:17.407517 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf948619-cb34-4e69-a57b-bc8a29b634e7 None admin] REQUEST_METHOD: `GET` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:01:17.407954 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf948619-cb34-4e69-a57b-bc8a29b634e7 None admin] SCRIPT_NAME: `/identity` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:01:17.408228 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf948619-cb34-4e69-a57b-bc8a29b634e7 None admin] PATH_INFO: `/v3/roles` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:01:17.409073 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf948619-cb34-4e69-a57b-bc8a29b634e7 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=80377) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 02 10:01:17.409442 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf948619-cb34-4e69-a57b-bc8a29b634e7 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:01:17.410956 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf948619-cb34-4e69-a57b-bc8a29b634e7 None admin] RBAC: Authorization granted {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:01:17.425873 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: [pid: 80377|app: 0|req: 95/193] 10.222.0.82 () {56 vars in 1111 bytes} [Tue Aug 2 10:01:17 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 312 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 02 10:01:17.427983 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf948619-cb34-4e69-a57b-bc8a29b634e7 None admin] Authenticating user token {{(pid=80377) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:01:17.462903 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f229c291-6ada-49fd-9232-8d8850a8b358 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '891005fa5ff74041b75747d16cc66798', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80378) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:01:17.464031 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f229c291-6ada-49fd-9232-8d8850a8b358 None admin] REQUEST_METHOD: `GET` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:01:17.464497 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f229c291-6ada-49fd-9232-8d8850a8b358 None admin] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:01:17.464963 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f229c291-6ada-49fd-9232-8d8850a8b358 None admin] PATH_INFO: `/v3/projects/f7db7cf528494fbb8836c08d79b752ef` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:01:17.467636 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f229c291-6ada-49fd-9232-8d8850a8b358 None admin] RBAC: Authorizing `identity:get_project(project_id=f7db7cf528494fbb8836c08d79b752ef)` {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:01:17.471054 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f229c291-6ada-49fd-9232-8d8850a8b358 None admin] RBAC: Authorization granted {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:01:17.474607 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 99/194] 10.222.0.82 () {56 vars in 1186 bytes} [Tue Aug 2 10:01:17 2022] GET /identity/v3/projects/f7db7cf528494fbb8836c08d79b752ef => generated 302 bytes in 253 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 02 10:01:17.476497 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d938113-40f1-487c-ab96-07c8a14ffbaf None None] REQUEST_METHOD: `POST` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:01:17.476946 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d938113-40f1-487c-ab96-07c8a14ffbaf None None] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:01:17.477356 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d938113-40f1-487c-ab96-07c8a14ffbaf None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:01:17.501966 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.auth.core [None req-2d938113-40f1-487c-ab96-07c8a14ffbaf None None] MFA Rules not processed for user `891005fa5ff74041b75747d16cc66798`. Rule list: `[]` (Enabled: `True`). {{(pid=80378) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 02 10:01:17.503190 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.fernet_utils [None req-2d938113-40f1-487c-ab96-07c8a14ffbaf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80378) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 10:01:17.561362 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG dbcounter [-] [80378] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80378) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 10:01:17.638691 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf948619-cb34-4e69-a57b-bc8a29b634e7 None admin] Validating token access rules against request {{(pid=80377) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:01:17.640938 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 100/195] 10.222.0.82 () {58 vars in 1023 bytes} [Tue Aug 2 10:01:17 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 166 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 02 10:01:17.642963 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d938113-40f1-487c-ab96-07c8a14ffbaf None None] Authenticating user token {{(pid=80378) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:01:17.703461 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe10d48f-17f6-407c-b9e9-4ceb9043c5ef None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '891005fa5ff74041b75747d16cc66798', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80377) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:01:17.704034 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe10d48f-17f6-407c-b9e9-4ceb9043c5ef None admin] REQUEST_METHOD: `GET` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:01:17.704954 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe10d48f-17f6-407c-b9e9-4ceb9043c5ef None admin] SCRIPT_NAME: `/identity` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:01:17.705248 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe10d48f-17f6-407c-b9e9-4ceb9043c5ef None admin] PATH_INFO: `/v3/projects/354cc945e1c64e598aefaee72be3cffc` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:01:17.707928 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe10d48f-17f6-407c-b9e9-4ceb9043c5ef None admin] RBAC: Authorizing `identity:get_project(project_id=354cc945e1c64e598aefaee72be3cffc)` {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:01:17.710083 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe10d48f-17f6-407c-b9e9-4ceb9043c5ef None admin] RBAC: Authorization granted {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:01:17.712031 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: [pid: 80377|app: 0|req: 96/196] 10.222.0.82 () {56 vars in 1186 bytes} [Tue Aug 2 10:01:17 2022] GET /identity/v3/projects/354cc945e1c64e598aefaee72be3cffc => generated 312 bytes in 285 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 02 10:01:17.713026 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe10d48f-17f6-407c-b9e9-4ceb9043c5ef None admin] Authenticating user token {{(pid=80377) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:01:17.738454 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG dbcounter [-] [80377] Writing DB stats keystone:SELECT=100 {{(pid=80377) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 10:01:17.797018 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d938113-40f1-487c-ab96-07c8a14ffbaf None None] Validating token access rules against request {{(pid=80378) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:01:17.842050 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df80a14c-685f-4393-b650-884e4f2eedf7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '891005fa5ff74041b75747d16cc66798', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80378) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:01:17.843000 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df80a14c-685f-4393-b650-884e4f2eedf7 None admin] REQUEST_METHOD: `GET` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:01:17.843402 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df80a14c-685f-4393-b650-884e4f2eedf7 None admin] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:01:17.843794 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df80a14c-685f-4393-b650-884e4f2eedf7 None admin] PATH_INFO: `/v3/users/4b50324ba62a45b892d69d01fa699f73` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:01:17.846538 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df80a14c-685f-4393-b650-884e4f2eedf7 None admin] RBAC: Authorizing `identity:get_user(user_id=4b50324ba62a45b892d69d01fa699f73)` {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:01:17.848631 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df80a14c-685f-4393-b650-884e4f2eedf7 None admin] RBAC: Authorization granted {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:01:17.851699 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 101/197] 10.222.0.82 () {56 vars in 1177 bytes} [Tue Aug 2 10:01:17 2022] GET /identity/v3/users/4b50324ba62a45b892d69d01fa699f73 => generated 283 bytes in 210 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 02 10:01:17.852883 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df80a14c-685f-4393-b650-884e4f2eedf7 None admin] Authenticating user token {{(pid=80378) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:01:17.907952 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe10d48f-17f6-407c-b9e9-4ceb9043c5ef None admin] Validating token access rules against request {{(pid=80377) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:01:17.921697 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG dbcounter [-] [80378] Writing DB stats keystone:SELECT=100 {{(pid=80378) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 10:01:17.972474 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2950c0f7-aaad-4de1-ba66-e0bdae1386f4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '891005fa5ff74041b75747d16cc66798', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80377) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:01:17.974251 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2950c0f7-aaad-4de1-ba66-e0bdae1386f4 None admin] REQUEST_METHOD: `GET` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:01:17.974797 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2950c0f7-aaad-4de1-ba66-e0bdae1386f4 None admin] SCRIPT_NAME: `/identity` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:01:17.975272 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2950c0f7-aaad-4de1-ba66-e0bdae1386f4 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:01:17.976653 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2950c0f7-aaad-4de1-ba66-e0bdae1386f4 None admin] RBAC: Adding query filter params (role.id=237966442f05411191105bf336bc14f3, scope.project.id=f7db7cf528494fbb8836c08d79b752ef, user.id=4b50324ba62a45b892d69d01fa699f73) {{(pid=80377) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 02 10:01:17.977151 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2950c0f7-aaad-4de1-ba66-e0bdae1386f4 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:01:17.979805 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2950c0f7-aaad-4de1-ba66-e0bdae1386f4 None admin] RBAC: Authorization granted {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:01:17.991270 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: [pid: 80377|app: 0|req: 97/198] 10.222.0.82 () {56 vars in 1374 bytes} [Tue Aug 2 10:01:17 2022] GET /identity/v3/role_assignments?user.id=4b50324ba62a45b892d69d01fa699f73&scope.project.id=f7db7cf528494fbb8836c08d79b752ef&role.id=237966442f05411191105bf336bc14f3 => generated 259 bytes in 279 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 02 10:01:17.992739 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2950c0f7-aaad-4de1-ba66-e0bdae1386f4 None admin] Authenticating user token {{(pid=80377) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:01:18.040133 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df80a14c-685f-4393-b650-884e4f2eedf7 None admin] Validating token access rules against request {{(pid=80378) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:01:18.119124 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7bcdd62-8999-4c7d-9858-55943bed33bc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '891005fa5ff74041b75747d16cc66798', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80378) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:01:18.120259 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7bcdd62-8999-4c7d-9858-55943bed33bc None admin] REQUEST_METHOD: `GET` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:01:18.120754 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7bcdd62-8999-4c7d-9858-55943bed33bc None admin] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:01:18.121239 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7bcdd62-8999-4c7d-9858-55943bed33bc None admin] PATH_INFO: `/v3/roles/reader` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:01:18.130186 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG dbcounter [-] [80377] Writing DB stats keystone:SELECT=100 {{(pid=80377) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 10:01:18.164534 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2950c0f7-aaad-4de1-ba66-e0bdae1386f4 None admin] Validating token access rules against request {{(pid=80377) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:01:18.176292 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7bcdd62-8999-4c7d-9858-55943bed33bc None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 02 10:01:18.189215 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application [None req-f7bcdd62-8999-4c7d-9858-55943bed33bc None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Aug 02 10:01:18.189215 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 02 10:01:18.189215 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 02 10:01:18.189215 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 02 10:01:18.189215 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 02 10:01:18.189215 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 02 10:01:18.189215 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 02 10:01:18.189215 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 02 10:01:18.189215 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 02 10:01:18.189215 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 02 10:01:18.189215 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 02 10:01:18.189215 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 02 10:01:18.189215 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 02 10:01:18.189215 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 02 10:01:18.189215 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 02 10:01:18.189215 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 02 10:01:18.190947 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 02 10:01:18.190947 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 02 10:01:18.190947 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 02 10:01:18.190947 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 02 10:01:18.190947 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 02 10:01:18.190947 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 02 10:01:18.190947 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 02 10:01:18.190947 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 02 10:01:18.190947 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 02 10:01:18.190947 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application return self.get_or_create( Aug 02 10:01:18.190947 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 02 10:01:18.190947 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application with Lock( Aug 02 10:01:18.190947 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 02 10:01:18.190947 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application return self._enter() Aug 02 10:01:18.190947 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 02 10:01:18.190947 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 02 10:01:18.191776 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 02 10:01:18.191776 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application return self.creator() Aug 02 10:01:18.191776 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 02 10:01:18.191776 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application created_value = creator( Aug 02 10:01:18.191776 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 02 10:01:18.191776 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 02 10:01:18.191776 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 02 10:01:18.191776 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 02 10:01:18.191776 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 02 10:01:18.191776 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 02 10:01:18.191776 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Aug 02 10:01:18.191776 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application Aug 02 10:01:18.199570 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 102/199] 10.222.0.82 () {56 vars in 1099 bytes} [Tue Aug 2 10:01:17 2022] GET /identity/v3/roles/reader => generated 84 bytes in 347 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 02 10:01:18.200892 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7bcdd62-8999-4c7d-9858-55943bed33bc None admin] Authenticating user token {{(pid=80378) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:01:18.260250 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f5a66f2-363c-488a-a077-bca844526582 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '891005fa5ff74041b75747d16cc66798', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80377) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:01:18.262949 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f5a66f2-363c-488a-a077-bca844526582 None admin] REQUEST_METHOD: `GET` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:01:18.266485 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f5a66f2-363c-488a-a077-bca844526582 None admin] SCRIPT_NAME: `/identity` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:01:18.267447 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f5a66f2-363c-488a-a077-bca844526582 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:01:18.269301 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f5a66f2-363c-488a-a077-bca844526582 None admin] RBAC: Adding query filter params (role.id=e1c04b19fc4543b8b54f58bbcffbc598, scope.project.id=354cc945e1c64e598aefaee72be3cffc, user.id=71a4a6ffabe74df39f0a7ca7ea7b33b8) {{(pid=80377) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 02 10:01:18.272211 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f5a66f2-363c-488a-a077-bca844526582 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:01:18.274199 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f5a66f2-363c-488a-a077-bca844526582 None admin] RBAC: Authorization granted {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:01:18.283555 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: [pid: 80377|app: 0|req: 98/200] 10.222.0.82 () {56 vars in 1374 bytes} [Tue Aug 2 10:01:17 2022] GET /identity/v3/role_assignments?user.id=71a4a6ffabe74df39f0a7ca7ea7b33b8&scope.project.id=354cc945e1c64e598aefaee72be3cffc&role.id=e1c04b19fc4543b8b54f58bbcffbc598 => generated 608 bytes in 292 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 02 10:01:18.285001 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f5a66f2-363c-488a-a077-bca844526582 None admin] Authenticating user token {{(pid=80377) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:01:18.506698 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG dbcounter [-] [80378] Writing DB stats keystone:SELECT=100 {{(pid=80378) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 10:01:18.540595 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7bcdd62-8999-4c7d-9858-55943bed33bc None admin] Validating token access rules against request {{(pid=80378) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:01:18.648186 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74daea2c-5a28-43dd-b8f8-d6a435d85293 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '891005fa5ff74041b75747d16cc66798', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80378) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:01:18.649198 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74daea2c-5a28-43dd-b8f8-d6a435d85293 None admin] REQUEST_METHOD: `GET` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:01:18.649700 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74daea2c-5a28-43dd-b8f8-d6a435d85293 None admin] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:01:18.650220 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74daea2c-5a28-43dd-b8f8-d6a435d85293 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:01:18.651962 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f5a66f2-363c-488a-a077-bca844526582 None admin] Validating token access rules against request {{(pid=80377) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:01:18.776764 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2eb36dff-82e3-4eb7-9149-233cb55542cb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '891005fa5ff74041b75747d16cc66798', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80377) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:01:18.777855 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2eb36dff-82e3-4eb7-9149-233cb55542cb None admin] REQUEST_METHOD: `GET` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:01:18.778361 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2eb36dff-82e3-4eb7-9149-233cb55542cb None admin] SCRIPT_NAME: `/identity` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:01:18.778846 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2eb36dff-82e3-4eb7-9149-233cb55542cb None admin] PATH_INFO: `/v3/roles` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:01:18.780152 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2eb36dff-82e3-4eb7-9149-233cb55542cb None admin] RBAC: Adding query filter params (name=reader) {{(pid=80377) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 02 10:01:18.780973 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2eb36dff-82e3-4eb7-9149-233cb55542cb None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:01:18.784133 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2eb36dff-82e3-4eb7-9149-233cb55542cb None admin] RBAC: Authorization granted {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:01:18.830771 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: [pid: 80377|app: 0|req: 99/201] 10.222.0.82 () {56 vars in 1101 bytes} [Tue Aug 2 10:01:18 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 541 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 02 10:01:18.832966 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2eb36dff-82e3-4eb7-9149-233cb55542cb None admin] Authenticating user token {{(pid=80377) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:01:18.841823 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG dbcounter [-] [80377] Writing DB stats keystone:SELECT=100 {{(pid=80377) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 10:01:18.912005 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74daea2c-5a28-43dd-b8f8-d6a435d85293 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:01:18.913508 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74daea2c-5a28-43dd-b8f8-d6a435d85293 None admin] RBAC: Authorization granted {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:01:19.097831 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2eb36dff-82e3-4eb7-9149-233cb55542cb None admin] Validating token access rules against request {{(pid=80377) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:01:19.111030 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG dbcounter [-] [80378] Writing DB stats keystone:SELECT=100 {{(pid=80378) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 10:01:19.127522 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 103/202] 10.222.0.82 () {58 vars in 1282 bytes} [Tue Aug 2 10:01:18 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 928 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 02 10:01:19.136336 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74daea2c-5a28-43dd-b8f8-d6a435d85293 None admin] Authenticating user token {{(pid=80378) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:01:19.179127 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ece2949a-75f2-42c1-8c59-6e6e8fbf8464 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '891005fa5ff74041b75747d16cc66798', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80377) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:01:19.180927 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ece2949a-75f2-42c1-8c59-6e6e8fbf8464 None admin] REQUEST_METHOD: `GET` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:01:19.181760 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ece2949a-75f2-42c1-8c59-6e6e8fbf8464 None admin] SCRIPT_NAME: `/identity` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:01:19.182571 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ece2949a-75f2-42c1-8c59-6e6e8fbf8464 None admin] PATH_INFO: `/v3/users/1c51a65536174654945f89334229c254` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:01:19.191243 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ece2949a-75f2-42c1-8c59-6e6e8fbf8464 None admin] RBAC: Authorizing `identity:get_user(user_id=1c51a65536174654945f89334229c254)` {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:01:19.194441 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ece2949a-75f2-42c1-8c59-6e6e8fbf8464 None admin] RBAC: Authorization granted {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:01:19.198385 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: [pid: 80377|app: 0|req: 100/203] 10.222.0.82 () {56 vars in 1177 bytes} [Tue Aug 2 10:01:18 2022] GET /identity/v3/users/1c51a65536174654945f89334229c254 => generated 289 bytes in 366 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 02 10:01:19.207798 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ece2949a-75f2-42c1-8c59-6e6e8fbf8464 None admin] Authenticating user token {{(pid=80377) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:01:19.297001 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74daea2c-5a28-43dd-b8f8-d6a435d85293 None admin] Validating token access rules against request {{(pid=80378) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:01:19.350650 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a89338a3-4420-41f0-a98a-ed65aa569f2e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '891005fa5ff74041b75747d16cc66798', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80378) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:01:19.351328 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a89338a3-4420-41f0-a98a-ed65aa569f2e None admin] REQUEST_METHOD: `GET` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:01:19.351694 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a89338a3-4420-41f0-a98a-ed65aa569f2e None admin] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:01:19.351974 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a89338a3-4420-41f0-a98a-ed65aa569f2e None admin] PATH_INFO: `/v3/projects/f7db7cf528494fbb8836c08d79b752ef` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:01:19.354910 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a89338a3-4420-41f0-a98a-ed65aa569f2e None admin] RBAC: Authorizing `identity:get_project(project_id=f7db7cf528494fbb8836c08d79b752ef)` {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:01:19.357355 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a89338a3-4420-41f0-a98a-ed65aa569f2e None admin] RBAC: Authorization granted {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:01:19.363496 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 104/204] 10.222.0.82 () {56 vars in 1186 bytes} [Tue Aug 2 10:01:19 2022] GET /identity/v3/projects/f7db7cf528494fbb8836c08d79b752ef => generated 302 bytes in 228 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 02 10:01:19.371394 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a89338a3-4420-41f0-a98a-ed65aa569f2e None admin] Authenticating user token {{(pid=80378) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:01:19.390351 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG dbcounter [-] [80377] Writing DB stats keystone:SELECT=100 {{(pid=80377) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 10:01:19.440871 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ece2949a-75f2-42c1-8c59-6e6e8fbf8464 None admin] Validating token access rules against request {{(pid=80377) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:01:19.509546 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf9ecb80-4e65-476c-bcb7-216a1e4f11b6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '891005fa5ff74041b75747d16cc66798', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80377) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:01:19.511148 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf9ecb80-4e65-476c-bcb7-216a1e4f11b6 None admin] REQUEST_METHOD: `GET` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:01:19.512077 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf9ecb80-4e65-476c-bcb7-216a1e4f11b6 None admin] SCRIPT_NAME: `/identity` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:01:19.512865 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf9ecb80-4e65-476c-bcb7-216a1e4f11b6 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:01:19.580554 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG dbcounter [-] [80378] Writing DB stats keystone:SELECT=100 {{(pid=80378) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 10:01:19.649614 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf9ecb80-4e65-476c-bcb7-216a1e4f11b6 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:01:19.652762 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf9ecb80-4e65-476c-bcb7-216a1e4f11b6 None admin] RBAC: Authorization granted {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:01:19.755233 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a89338a3-4420-41f0-a98a-ed65aa569f2e None admin] Validating token access rules against request {{(pid=80378) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:01:19.789943 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG dbcounter [-] [80377] Writing DB stats keystone:SELECT=100 {{(pid=80377) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 10:01:19.824885 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: [pid: 80377|app: 0|req: 101/205] 10.222.0.82 () {58 vars in 1282 bytes} [Tue Aug 2 10:01:19 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 618 msecs (HTTP/1.1 200) 6 headers in 358 bytes (2 switches on core 0) Aug 02 10:01:19.835292 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf9ecb80-4e65-476c-bcb7-216a1e4f11b6 None admin] Authenticating user token {{(pid=80377) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:01:19.926916 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-539de3f6-0d88-4283-a99f-37e8e12c8e22 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '891005fa5ff74041b75747d16cc66798', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80378) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:01:19.927695 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-539de3f6-0d88-4283-a99f-37e8e12c8e22 None admin] REQUEST_METHOD: `GET` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:01:19.928019 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-539de3f6-0d88-4283-a99f-37e8e12c8e22 None admin] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:01:19.928181 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-539de3f6-0d88-4283-a99f-37e8e12c8e22 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:01:19.929008 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-539de3f6-0d88-4283-a99f-37e8e12c8e22 None admin] RBAC: Adding query filter params (role.id=649993d933d44c50b268df7fdf336fbf, scope.project.id=f7db7cf528494fbb8836c08d79b752ef, user.id=4b50324ba62a45b892d69d01fa699f73) {{(pid=80378) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 02 10:01:19.929322 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-539de3f6-0d88-4283-a99f-37e8e12c8e22 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:01:19.930910 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-539de3f6-0d88-4283-a99f-37e8e12c8e22 None admin] RBAC: Authorization granted {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:01:19.951650 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 105/206] 10.222.0.82 () {56 vars in 1374 bytes} [Tue Aug 2 10:01:19 2022] GET /identity/v3/role_assignments?user.id=4b50324ba62a45b892d69d01fa699f73&scope.project.id=f7db7cf528494fbb8836c08d79b752ef&role.id=649993d933d44c50b268df7fdf336fbf => generated 259 bytes in 581 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 02 10:01:19.999961 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf9ecb80-4e65-476c-bcb7-216a1e4f11b6 None admin] Validating token access rules against request {{(pid=80377) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:01:20.058158 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec7c901b-a950-4da6-b24c-5ad31b189b65 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '891005fa5ff74041b75747d16cc66798', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80377) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:01:20.059798 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec7c901b-a950-4da6-b24c-5ad31b189b65 None admin] REQUEST_METHOD: `GET` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:01:20.060638 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec7c901b-a950-4da6-b24c-5ad31b189b65 None admin] SCRIPT_NAME: `/identity` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:01:20.061425 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec7c901b-a950-4da6-b24c-5ad31b189b65 None admin] PATH_INFO: `/v3/projects/c1809c9e380b45ec803130fe38eeaf24` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:01:20.064732 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec7c901b-a950-4da6-b24c-5ad31b189b65 None admin] RBAC: Authorizing `identity:get_project(project_id=c1809c9e380b45ec803130fe38eeaf24)` {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:01:20.067929 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec7c901b-a950-4da6-b24c-5ad31b189b65 None admin] RBAC: Authorization granted {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:01:20.071760 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: [pid: 80377|app: 0|req: 102/207] 10.222.0.82 () {56 vars in 1186 bytes} [Tue Aug 2 10:01:19 2022] GET /identity/v3/projects/c1809c9e380b45ec803130fe38eeaf24 => generated 298 bytes in 238 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 02 10:01:20.081940 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-539de3f6-0d88-4283-a99f-37e8e12c8e22 None admin] Authenticating user token {{(pid=80378) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:01:20.296236 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-539de3f6-0d88-4283-a99f-37e8e12c8e22 None admin] Validating token access rules against request {{(pid=80378) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:01:20.354926 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG dbcounter [-] [80378] Writing DB stats keystone:SELECT=100 {{(pid=80378) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 10:01:20.371916 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f45e272-0260-4d66-8f61-cce159c8ad2a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '891005fa5ff74041b75747d16cc66798', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80378) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:01:20.372559 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f45e272-0260-4d66-8f61-cce159c8ad2a None admin] REQUEST_METHOD: `GET` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:01:20.372847 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f45e272-0260-4d66-8f61-cce159c8ad2a None admin] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:01:20.373107 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f45e272-0260-4d66-8f61-cce159c8ad2a None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:01:20.373957 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f45e272-0260-4d66-8f61-cce159c8ad2a None admin] RBAC: Adding query filter params (role.id=444e768aa5fb44bbab6c20afad6a0218, scope.project.id=c1809c9e380b45ec803130fe38eeaf24, user.id=1c51a65536174654945f89334229c254) {{(pid=80378) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 02 10:01:20.374273 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f45e272-0260-4d66-8f61-cce159c8ad2a None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:01:20.375764 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f45e272-0260-4d66-8f61-cce159c8ad2a None admin] RBAC: Authorization granted {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:01:20.383311 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 106/208] 10.222.0.82 () {56 vars in 1374 bytes} [Tue Aug 2 10:01:20 2022] GET /identity/v3/role_assignments?user.id=1c51a65536174654945f89334229c254&scope.project.id=c1809c9e380b45ec803130fe38eeaf24&role.id=444e768aa5fb44bbab6c20afad6a0218 => generated 608 bytes in 302 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 02 10:01:20.608236 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-982b8e16-1248-46f3-8c77-e76d27aa1219 None None] REQUEST_METHOD: `GET` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:01:20.609230 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-982b8e16-1248-46f3-8c77-e76d27aa1219 None None] SCRIPT_NAME: `/identity` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:01:20.609894 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-982b8e16-1248-46f3-8c77-e76d27aa1219 None None] PATH_INFO: `/` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:01:20.612773 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: [pid: 80377|app: 0|req: 103/209] 10.222.0.82 () {54 vars in 926 bytes} [Tue Aug 2 10:01:20 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 02 10:01:20.619651 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25d15dca-5441-46a7-ac93-5fd0fc8c1660 None None] REQUEST_METHOD: `POST` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:01:20.620102 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25d15dca-5441-46a7-ac93-5fd0fc8c1660 None None] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:01:20.620536 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25d15dca-5441-46a7-ac93-5fd0fc8c1660 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:01:20.655943 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.auth.core [None req-25d15dca-5441-46a7-ac93-5fd0fc8c1660 None None] MFA Rules not processed for user `891005fa5ff74041b75747d16cc66798`. Rule list: `[]` (Enabled: `True`). {{(pid=80378) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 02 10:01:20.658524 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.fernet_utils [None req-25d15dca-5441-46a7-ac93-5fd0fc8c1660 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80378) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 10:01:20.776358 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 107/210] 10.222.0.82 () {58 vars in 1023 bytes} [Tue Aug 2 10:01:20 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 159 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 02 10:01:20.800562 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-982b8e16-1248-46f3-8c77-e76d27aa1219 None None] Authenticating user token {{(pid=80377) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:01:20.882636 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG dbcounter [-] [80377] Writing DB stats keystone:SELECT=100 {{(pid=80377) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 10:01:21.008192 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-982b8e16-1248-46f3-8c77-e76d27aa1219 None None] Validating token access rules against request {{(pid=80377) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:01:21.020075 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f6a778a-9435-4c99-8322-0b693beae9a9 None None] REQUEST_METHOD: `GET` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:01:21.020933 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f6a778a-9435-4c99-8322-0b693beae9a9 None None] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:01:21.020933 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f6a778a-9435-4c99-8322-0b693beae9a9 None None] PATH_INFO: `/` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:01:21.022666 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 108/211] 10.222.0.82 () {54 vars in 926 bytes} [Tue Aug 2 10:01:21 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 02 10:01:21.028891 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f5d238c-ed45-48fc-a6fc-577ef76375ce None None] REQUEST_METHOD: `POST` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:01:21.029183 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f5d238c-ed45-48fc-a6fc-577ef76375ce None None] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:01:21.029470 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f5d238c-ed45-48fc-a6fc-577ef76375ce None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:01:21.053467 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.auth.core [None req-3f5d238c-ed45-48fc-a6fc-577ef76375ce None None] MFA Rules not processed for user `891005fa5ff74041b75747d16cc66798`. Rule list: `[]` (Enabled: `True`). {{(pid=80378) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 02 10:01:21.054864 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.fernet_utils [None req-3f5d238c-ed45-48fc-a6fc-577ef76375ce None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80378) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 10:01:21.071396 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-378b7ee6-3fc2-4fab-8ece-07c0b5801734 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '891005fa5ff74041b75747d16cc66798', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80377) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:01:21.072972 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-378b7ee6-3fc2-4fab-8ece-07c0b5801734 None admin] REQUEST_METHOD: `GET` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:01:21.073756 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-378b7ee6-3fc2-4fab-8ece-07c0b5801734 None admin] SCRIPT_NAME: `/identity` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:01:21.074565 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-378b7ee6-3fc2-4fab-8ece-07c0b5801734 None admin] PATH_INFO: `/v3/roles/member` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:01:21.092959 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-378b7ee6-3fc2-4fab-8ece-07c0b5801734 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 02 10:01:21.103057 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application [None req-378b7ee6-3fc2-4fab-8ece-07c0b5801734 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Aug 02 10:01:21.103057 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 02 10:01:21.103057 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 02 10:01:21.103057 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 02 10:01:21.103057 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 02 10:01:21.103057 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 02 10:01:21.103057 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 02 10:01:21.103057 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 02 10:01:21.103057 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 02 10:01:21.103057 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 02 10:01:21.103057 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 02 10:01:21.103057 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 02 10:01:21.103057 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 02 10:01:21.103057 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 02 10:01:21.103057 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 02 10:01:21.103057 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 02 10:01:21.105119 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 02 10:01:21.105119 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 02 10:01:21.105119 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 02 10:01:21.105119 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 02 10:01:21.105119 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 02 10:01:21.105119 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 02 10:01:21.105119 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 02 10:01:21.105119 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 02 10:01:21.105119 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 02 10:01:21.105119 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application return self.get_or_create( Aug 02 10:01:21.105119 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 02 10:01:21.105119 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application with Lock( Aug 02 10:01:21.105119 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 02 10:01:21.105119 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application return self._enter() Aug 02 10:01:21.105119 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 02 10:01:21.105119 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 02 10:01:21.106994 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 02 10:01:21.106994 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application return self.creator() Aug 02 10:01:21.106994 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 02 10:01:21.106994 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application created_value = creator( Aug 02 10:01:21.106994 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 02 10:01:21.106994 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 02 10:01:21.106994 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 02 10:01:21.106994 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 02 10:01:21.106994 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 02 10:01:21.106994 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 02 10:01:21.106994 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Aug 02 10:01:21.106994 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application Aug 02 10:01:21.111728 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: [pid: 80377|app: 0|req: 104/212] 10.222.0.82 () {56 vars in 1099 bytes} [Tue Aug 2 10:01:20 2022] GET /identity/v3/roles/member => generated 84 bytes in 312 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 02 10:01:21.121761 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-378b7ee6-3fc2-4fab-8ece-07c0b5801734 None admin] Authenticating user token {{(pid=80377) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:01:21.143682 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 109/213] 10.222.0.82 () {58 vars in 1023 bytes} [Tue Aug 2 10:01:21 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 115 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Aug 02 10:01:21.176314 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f5d238c-ed45-48fc-a6fc-577ef76375ce None None] Authenticating user token {{(pid=80378) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:01:21.246041 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG dbcounter [-] [80378] Writing DB stats keystone:SELECT=100 {{(pid=80378) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 10:01:21.308705 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-378b7ee6-3fc2-4fab-8ece-07c0b5801734 None admin] Validating token access rules against request {{(pid=80377) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:01:21.311365 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG dbcounter [-] [80377] Writing DB stats keystone:SELECT=100 {{(pid=80377) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 10:01:21.364139 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f5d238c-ed45-48fc-a6fc-577ef76375ce None None] Validating token access rules against request {{(pid=80378) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:01:21.373588 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26089b8e-be7e-450e-8c83-c4c861e60ef0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '891005fa5ff74041b75747d16cc66798', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80377) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:01:21.374496 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26089b8e-be7e-450e-8c83-c4c861e60ef0 None admin] REQUEST_METHOD: `GET` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:01:21.375080 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26089b8e-be7e-450e-8c83-c4c861e60ef0 None admin] SCRIPT_NAME: `/identity` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:01:21.375614 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26089b8e-be7e-450e-8c83-c4c861e60ef0 None admin] PATH_INFO: `/v3/roles` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:01:21.377205 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26089b8e-be7e-450e-8c83-c4c861e60ef0 None admin] RBAC: Adding query filter params (name=member) {{(pid=80377) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 02 10:01:21.377713 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26089b8e-be7e-450e-8c83-c4c861e60ef0 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:01:21.379313 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26089b8e-be7e-450e-8c83-c4c861e60ef0 None admin] RBAC: Authorization granted {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:01:21.390642 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: [pid: 80377|app: 0|req: 105/214] 10.222.0.82 () {56 vars in 1101 bytes} [Tue Aug 2 10:01:21 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 272 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 02 10:01:21.398525 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26089b8e-be7e-450e-8c83-c4c861e60ef0 None admin] Authenticating user token {{(pid=80377) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:01:21.460072 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a39f8aa-9543-4562-ad97-ca1e396ad878 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '891005fa5ff74041b75747d16cc66798', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80378) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:01:21.461574 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a39f8aa-9543-4562-ad97-ca1e396ad878 None admin] REQUEST_METHOD: `GET` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:01:21.462253 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a39f8aa-9543-4562-ad97-ca1e396ad878 None admin] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:01:21.462863 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a39f8aa-9543-4562-ad97-ca1e396ad878 None admin] PATH_INFO: `/v3/domains/default` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:01:21.465453 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a39f8aa-9543-4562-ad97-ca1e396ad878 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:01:21.468349 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a39f8aa-9543-4562-ad97-ca1e396ad878 None admin] RBAC: Authorization granted {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:01:21.472157 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 110/215] 10.222.0.82 () {56 vars in 1108 bytes} [Tue Aug 2 10:01:21 2022] GET /identity/v3/domains/default => generated 199 bytes in 297 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 02 10:01:21.481811 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a39f8aa-9543-4562-ad97-ca1e396ad878 None admin] Authenticating user token {{(pid=80378) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:01:21.598651 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26089b8e-be7e-450e-8c83-c4c861e60ef0 None admin] Validating token access rules against request {{(pid=80377) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:01:21.667901 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a39f8aa-9543-4562-ad97-ca1e396ad878 None admin] Validating token access rules against request {{(pid=80378) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:01:21.680821 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5b72819-1700-4b19-bd46-8e538fc2212c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '891005fa5ff74041b75747d16cc66798', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80377) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:01:21.681674 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5b72819-1700-4b19-bd46-8e538fc2212c None admin] REQUEST_METHOD: `GET` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:01:21.682285 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5b72819-1700-4b19-bd46-8e538fc2212c None admin] SCRIPT_NAME: `/identity` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:01:21.683068 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5b72819-1700-4b19-bd46-8e538fc2212c None admin] PATH_INFO: `/v3/users/e052f8df6a754a65b0c54a30f6794a67` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:01:21.691360 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG dbcounter [-] [80378] Writing DB stats keystone:SELECT=100 {{(pid=80378) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 10:01:21.711513 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5b72819-1700-4b19-bd46-8e538fc2212c None admin] RBAC: Authorizing `identity:get_user(user_id=e052f8df6a754a65b0c54a30f6794a67)` {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:01:21.713004 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5b72819-1700-4b19-bd46-8e538fc2212c None admin] RBAC: Authorization granted {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:01:21.722935 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: [pid: 80377|app: 0|req: 106/216] 10.222.0.82 () {56 vars in 1177 bytes} [Tue Aug 2 10:01:21 2022] GET /identity/v3/users/e052f8df6a754a65b0c54a30f6794a67 => generated 297 bytes in 318 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 02 10:01:21.722935 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cd99fd4-922e-4276-81c0-83c4b56bf5fc None None] REQUEST_METHOD: `GET` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:01:21.722935 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cd99fd4-922e-4276-81c0-83c4b56bf5fc None None] SCRIPT_NAME: `/identity` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:01:21.722935 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cd99fd4-922e-4276-81c0-83c4b56bf5fc None None] PATH_INFO: `/` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:01:21.722935 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: [pid: 80377|app: 0|req: 107/217] 10.222.0.82 () {54 vars in 926 bytes} [Tue Aug 2 10:01:21 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 02 10:01:21.725905 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6cd99fd4-922e-4276-81c0-83c4b56bf5fc None None] Authenticating user token {{(pid=80377) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:01:21.739819 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-794a5e2f-a69d-4f6f-a708-f975531da7f5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '891005fa5ff74041b75747d16cc66798', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80378) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:01:21.740638 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-794a5e2f-a69d-4f6f-a708-f975531da7f5 None admin] REQUEST_METHOD: `POST` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:01:21.741074 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-794a5e2f-a69d-4f6f-a708-f975531da7f5 None admin] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:01:21.741468 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-794a5e2f-a69d-4f6f-a708-f975531da7f5 None admin] PATH_INFO: `/v3/users` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:01:21.742646 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-794a5e2f-a69d-4f6f-a708-f975531da7f5 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:01:21.744279 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-794a5e2f-a69d-4f6f-a708-f975531da7f5 None admin] RBAC: Authorization granted {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:01:21.760963 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG dbcounter [-] [80377] Writing DB stats keystone:SELECT=100 {{(pid=80377) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 10:01:21.773242 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 111/218] 10.222.0.82 () {60 vars in 1132 bytes} [Tue Aug 2 10:01:21 2022] POST /identity/v3/users => generated 297 bytes in 292 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 02 10:01:21.776125 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d05baf9b-8e06-44ed-9fa8-99a92be2f08a None None] REQUEST_METHOD: `POST` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:01:21.776436 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d05baf9b-8e06-44ed-9fa8-99a92be2f08a None None] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:01:21.776963 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d05baf9b-8e06-44ed-9fa8-99a92be2f08a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:01:21.816275 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.auth.core [None req-d05baf9b-8e06-44ed-9fa8-99a92be2f08a None None] MFA Rules not processed for user `891005fa5ff74041b75747d16cc66798`. Rule list: `[]` (Enabled: `True`). {{(pid=80378) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 02 10:01:21.818328 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.fernet_utils [None req-d05baf9b-8e06-44ed-9fa8-99a92be2f08a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80378) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 10:01:21.922559 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6cd99fd4-922e-4276-81c0-83c4b56bf5fc None None] Validating token access rules against request {{(pid=80377) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:01:21.967457 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 112/219] 10.222.0.82 () {58 vars in 1023 bytes} [Tue Aug 2 10:01:21 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 194 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 02 10:01:21.989654 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce3391eb-a0c3-46df-9aef-bcc985c98eb9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '891005fa5ff74041b75747d16cc66798', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80377) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:01:21.991299 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce3391eb-a0c3-46df-9aef-bcc985c98eb9 None admin] REQUEST_METHOD: `GET` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:01:21.992097 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce3391eb-a0c3-46df-9aef-bcc985c98eb9 None admin] SCRIPT_NAME: `/identity` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:01:21.992841 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce3391eb-a0c3-46df-9aef-bcc985c98eb9 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:01:21.998863 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d05baf9b-8e06-44ed-9fa8-99a92be2f08a None None] Authenticating user token {{(pid=80378) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:01:22.116509 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce3391eb-a0c3-46df-9aef-bcc985c98eb9 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:01:22.120656 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce3391eb-a0c3-46df-9aef-bcc985c98eb9 None admin] RBAC: Authorization granted {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:01:22.153967 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG dbcounter [-] [80377] Writing DB stats keystone:SELECT=100 {{(pid=80377) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 10:01:22.193764 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG dbcounter [-] [80378] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80378) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 10:01:22.198266 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d05baf9b-8e06-44ed-9fa8-99a92be2f08a None None] Validating token access rules against request {{(pid=80378) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:01:22.272070 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e19174f4-bbf4-4a54-95a6-96115f353a7b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '891005fa5ff74041b75747d16cc66798', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80378) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:01:22.272737 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e19174f4-bbf4-4a54-95a6-96115f353a7b None admin] REQUEST_METHOD: `GET` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:01:22.273039 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e19174f4-bbf4-4a54-95a6-96115f353a7b None admin] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:01:22.273339 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e19174f4-bbf4-4a54-95a6-96115f353a7b None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:01:22.296156 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e19174f4-bbf4-4a54-95a6-96115f353a7b None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 02 10:01:22.304938 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application [None req-e19174f4-bbf4-4a54-95a6-96115f353a7b None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 02 10:01:22.304938 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 02 10:01:22.304938 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 02 10:01:22.304938 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 02 10:01:22.304938 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 02 10:01:22.304938 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 02 10:01:22.304938 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 02 10:01:22.304938 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 02 10:01:22.304938 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 02 10:01:22.304938 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 02 10:01:22.304938 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 02 10:01:22.304938 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 02 10:01:22.304938 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 02 10:01:22.304938 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 02 10:01:22.304938 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 02 10:01:22.304938 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 02 10:01:22.305917 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 02 10:01:22.305917 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 02 10:01:22.305917 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 02 10:01:22.305917 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 02 10:01:22.305917 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 02 10:01:22.305917 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 02 10:01:22.305917 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 02 10:01:22.305917 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 02 10:01:22.305917 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 02 10:01:22.305917 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application return self.get_or_create( Aug 02 10:01:22.305917 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 02 10:01:22.305917 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application with Lock( Aug 02 10:01:22.305917 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 02 10:01:22.305917 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application return self._enter() Aug 02 10:01:22.305917 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 02 10:01:22.305917 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 02 10:01:22.306693 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 02 10:01:22.306693 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application return self.creator() Aug 02 10:01:22.306693 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 02 10:01:22.306693 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application created_value = creator( Aug 02 10:01:22.306693 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 02 10:01:22.306693 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 02 10:01:22.306693 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 02 10:01:22.306693 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 02 10:01:22.306693 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 02 10:01:22.306693 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 02 10:01:22.306693 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 02 10:01:22.306693 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application Aug 02 10:01:22.307297 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 113/220] 10.222.0.82 () {56 vars in 1096 bytes} [Tue Aug 2 10:01:21 2022] GET /identity/v3/roles/admin => generated 83 bytes in 309 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 02 10:01:22.310303 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: [pid: 80377|app: 0|req: 108/221] 10.222.0.82 () {58 vars in 1282 bytes} [Tue Aug 2 10:01:21 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 587 msecs (HTTP/1.1 200) 6 headers in 358 bytes (2 switches on core 0) Aug 02 10:01:22.316595 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e19174f4-bbf4-4a54-95a6-96115f353a7b None admin] Authenticating user token {{(pid=80378) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:01:22.319510 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce3391eb-a0c3-46df-9aef-bcc985c98eb9 None admin] Authenticating user token {{(pid=80377) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:01:22.518489 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce3391eb-a0c3-46df-9aef-bcc985c98eb9 None admin] Validating token access rules against request {{(pid=80377) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:01:22.542650 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e19174f4-bbf4-4a54-95a6-96115f353a7b None admin] Validating token access rules against request {{(pid=80378) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:01:22.554070 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG dbcounter [-] [80377] Writing DB stats keystone:SELECT=100 {{(pid=80377) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 10:01:22.604730 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46e44710-6df5-4fac-b936-9a8b3430cf03 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '891005fa5ff74041b75747d16cc66798', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80377) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:01:22.605474 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46e44710-6df5-4fac-b936-9a8b3430cf03 None admin] REQUEST_METHOD: `GET` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:01:22.605980 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46e44710-6df5-4fac-b936-9a8b3430cf03 None admin] SCRIPT_NAME: `/identity` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:01:22.608290 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46e44710-6df5-4fac-b936-9a8b3430cf03 None admin] PATH_INFO: `/v3/projects/f7db7cf528494fbb8836c08d79b752ef` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:01:22.609883 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46e44710-6df5-4fac-b936-9a8b3430cf03 None admin] RBAC: Authorizing `identity:get_project(project_id=f7db7cf528494fbb8836c08d79b752ef)` {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:01:22.612969 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46e44710-6df5-4fac-b936-9a8b3430cf03 None admin] RBAC: Authorization granted {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:01:22.619113 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: [pid: 80377|app: 0|req: 109/222] 10.222.0.82 () {56 vars in 1186 bytes} [Tue Aug 2 10:01:22 2022] GET /identity/v3/projects/f7db7cf528494fbb8836c08d79b752ef => generated 302 bytes in 301 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 02 10:01:22.628921 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46e44710-6df5-4fac-b936-9a8b3430cf03 None admin] Authenticating user token {{(pid=80377) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:01:22.639357 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0f72dac-b194-48e0-a393-9591754d095c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '891005fa5ff74041b75747d16cc66798', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80378) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:01:22.640371 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0f72dac-b194-48e0-a393-9591754d095c None admin] REQUEST_METHOD: `GET` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:01:22.640848 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0f72dac-b194-48e0-a393-9591754d095c None admin] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:01:22.641321 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0f72dac-b194-48e0-a393-9591754d095c None admin] PATH_INFO: `/v3/roles` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:01:22.643163 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0f72dac-b194-48e0-a393-9591754d095c None admin] RBAC: Adding query filter params (name=admin) {{(pid=80378) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 02 10:01:22.643710 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0f72dac-b194-48e0-a393-9591754d095c None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:01:22.646314 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0f72dac-b194-48e0-a393-9591754d095c None admin] RBAC: Authorization granted {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:01:22.693600 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 114/223] 10.222.0.82 () {56 vars in 1099 bytes} [Tue Aug 2 10:01:22 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 378 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 02 10:01:22.695045 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0f72dac-b194-48e0-a393-9591754d095c None admin] Authenticating user token {{(pid=80378) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:01:22.767379 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG dbcounter [-] [80378] Writing DB stats keystone:SELECT=100 {{(pid=80378) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 10:01:22.827788 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46e44710-6df5-4fac-b936-9a8b3430cf03 None admin] Validating token access rules against request {{(pid=80377) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:01:22.888162 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af878e43-2a44-4795-8665-f958ecb8fe87 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '891005fa5ff74041b75747d16cc66798', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80377) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:01:22.889299 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af878e43-2a44-4795-8665-f958ecb8fe87 None admin] REQUEST_METHOD: `GET` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:01:22.890723 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af878e43-2a44-4795-8665-f958ecb8fe87 None admin] SCRIPT_NAME: `/identity` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:01:22.891400 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af878e43-2a44-4795-8665-f958ecb8fe87 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:01:22.894285 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af878e43-2a44-4795-8665-f958ecb8fe87 None admin] RBAC: Adding query filter params (role.id=e1c04b19fc4543b8b54f58bbcffbc598, scope.project.id=f7db7cf528494fbb8836c08d79b752ef, user.id=e052f8df6a754a65b0c54a30f6794a67) {{(pid=80377) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 02 10:01:22.895089 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af878e43-2a44-4795-8665-f958ecb8fe87 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:01:22.896985 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af878e43-2a44-4795-8665-f958ecb8fe87 None admin] RBAC: Authorization granted {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:01:22.912463 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: [pid: 80377|app: 0|req: 110/224] 10.222.0.82 () {56 vars in 1374 bytes} [Tue Aug 2 10:01:22 2022] GET /identity/v3/role_assignments?user.id=e052f8df6a754a65b0c54a30f6794a67&scope.project.id=f7db7cf528494fbb8836c08d79b752ef&role.id=e1c04b19fc4543b8b54f58bbcffbc598 => generated 259 bytes in 284 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 02 10:01:22.983361 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0f72dac-b194-48e0-a393-9591754d095c None admin] Validating token access rules against request {{(pid=80378) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:01:23.060751 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1a2a70e-30fd-48a4-994c-4cd8ff949d13 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '891005fa5ff74041b75747d16cc66798', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80378) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:01:23.061954 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1a2a70e-30fd-48a4-994c-4cd8ff949d13 None admin] REQUEST_METHOD: `GET` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:01:23.062615 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1a2a70e-30fd-48a4-994c-4cd8ff949d13 None admin] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:01:23.063321 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1a2a70e-30fd-48a4-994c-4cd8ff949d13 None admin] PATH_INFO: `/v3/users/4b50324ba62a45b892d69d01fa699f73` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:01:23.065303 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1a2a70e-30fd-48a4-994c-4cd8ff949d13 None admin] RBAC: Authorizing `identity:get_user(user_id=4b50324ba62a45b892d69d01fa699f73)` {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:01:23.067134 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1a2a70e-30fd-48a4-994c-4cd8ff949d13 None admin] RBAC: Authorization granted {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:01:23.069139 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 115/225] 10.222.0.82 () {56 vars in 1177 bytes} [Tue Aug 2 10:01:22 2022] GET /identity/v3/users/4b50324ba62a45b892d69d01fa699f73 => generated 283 bytes in 375 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 02 10:01:23.077462 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af878e43-2a44-4795-8665-f958ecb8fe87 None admin] Authenticating user token {{(pid=80377) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:01:23.163669 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG dbcounter [-] [80377] Writing DB stats keystone:SELECT=100 {{(pid=80377) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 10:01:23.250495 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af878e43-2a44-4795-8665-f958ecb8fe87 None admin] Validating token access rules against request {{(pid=80377) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:01:23.303794 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1cdd0d52-cddf-43f4-a435-31d52e6e0a4c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '891005fa5ff74041b75747d16cc66798', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80377) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:01:23.304967 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cdd0d52-cddf-43f4-a435-31d52e6e0a4c None admin] REQUEST_METHOD: `GET` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:01:23.305683 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cdd0d52-cddf-43f4-a435-31d52e6e0a4c None admin] SCRIPT_NAME: `/identity` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:01:23.306446 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cdd0d52-cddf-43f4-a435-31d52e6e0a4c None admin] PATH_INFO: `/v3/projects/f7db7cf528494fbb8836c08d79b752ef` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:01:23.308541 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1cdd0d52-cddf-43f4-a435-31d52e6e0a4c None admin] RBAC: Authorizing `identity:get_project(project_id=f7db7cf528494fbb8836c08d79b752ef)` {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:01:23.311516 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1cdd0d52-cddf-43f4-a435-31d52e6e0a4c None admin] RBAC: Authorization granted {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:01:23.314752 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: [pid: 80377|app: 0|req: 111/226] 10.222.0.82 () {56 vars in 1186 bytes} [Tue Aug 2 10:01:23 2022] GET /identity/v3/projects/f7db7cf528494fbb8836c08d79b752ef => generated 302 bytes in 238 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 02 10:01:23.323706 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1a2a70e-30fd-48a4-994c-4cd8ff949d13 None admin] Authenticating user token {{(pid=80378) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:01:23.366838 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bce1d289-7ff6-496b-8ead-8cb766638901 None None] REQUEST_METHOD: `GET` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:01:23.367326 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bce1d289-7ff6-496b-8ead-8cb766638901 None None] SCRIPT_NAME: `/identity` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:01:23.367814 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bce1d289-7ff6-496b-8ead-8cb766638901 None None] PATH_INFO: `/` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:01:23.370306 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: [pid: 80377|app: 0|req: 112/227] 10.222.0.82 () {54 vars in 926 bytes} [Tue Aug 2 10:01:23 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 02 10:01:23.381533 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50ff8697-3929-408f-8a11-c8c985a36bc8 None None] REQUEST_METHOD: `POST` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:01:23.382072 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50ff8697-3929-408f-8a11-c8c985a36bc8 None None] SCRIPT_NAME: `/identity` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:01:23.382558 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50ff8697-3929-408f-8a11-c8c985a36bc8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:01:23.435180 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.auth.core [None req-50ff8697-3929-408f-8a11-c8c985a36bc8 None None] MFA Rules not processed for user `891005fa5ff74041b75747d16cc66798`. Rule list: `[]` (Enabled: `True`). {{(pid=80377) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 02 10:01:23.436299 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.fernet_utils [None req-50ff8697-3929-408f-8a11-c8c985a36bc8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80377) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 10:01:23.606743 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG dbcounter [-] [80378] Writing DB stats keystone:SELECT=100 {{(pid=80378) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 10:01:23.623783 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1a2a70e-30fd-48a4-994c-4cd8ff949d13 None admin] Validating token access rules against request {{(pid=80378) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:01:23.634861 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: [pid: 80377|app: 0|req: 113/228] 10.222.0.82 () {58 vars in 1023 bytes} [Tue Aug 2 10:01:23 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 255 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Aug 02 10:01:23.683564 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50ff8697-3929-408f-8a11-c8c985a36bc8 None None] Authenticating user token {{(pid=80377) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:01:23.710075 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3fa93ff-2681-42c6-a20a-5bb6a90d5860 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '891005fa5ff74041b75747d16cc66798', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80378) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:01:23.711660 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3fa93ff-2681-42c6-a20a-5bb6a90d5860 None admin] REQUEST_METHOD: `PUT` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:01:23.712399 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3fa93ff-2681-42c6-a20a-5bb6a90d5860 None admin] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:01:23.713156 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3fa93ff-2681-42c6-a20a-5bb6a90d5860 None admin] PATH_INFO: `/v3/projects/f7db7cf528494fbb8836c08d79b752ef/users/4b50324ba62a45b892d69d01fa699f73/roles/237966442f05411191105bf336bc14f3` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:01:23.721176 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3fa93ff-2681-42c6-a20a-5bb6a90d5860 None admin] RBAC: Authorizing `identity:create_grant(project_id=f7db7cf528494fbb8836c08d79b752ef, user_id=4b50324ba62a45b892d69d01fa699f73, role_id=237966442f05411191105bf336bc14f3)` {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:01:23.723633 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3fa93ff-2681-42c6-a20a-5bb6a90d5860 None admin] RBAC: Authorization granted {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:01:23.756389 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 116/229] 10.222.0.82 () {58 vars in 1439 bytes} [Tue Aug 2 10:01:23 2022] PUT /identity/v3/projects/f7db7cf528494fbb8836c08d79b752ef/users/4b50324ba62a45b892d69d01fa699f73/roles/237966442f05411191105bf336bc14f3 => generated 0 bytes in 434 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 02 10:01:23.784730 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG dbcounter [-] [80377] Writing DB stats keystone:SELECT=100 {{(pid=80377) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 10:01:23.876033 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50ff8697-3929-408f-8a11-c8c985a36bc8 None None] Validating token access rules against request {{(pid=80377) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:01:23.931999 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12d9eca7-4012-4a00-87fe-e893e01d1fc4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '891005fa5ff74041b75747d16cc66798', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80377) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:01:23.933056 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12d9eca7-4012-4a00-87fe-e893e01d1fc4 None admin] REQUEST_METHOD: `GET` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:01:23.933509 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12d9eca7-4012-4a00-87fe-e893e01d1fc4 None admin] SCRIPT_NAME: `/identity` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:01:23.933990 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12d9eca7-4012-4a00-87fe-e893e01d1fc4 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:01:23.953013 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12d9eca7-4012-4a00-87fe-e893e01d1fc4 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 02 10:01:23.958981 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application [None req-12d9eca7-4012-4a00-87fe-e893e01d1fc4 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Aug 02 10:01:23.958981 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 02 10:01:23.958981 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 02 10:01:23.958981 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 02 10:01:23.958981 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 02 10:01:23.958981 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 02 10:01:23.958981 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 02 10:01:23.958981 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 02 10:01:23.958981 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 02 10:01:23.958981 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 02 10:01:23.958981 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 02 10:01:23.958981 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 02 10:01:23.958981 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 02 10:01:23.958981 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 02 10:01:23.958981 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 02 10:01:23.958981 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 02 10:01:23.961434 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 02 10:01:23.961434 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 02 10:01:23.961434 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 02 10:01:23.961434 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 02 10:01:23.961434 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 02 10:01:23.961434 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 02 10:01:23.961434 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 02 10:01:23.961434 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 02 10:01:23.961434 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 02 10:01:23.961434 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application return self.get_or_create( Aug 02 10:01:23.961434 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 02 10:01:23.961434 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application with Lock( Aug 02 10:01:23.961434 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 02 10:01:23.961434 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application return self._enter() Aug 02 10:01:23.961434 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 02 10:01:23.961434 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 02 10:01:23.962855 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 02 10:01:23.962855 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application return self.creator() Aug 02 10:01:23.962855 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 02 10:01:23.962855 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application created_value = creator( Aug 02 10:01:23.962855 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 02 10:01:23.962855 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 02 10:01:23.962855 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 02 10:01:23.962855 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 02 10:01:23.962855 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 02 10:01:23.962855 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 02 10:01:23.962855 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Aug 02 10:01:23.962855 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application Aug 02 10:01:23.962855 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: [pid: 80377|app: 0|req: 114/230] 10.222.0.82 () {56 vars in 1114 bytes} [Tue Aug 2 10:01:23 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 281 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 02 10:01:23.967848 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3fa93ff-2681-42c6-a20a-5bb6a90d5860 None admin] Authenticating user token {{(pid=80378) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:01:24.166058 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3fa93ff-2681-42c6-a20a-5bb6a90d5860 None admin] Validating token access rules against request {{(pid=80378) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:01:24.239779 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1a7a748-31c0-4f24-bc57-ebed6205b233 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '891005fa5ff74041b75747d16cc66798', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80378) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:01:24.240832 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1a7a748-31c0-4f24-bc57-ebed6205b233 None admin] REQUEST_METHOD: `GET` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:01:24.241321 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1a7a748-31c0-4f24-bc57-ebed6205b233 None admin] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:01:24.241810 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1a7a748-31c0-4f24-bc57-ebed6205b233 None admin] PATH_INFO: `/v3/roles` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:01:24.243163 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1a7a748-31c0-4f24-bc57-ebed6205b233 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=80378) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 02 10:01:24.243727 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1a7a748-31c0-4f24-bc57-ebed6205b233 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:01:24.246403 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1a7a748-31c0-4f24-bc57-ebed6205b233 None admin] RBAC: Authorization granted {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:01:24.260708 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 117/231] 10.222.0.82 () {56 vars in 1111 bytes} [Tue Aug 2 10:01:23 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 295 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 02 10:01:24.267598 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12d9eca7-4012-4a00-87fe-e893e01d1fc4 None admin] Authenticating user token {{(pid=80377) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:01:24.445942 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12d9eca7-4012-4a00-87fe-e893e01d1fc4 None admin] Validating token access rules against request {{(pid=80377) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:01:24.458125 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG dbcounter [-] [80377] Writing DB stats keystone:SELECT=100 {{(pid=80377) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 10:01:24.495289 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fcd2a21c-a982-4ce4-8418-9a099d0acfe7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '891005fa5ff74041b75747d16cc66798', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80377) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:01:24.496208 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcd2a21c-a982-4ce4-8418-9a099d0acfe7 None admin] REQUEST_METHOD: `GET` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:01:24.496612 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcd2a21c-a982-4ce4-8418-9a099d0acfe7 None admin] SCRIPT_NAME: `/identity` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:01:24.497040 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcd2a21c-a982-4ce4-8418-9a099d0acfe7 None admin] PATH_INFO: `/v3/users/4b50324ba62a45b892d69d01fa699f73` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:01:24.498773 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fcd2a21c-a982-4ce4-8418-9a099d0acfe7 None admin] RBAC: Authorizing `identity:get_user(user_id=4b50324ba62a45b892d69d01fa699f73)` {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:01:24.500578 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fcd2a21c-a982-4ce4-8418-9a099d0acfe7 None admin] RBAC: Authorization granted {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:01:24.503373 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: [pid: 80377|app: 0|req: 115/232] 10.222.0.82 () {56 vars in 1177 bytes} [Tue Aug 2 10:01:24 2022] GET /identity/v3/users/4b50324ba62a45b892d69d01fa699f73 => generated 283 bytes in 237 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 02 10:01:24.509464 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1a7a748-31c0-4f24-bc57-ebed6205b233 None admin] Authenticating user token {{(pid=80378) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:01:24.533556 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG dbcounter [-] [80378] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80378) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 10:01:24.703713 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1a7a748-31c0-4f24-bc57-ebed6205b233 None admin] Validating token access rules against request {{(pid=80378) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:01:24.765575 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30376a4d-eed3-427a-9136-1a44bfadd77a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '891005fa5ff74041b75747d16cc66798', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80378) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:01:24.767139 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30376a4d-eed3-427a-9136-1a44bfadd77a None admin] REQUEST_METHOD: `GET` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:01:24.767645 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30376a4d-eed3-427a-9136-1a44bfadd77a None admin] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:01:24.768123 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30376a4d-eed3-427a-9136-1a44bfadd77a None admin] PATH_INFO: `/v3/projects/f7db7cf528494fbb8836c08d79b752ef` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:01:24.770237 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30376a4d-eed3-427a-9136-1a44bfadd77a None admin] RBAC: Authorizing `identity:get_project(project_id=f7db7cf528494fbb8836c08d79b752ef)` {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:01:24.772893 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30376a4d-eed3-427a-9136-1a44bfadd77a None admin] RBAC: Authorization granted {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:01:24.776162 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 118/233] 10.222.0.82 () {56 vars in 1186 bytes} [Tue Aug 2 10:01:24 2022] GET /identity/v3/projects/f7db7cf528494fbb8836c08d79b752ef => generated 302 bytes in 268 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 02 10:01:24.786548 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fcd2a21c-a982-4ce4-8418-9a099d0acfe7 None admin] Authenticating user token {{(pid=80377) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:01:24.972372 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fcd2a21c-a982-4ce4-8418-9a099d0acfe7 None admin] Validating token access rules against request {{(pid=80377) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:01:25.035304 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbb09bb2-d257-41eb-bba9-3cc2893a239e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '891005fa5ff74041b75747d16cc66798', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80377) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:01:25.036371 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbb09bb2-d257-41eb-bba9-3cc2893a239e None admin] REQUEST_METHOD: `PUT` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:01:25.036861 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbb09bb2-d257-41eb-bba9-3cc2893a239e None admin] SCRIPT_NAME: `/identity` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:01:25.037337 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbb09bb2-d257-41eb-bba9-3cc2893a239e None admin] PATH_INFO: `/v3/projects/f7db7cf528494fbb8836c08d79b752ef/users/4b50324ba62a45b892d69d01fa699f73/roles/649993d933d44c50b268df7fdf336fbf` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:01:25.041172 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fbb09bb2-d257-41eb-bba9-3cc2893a239e None admin] RBAC: Authorizing `identity:create_grant(project_id=f7db7cf528494fbb8836c08d79b752ef, user_id=4b50324ba62a45b892d69d01fa699f73, role_id=649993d933d44c50b268df7fdf336fbf)` {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:01:25.043959 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fbb09bb2-d257-41eb-bba9-3cc2893a239e None admin] RBAC: Authorization granted {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:01:25.068876 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: [pid: 80377|app: 0|req: 116/234] 10.222.0.82 () {58 vars in 1439 bytes} [Tue Aug 2 10:01:24 2022] PUT /identity/v3/projects/f7db7cf528494fbb8836c08d79b752ef/users/4b50324ba62a45b892d69d01fa699f73/roles/649993d933d44c50b268df7fdf336fbf => generated 0 bytes in 283 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 02 10:01:25.456746 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8446fe9d-b617-4c4c-9009-cc7bf03482c8 None None] REQUEST_METHOD: `GET` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:01:25.457459 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8446fe9d-b617-4c4c-9009-cc7bf03482c8 None None] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:01:25.458274 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8446fe9d-b617-4c4c-9009-cc7bf03482c8 None None] PATH_INFO: `/` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:01:25.461035 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 119/235] 10.222.0.82 () {54 vars in 926 bytes} [Tue Aug 2 10:01:25 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 02 10:01:25.468537 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ea4da56-5d7f-4491-aae5-43ae1c2495f1 None None] REQUEST_METHOD: `POST` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:01:25.470021 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ea4da56-5d7f-4491-aae5-43ae1c2495f1 None None] SCRIPT_NAME: `/identity` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:01:25.470598 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ea4da56-5d7f-4491-aae5-43ae1c2495f1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:01:25.501676 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.auth.core [None req-1ea4da56-5d7f-4491-aae5-43ae1c2495f1 None None] MFA Rules not processed for user `891005fa5ff74041b75747d16cc66798`. Rule list: `[]` (Enabled: `True`). {{(pid=80377) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 02 10:01:25.504336 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.fernet_utils [None req-1ea4da56-5d7f-4491-aae5-43ae1c2495f1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80377) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 10:01:25.548768 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG dbcounter [-] [80377] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80377) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 10:01:25.613888 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: [pid: 80377|app: 0|req: 117/236] 10.222.0.82 () {58 vars in 1023 bytes} [Tue Aug 2 10:01:25 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 147 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 02 10:01:25.659320 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8446fe9d-b617-4c4c-9009-cc7bf03482c8 None None] Authenticating user token {{(pid=80378) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:01:25.722317 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d10e1f2e-0b52-4e97-b675-762cb781d59c None None] REQUEST_METHOD: `GET` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:01:25.722585 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d10e1f2e-0b52-4e97-b675-762cb781d59c None None] SCRIPT_NAME: `/identity` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:01:25.722890 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d10e1f2e-0b52-4e97-b675-762cb781d59c None None] PATH_INFO: `/` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:01:25.725204 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: [pid: 80377|app: 0|req: 118/237] 10.222.0.82 () {54 vars in 926 bytes} [Tue Aug 2 10:01:25 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 02 10:01:25.732737 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fae725da-e176-4ca1-8ef4-0fe73fccf420 None None] REQUEST_METHOD: `POST` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:01:25.733012 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fae725da-e176-4ca1-8ef4-0fe73fccf420 None None] SCRIPT_NAME: `/identity` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:01:25.733353 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fae725da-e176-4ca1-8ef4-0fe73fccf420 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:01:25.756637 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.auth.core [None req-fae725da-e176-4ca1-8ef4-0fe73fccf420 None None] MFA Rules not processed for user `891005fa5ff74041b75747d16cc66798`. Rule list: `[]` (Enabled: `True`). {{(pid=80377) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 02 10:01:25.758626 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.fernet_utils [None req-fae725da-e176-4ca1-8ef4-0fe73fccf420 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80377) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 10:01:25.870369 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG dbcounter [-] [80378] Writing DB stats keystone:SELECT=100 {{(pid=80378) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 10:01:25.910161 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8446fe9d-b617-4c4c-9009-cc7bf03482c8 None None] Validating token access rules against request {{(pid=80378) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:01:25.947951 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: [pid: 80377|app: 0|req: 119/238] 10.222.0.82 () {58 vars in 1023 bytes} [Tue Aug 2 10:01:25 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 216 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 02 10:01:25.979755 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fae725da-e176-4ca1-8ef4-0fe73fccf420 None None] Authenticating user token {{(pid=80377) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:01:26.026142 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c54e7df-4cf3-498d-a88d-4d8402aa38f1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '891005fa5ff74041b75747d16cc66798', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80378) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:01:26.027228 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c54e7df-4cf3-498d-a88d-4d8402aa38f1 None admin] REQUEST_METHOD: `GET` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:01:26.027702 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c54e7df-4cf3-498d-a88d-4d8402aa38f1 None admin] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:01:26.028170 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c54e7df-4cf3-498d-a88d-4d8402aa38f1 None admin] PATH_INFO: `/v3/roles/reader` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:01:26.057849 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c54e7df-4cf3-498d-a88d-4d8402aa38f1 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 02 10:01:26.070725 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application [None req-0c54e7df-4cf3-498d-a88d-4d8402aa38f1 None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Aug 02 10:01:26.070725 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 02 10:01:26.070725 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 02 10:01:26.070725 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 02 10:01:26.070725 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 02 10:01:26.070725 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 02 10:01:26.070725 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 02 10:01:26.070725 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 02 10:01:26.070725 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 02 10:01:26.070725 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 02 10:01:26.070725 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 02 10:01:26.070725 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 02 10:01:26.070725 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 02 10:01:26.070725 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 02 10:01:26.070725 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 02 10:01:26.070725 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 02 10:01:26.071686 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 02 10:01:26.071686 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 02 10:01:26.071686 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 02 10:01:26.071686 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 02 10:01:26.071686 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 02 10:01:26.071686 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 02 10:01:26.071686 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 02 10:01:26.071686 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 02 10:01:26.071686 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 02 10:01:26.071686 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application return self.get_or_create( Aug 02 10:01:26.071686 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 02 10:01:26.071686 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application with Lock( Aug 02 10:01:26.071686 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 02 10:01:26.071686 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application return self._enter() Aug 02 10:01:26.071686 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 02 10:01:26.071686 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 02 10:01:26.072474 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 02 10:01:26.072474 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application return self.creator() Aug 02 10:01:26.072474 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 02 10:01:26.072474 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application created_value = creator( Aug 02 10:01:26.072474 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 02 10:01:26.072474 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 02 10:01:26.072474 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 02 10:01:26.072474 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 02 10:01:26.072474 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 02 10:01:26.072474 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 02 10:01:26.072474 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Aug 02 10:01:26.072474 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application Aug 02 10:01:26.076683 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 120/239] 10.222.0.82 () {56 vars in 1099 bytes} [Tue Aug 2 10:01:25 2022] GET /identity/v3/roles/reader => generated 84 bytes in 419 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 02 10:01:26.087633 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c54e7df-4cf3-498d-a88d-4d8402aa38f1 None admin] Authenticating user token {{(pid=80378) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:01:26.192225 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG dbcounter [-] [80377] Writing DB stats keystone:SELECT=100 {{(pid=80377) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 10:01:26.199141 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fae725da-e176-4ca1-8ef4-0fe73fccf420 None None] Validating token access rules against request {{(pid=80377) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:01:26.252932 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed523af0-2c65-4dd5-a31d-d524fc413cfe None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '891005fa5ff74041b75747d16cc66798', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80377) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:01:26.254001 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed523af0-2c65-4dd5-a31d-d524fc413cfe None admin] REQUEST_METHOD: `GET` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:01:26.254527 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed523af0-2c65-4dd5-a31d-d524fc413cfe None admin] SCRIPT_NAME: `/identity` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:01:26.262495 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed523af0-2c65-4dd5-a31d-d524fc413cfe None admin] PATH_INFO: `/v3/domains/default` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:01:26.265629 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed523af0-2c65-4dd5-a31d-d524fc413cfe None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:01:26.268004 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed523af0-2c65-4dd5-a31d-d524fc413cfe None admin] RBAC: Authorization granted {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:01:26.279405 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: [pid: 80377|app: 0|req: 120/240] 10.222.0.82 () {56 vars in 1108 bytes} [Tue Aug 2 10:01:25 2022] GET /identity/v3/domains/default => generated 199 bytes in 301 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 02 10:01:26.307436 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed523af0-2c65-4dd5-a31d-d524fc413cfe None admin] Authenticating user token {{(pid=80377) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:01:26.317929 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c54e7df-4cf3-498d-a88d-4d8402aa38f1 None admin] Validating token access rules against request {{(pid=80378) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:01:26.372064 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46984392-dbb6-45cd-96f1-6cf49440c548 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '891005fa5ff74041b75747d16cc66798', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80378) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:01:26.373204 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46984392-dbb6-45cd-96f1-6cf49440c548 None admin] REQUEST_METHOD: `GET` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:01:26.373688 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46984392-dbb6-45cd-96f1-6cf49440c548 None admin] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:01:26.374466 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46984392-dbb6-45cd-96f1-6cf49440c548 None admin] PATH_INFO: `/v3/roles` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:01:26.375402 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46984392-dbb6-45cd-96f1-6cf49440c548 None admin] RBAC: Adding query filter params (name=reader) {{(pid=80378) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 02 10:01:26.375910 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46984392-dbb6-45cd-96f1-6cf49440c548 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:01:26.379014 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46984392-dbb6-45cd-96f1-6cf49440c548 None admin] RBAC: Authorization granted {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:01:26.393403 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 121/241] 10.222.0.82 () {56 vars in 1101 bytes} [Tue Aug 2 10:01:26 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 307 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 02 10:01:26.403398 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46984392-dbb6-45cd-96f1-6cf49440c548 None admin] Authenticating user token {{(pid=80378) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:01:26.407398 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG dbcounter [-] [80378] Writing DB stats keystone:SELECT=100 {{(pid=80378) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 10:01:26.661553 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed523af0-2c65-4dd5-a31d-d524fc413cfe None admin] Validating token access rules against request {{(pid=80377) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:01:26.707446 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39bedd96-9837-4527-94dc-d7d913557565 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '891005fa5ff74041b75747d16cc66798', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80377) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:01:26.708418 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39bedd96-9837-4527-94dc-d7d913557565 None admin] REQUEST_METHOD: `POST` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:01:26.708852 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39bedd96-9837-4527-94dc-d7d913557565 None admin] SCRIPT_NAME: `/identity` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:01:26.709311 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39bedd96-9837-4527-94dc-d7d913557565 None admin] PATH_INFO: `/v3/users` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:01:26.710223 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39bedd96-9837-4527-94dc-d7d913557565 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:01:26.712336 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39bedd96-9837-4527-94dc-d7d913557565 None admin] RBAC: Authorization granted {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:01:26.737987 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: [pid: 80377|app: 0|req: 121/242] 10.222.0.82 () {60 vars in 1132 bytes} [Tue Aug 2 10:01:26 2022] POST /identity/v3/users => generated 293 bytes in 432 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 02 10:01:26.739687 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2f5e562-23c3-4286-a374-3a9395916a24 None None] REQUEST_METHOD: `GET` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:01:26.740269 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2f5e562-23c3-4286-a374-3a9395916a24 None None] SCRIPT_NAME: `/identity` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:01:26.740746 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2f5e562-23c3-4286-a374-3a9395916a24 None None] PATH_INFO: `/` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:01:26.742611 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: [pid: 80377|app: 0|req: 122/243] 10.222.0.82 () {54 vars in 926 bytes} [Tue Aug 2 10:01:26 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 02 10:01:26.768610 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08573b27-3e9a-45f5-b7a2-f9a6a3b83abf None None] REQUEST_METHOD: `POST` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:01:26.769314 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08573b27-3e9a-45f5-b7a2-f9a6a3b83abf None None] SCRIPT_NAME: `/identity` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:01:26.769836 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08573b27-3e9a-45f5-b7a2-f9a6a3b83abf None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:01:26.777820 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46984392-dbb6-45cd-96f1-6cf49440c548 None admin] Validating token access rules against request {{(pid=80378) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:01:26.811016 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.auth.core [None req-08573b27-3e9a-45f5-b7a2-f9a6a3b83abf None None] MFA Rules not processed for user `891005fa5ff74041b75747d16cc66798`. Rule list: `[]` (Enabled: `True`). {{(pid=80377) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 02 10:01:26.812634 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.fernet_utils [None req-08573b27-3e9a-45f5-b7a2-f9a6a3b83abf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80377) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 10:01:26.828421 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-346348ae-d998-4080-b09a-d8de0b9bca46 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '891005fa5ff74041b75747d16cc66798', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80378) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:01:26.829896 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-346348ae-d998-4080-b09a-d8de0b9bca46 None admin] REQUEST_METHOD: `GET` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:01:26.830277 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-346348ae-d998-4080-b09a-d8de0b9bca46 None admin] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:01:26.830559 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-346348ae-d998-4080-b09a-d8de0b9bca46 None admin] PATH_INFO: `/v3/users/b5222f0dc11f4c6998b83c676ee8e6b8` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:01:26.851062 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG dbcounter [-] [80377] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80377) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 10:01:26.859362 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-346348ae-d998-4080-b09a-d8de0b9bca46 None admin] RBAC: Authorizing `identity:get_user(user_id=b5222f0dc11f4c6998b83c676ee8e6b8)` {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:01:26.863166 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-346348ae-d998-4080-b09a-d8de0b9bca46 None admin] RBAC: Authorization granted {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:01:26.866138 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 122/244] 10.222.0.82 () {56 vars in 1177 bytes} [Tue Aug 2 10:01:26 2022] GET /identity/v3/users/b5222f0dc11f4c6998b83c676ee8e6b8 => generated 297 bytes in 464 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 02 10:01:26.882853 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-346348ae-d998-4080-b09a-d8de0b9bca46 None admin] Authenticating user token {{(pid=80378) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:01:26.961428 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: [pid: 80377|app: 0|req: 123/245] 10.222.0.82 () {58 vars in 1023 bytes} [Tue Aug 2 10:01:26 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 194 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 02 10:01:26.975206 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG dbcounter [-] [80378] Writing DB stats keystone:SELECT=100 {{(pid=80378) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 10:01:26.995274 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08573b27-3e9a-45f5-b7a2-f9a6a3b83abf None None] Authenticating user token {{(pid=80377) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:01:27.026250 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-346348ae-d998-4080-b09a-d8de0b9bca46 None admin] Validating token access rules against request {{(pid=80378) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:01:27.087512 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79470765-a3ad-4416-9940-de19be0243e7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '891005fa5ff74041b75747d16cc66798', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80378) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:01:27.089017 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79470765-a3ad-4416-9940-de19be0243e7 None admin] REQUEST_METHOD: `GET` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:01:27.089777 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79470765-a3ad-4416-9940-de19be0243e7 None admin] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:01:27.090600 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79470765-a3ad-4416-9940-de19be0243e7 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:01:27.224589 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79470765-a3ad-4416-9940-de19be0243e7 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:01:27.228659 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79470765-a3ad-4416-9940-de19be0243e7 None admin] RBAC: Authorization granted {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:01:27.261292 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08573b27-3e9a-45f5-b7a2-f9a6a3b83abf None None] Validating token access rules against request {{(pid=80377) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:01:27.327194 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d85c37e5-fad3-4cc8-bdf9-c850a8243455 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '891005fa5ff74041b75747d16cc66798', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80377) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:01:27.328299 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d85c37e5-fad3-4cc8-bdf9-c850a8243455 None admin] REQUEST_METHOD: `GET` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:01:27.328791 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d85c37e5-fad3-4cc8-bdf9-c850a8243455 None admin] SCRIPT_NAME: `/identity` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:01:27.329309 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d85c37e5-fad3-4cc8-bdf9-c850a8243455 None admin] PATH_INFO: `/v3/roles/member` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:01:27.345709 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d85c37e5-fad3-4cc8-bdf9-c850a8243455 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 02 10:01:27.351843 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application [None req-d85c37e5-fad3-4cc8-bdf9-c850a8243455 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Aug 02 10:01:27.351843 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 02 10:01:27.351843 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 02 10:01:27.351843 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 02 10:01:27.351843 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 02 10:01:27.351843 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 02 10:01:27.351843 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 02 10:01:27.351843 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 02 10:01:27.351843 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 02 10:01:27.351843 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 02 10:01:27.351843 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 02 10:01:27.351843 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 02 10:01:27.351843 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 02 10:01:27.351843 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 02 10:01:27.351843 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 02 10:01:27.351843 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 02 10:01:27.352794 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 02 10:01:27.352794 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 02 10:01:27.352794 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 02 10:01:27.352794 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 02 10:01:27.352794 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 02 10:01:27.352794 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 02 10:01:27.352794 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 02 10:01:27.352794 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 02 10:01:27.352794 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 02 10:01:27.352794 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application return self.get_or_create( Aug 02 10:01:27.352794 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 02 10:01:27.352794 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application with Lock( Aug 02 10:01:27.352794 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 02 10:01:27.352794 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application return self._enter() Aug 02 10:01:27.352794 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 02 10:01:27.352794 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 02 10:01:27.353614 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 02 10:01:27.353614 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application return self.creator() Aug 02 10:01:27.353614 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 02 10:01:27.353614 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application created_value = creator( Aug 02 10:01:27.353614 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 02 10:01:27.353614 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 02 10:01:27.353614 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 02 10:01:27.353614 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 02 10:01:27.353614 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 02 10:01:27.353614 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 02 10:01:27.353614 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Aug 02 10:01:27.353614 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application Aug 02 10:01:27.356297 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: [pid: 80377|app: 0|req: 124/246] 10.222.0.82 () {56 vars in 1099 bytes} [Tue Aug 2 10:01:26 2022] GET /identity/v3/roles/member => generated 84 bytes in 362 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 02 10:01:27.366532 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d85c37e5-fad3-4cc8-bdf9-c850a8243455 None admin] Authenticating user token {{(pid=80377) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:01:27.377226 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG dbcounter [-] [80378] Writing DB stats keystone:SELECT=100 {{(pid=80378) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 10:01:27.398497 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG dbcounter [-] [80377] Writing DB stats keystone:SELECT=100 {{(pid=80377) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 10:01:27.462256 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 123/247] 10.222.0.82 () {58 vars in 1282 bytes} [Tue Aug 2 10:01:26 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 579 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 02 10:01:27.462729 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0ea277a-f45d-4920-b9a3-556287d4e4e0 None None] REQUEST_METHOD: `GET` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:01:27.462864 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0ea277a-f45d-4920-b9a3-556287d4e4e0 None None] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:01:27.463233 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0ea277a-f45d-4920-b9a3-556287d4e4e0 None None] PATH_INFO: `/` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:01:27.465479 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 124/248] 10.222.0.82 () {54 vars in 926 bytes} [Tue Aug 2 10:01:27 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 02 10:01:27.470653 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0ea277a-f45d-4920-b9a3-556287d4e4e0 None None] Authenticating user token {{(pid=80378) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:01:27.591513 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d85c37e5-fad3-4cc8-bdf9-c850a8243455 None admin] Validating token access rules against request {{(pid=80377) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:01:27.673430 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d638241-400e-4dd2-9384-bcf7358d200a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '891005fa5ff74041b75747d16cc66798', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80377) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:01:27.675661 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d638241-400e-4dd2-9384-bcf7358d200a None admin] REQUEST_METHOD: `GET` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:01:27.676364 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d638241-400e-4dd2-9384-bcf7358d200a None admin] SCRIPT_NAME: `/identity` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:01:27.677632 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0ea277a-f45d-4920-b9a3-556287d4e4e0 None None] Validating token access rules against request {{(pid=80378) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:01:27.678427 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d638241-400e-4dd2-9384-bcf7358d200a None admin] PATH_INFO: `/v3/roles` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:01:27.680029 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d638241-400e-4dd2-9384-bcf7358d200a None admin] RBAC: Adding query filter params (name=member) {{(pid=80377) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 02 10:01:27.684030 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d638241-400e-4dd2-9384-bcf7358d200a None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:01:27.687005 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d638241-400e-4dd2-9384-bcf7358d200a None admin] RBAC: Authorization granted {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:01:27.718212 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: [pid: 80377|app: 0|req: 125/249] 10.222.0.82 () {56 vars in 1101 bytes} [Tue Aug 2 10:01:27 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 353 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 02 10:01:27.721932 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fda2a003-315e-4c63-8a1d-f9b7222c5194 None None] REQUEST_METHOD: `POST` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:01:27.722841 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fda2a003-315e-4c63-8a1d-f9b7222c5194 None None] SCRIPT_NAME: `/identity` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:01:27.726715 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fda2a003-315e-4c63-8a1d-f9b7222c5194 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:01:27.767352 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.auth.core [None req-fda2a003-315e-4c63-8a1d-f9b7222c5194 None None] MFA Rules not processed for user `891005fa5ff74041b75747d16cc66798`. Rule list: `[]` (Enabled: `True`). {{(pid=80377) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 02 10:01:27.769905 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.fernet_utils [None req-fda2a003-315e-4c63-8a1d-f9b7222c5194 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80377) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 10:01:27.809072 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8516cd28-3715-4862-aa13-186e657b5858 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '891005fa5ff74041b75747d16cc66798', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80378) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:01:27.810151 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8516cd28-3715-4862-aa13-186e657b5858 None admin] REQUEST_METHOD: `GET` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:01:27.810629 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8516cd28-3715-4862-aa13-186e657b5858 None admin] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:01:27.811178 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8516cd28-3715-4862-aa13-186e657b5858 None admin] PATH_INFO: `/v3/projects/f7db7cf528494fbb8836c08d79b752ef` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:01:27.819163 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8516cd28-3715-4862-aa13-186e657b5858 None admin] RBAC: Authorizing `identity:get_project(project_id=f7db7cf528494fbb8836c08d79b752ef)` {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:01:27.821854 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8516cd28-3715-4862-aa13-186e657b5858 None admin] RBAC: Authorization granted {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:01:27.825022 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 125/250] 10.222.0.82 () {56 vars in 1186 bytes} [Tue Aug 2 10:01:27 2022] GET /identity/v3/projects/f7db7cf528494fbb8836c08d79b752ef => generated 302 bytes in 356 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 02 10:01:27.826376 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8516cd28-3715-4862-aa13-186e657b5858 None admin] Authenticating user token {{(pid=80378) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:01:27.920845 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: [pid: 80377|app: 0|req: 126/251] 10.222.0.82 () {58 vars in 1023 bytes} [Tue Aug 2 10:01:27 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 202 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 02 10:01:27.923391 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fda2a003-315e-4c63-8a1d-f9b7222c5194 None None] Authenticating user token {{(pid=80377) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:01:27.927609 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG dbcounter [-] [80377] Writing DB stats keystone:SELECT=100 {{(pid=80377) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 10:01:27.927802 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG dbcounter [-] [80378] Writing DB stats keystone:SELECT=100 {{(pid=80378) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 10:01:28.109983 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8516cd28-3715-4862-aa13-186e657b5858 None admin] Validating token access rules against request {{(pid=80378) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:01:28.208627 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e06350a-dec4-4f04-90bc-0f22428ee1be None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '891005fa5ff74041b75747d16cc66798', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80378) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:01:28.209715 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e06350a-dec4-4f04-90bc-0f22428ee1be None admin] REQUEST_METHOD: `GET` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:01:28.210213 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e06350a-dec4-4f04-90bc-0f22428ee1be None admin] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:01:28.213306 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e06350a-dec4-4f04-90bc-0f22428ee1be None admin] PATH_INFO: `/v3/users/e052f8df6a754a65b0c54a30f6794a67` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:01:28.216649 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e06350a-dec4-4f04-90bc-0f22428ee1be None admin] RBAC: Authorizing `identity:get_user(user_id=e052f8df6a754a65b0c54a30f6794a67)` {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:01:28.218215 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e06350a-dec4-4f04-90bc-0f22428ee1be None admin] RBAC: Authorization granted {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:01:28.223218 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 126/252] 10.222.0.82 () {56 vars in 1177 bytes} [Tue Aug 2 10:01:27 2022] GET /identity/v3/users/e052f8df6a754a65b0c54a30f6794a67 => generated 297 bytes in 398 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 02 10:01:28.224477 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e06350a-dec4-4f04-90bc-0f22428ee1be None admin] Authenticating user token {{(pid=80378) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:01:28.226266 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fda2a003-315e-4c63-8a1d-f9b7222c5194 None None] Validating token access rules against request {{(pid=80377) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:01:28.286088 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f66713b3-799a-4850-9190-d1e813477c77 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '891005fa5ff74041b75747d16cc66798', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80377) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:01:28.286741 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f66713b3-799a-4850-9190-d1e813477c77 None admin] REQUEST_METHOD: `GET` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:01:28.287021 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f66713b3-799a-4850-9190-d1e813477c77 None admin] SCRIPT_NAME: `/identity` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:01:28.287325 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f66713b3-799a-4850-9190-d1e813477c77 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:01:28.288178 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f66713b3-799a-4850-9190-d1e813477c77 None admin] RBAC: Adding query filter params (role.id=444e768aa5fb44bbab6c20afad6a0218, scope.project.id=f7db7cf528494fbb8836c08d79b752ef, user.id=b5222f0dc11f4c6998b83c676ee8e6b8) {{(pid=80377) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 02 10:01:28.288488 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f66713b3-799a-4850-9190-d1e813477c77 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:01:28.290829 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f66713b3-799a-4850-9190-d1e813477c77 None admin] RBAC: Authorization granted {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:01:28.300804 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: [pid: 80377|app: 0|req: 127/253] 10.222.0.82 () {56 vars in 1374 bytes} [Tue Aug 2 10:01:27 2022] GET /identity/v3/role_assignments?user.id=b5222f0dc11f4c6998b83c676ee8e6b8&scope.project.id=f7db7cf528494fbb8836c08d79b752ef&role.id=444e768aa5fb44bbab6c20afad6a0218 => generated 259 bytes in 379 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 02 10:01:28.301462 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f66713b3-799a-4850-9190-d1e813477c77 None admin] Authenticating user token {{(pid=80377) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:01:28.416009 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG dbcounter [-] [80377] Writing DB stats keystone:SELECT=100 {{(pid=80377) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 10:01:28.482512 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f66713b3-799a-4850-9190-d1e813477c77 None admin] Validating token access rules against request {{(pid=80377) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:01:28.547236 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG dbcounter [-] [80378] Writing DB stats keystone:SELECT=100 {{(pid=80378) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 10:01:28.555623 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e06350a-dec4-4f04-90bc-0f22428ee1be None admin] Validating token access rules against request {{(pid=80378) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:01:28.559840 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a99106cf-19f5-4e8a-903c-77b3b1da58f0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '891005fa5ff74041b75747d16cc66798', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80377) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:01:28.561709 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a99106cf-19f5-4e8a-903c-77b3b1da58f0 None admin] REQUEST_METHOD: `GET` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:01:28.562507 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a99106cf-19f5-4e8a-903c-77b3b1da58f0 None admin] SCRIPT_NAME: `/identity` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:01:28.562999 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a99106cf-19f5-4e8a-903c-77b3b1da58f0 None admin] PATH_INFO: `/v3/projects/f7db7cf528494fbb8836c08d79b752ef` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:01:28.571006 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a99106cf-19f5-4e8a-903c-77b3b1da58f0 None admin] RBAC: Authorizing `identity:get_project(project_id=f7db7cf528494fbb8836c08d79b752ef)` {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:01:28.573178 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a99106cf-19f5-4e8a-903c-77b3b1da58f0 None admin] RBAC: Authorization granted {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:01:28.576385 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: [pid: 80377|app: 0|req: 128/254] 10.222.0.82 () {56 vars in 1186 bytes} [Tue Aug 2 10:01:28 2022] GET /identity/v3/projects/f7db7cf528494fbb8836c08d79b752ef => generated 302 bytes in 275 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 02 10:01:28.586536 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a99106cf-19f5-4e8a-903c-77b3b1da58f0 None admin] Authenticating user token {{(pid=80377) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:01:28.683203 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfe8a584-7a28-4611-aa24-cb295bba478e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '891005fa5ff74041b75747d16cc66798', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80378) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:01:28.684267 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfe8a584-7a28-4611-aa24-cb295bba478e None admin] REQUEST_METHOD: `GET` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:01:28.684765 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfe8a584-7a28-4611-aa24-cb295bba478e None admin] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:01:28.685246 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfe8a584-7a28-4611-aa24-cb295bba478e None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:01:28.720169 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cfe8a584-7a28-4611-aa24-cb295bba478e None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 02 10:01:28.725718 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application [None req-cfe8a584-7a28-4611-aa24-cb295bba478e None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 02 10:01:28.725718 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 02 10:01:28.725718 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 02 10:01:28.725718 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 02 10:01:28.725718 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 02 10:01:28.725718 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 02 10:01:28.725718 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 02 10:01:28.725718 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 02 10:01:28.725718 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 02 10:01:28.725718 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 02 10:01:28.725718 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 02 10:01:28.725718 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 02 10:01:28.725718 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 02 10:01:28.725718 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 02 10:01:28.725718 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 02 10:01:28.725718 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 02 10:01:28.727123 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 02 10:01:28.727123 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 02 10:01:28.727123 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 02 10:01:28.727123 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 02 10:01:28.727123 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 02 10:01:28.727123 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 02 10:01:28.727123 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 02 10:01:28.727123 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 02 10:01:28.727123 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 02 10:01:28.727123 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application return self.get_or_create( Aug 02 10:01:28.727123 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 02 10:01:28.727123 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application with Lock( Aug 02 10:01:28.727123 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 02 10:01:28.727123 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application return self._enter() Aug 02 10:01:28.727123 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 02 10:01:28.727123 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 02 10:01:28.728311 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 02 10:01:28.728311 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application return self.creator() Aug 02 10:01:28.728311 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 02 10:01:28.728311 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application created_value = creator( Aug 02 10:01:28.728311 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 02 10:01:28.728311 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 02 10:01:28.728311 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 02 10:01:28.728311 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 02 10:01:28.728311 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 02 10:01:28.728311 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 02 10:01:28.728311 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 02 10:01:28.728311 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application Aug 02 10:01:28.728311 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 127/255] 10.222.0.82 () {56 vars in 1096 bytes} [Tue Aug 2 10:01:28 2022] GET /identity/v3/roles/admin => generated 83 bytes in 504 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 02 10:01:28.752450 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfe8a584-7a28-4611-aa24-cb295bba478e None admin] Authenticating user token {{(pid=80378) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:01:28.928303 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a99106cf-19f5-4e8a-903c-77b3b1da58f0 None admin] Validating token access rules against request {{(pid=80377) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:01:29.032213 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a97753f9-b423-4195-bfcb-54d4237d7428 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '891005fa5ff74041b75747d16cc66798', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80377) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:01:29.033231 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a97753f9-b423-4195-bfcb-54d4237d7428 None admin] REQUEST_METHOD: `PUT` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:01:29.033708 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a97753f9-b423-4195-bfcb-54d4237d7428 None admin] SCRIPT_NAME: `/identity` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:01:29.034329 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a97753f9-b423-4195-bfcb-54d4237d7428 None admin] PATH_INFO: `/v3/projects/f7db7cf528494fbb8836c08d79b752ef/users/e052f8df6a754a65b0c54a30f6794a67/roles/e1c04b19fc4543b8b54f58bbcffbc598` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:01:29.038296 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a97753f9-b423-4195-bfcb-54d4237d7428 None admin] RBAC: Authorizing `identity:create_grant(project_id=f7db7cf528494fbb8836c08d79b752ef, user_id=e052f8df6a754a65b0c54a30f6794a67, role_id=e1c04b19fc4543b8b54f58bbcffbc598)` {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:01:29.048700 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a97753f9-b423-4195-bfcb-54d4237d7428 None admin] RBAC: Authorization granted {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:01:29.049486 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG dbcounter [-] [80377] Writing DB stats keystone:SELECT=100 {{(pid=80377) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 10:01:29.080792 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: [pid: 80377|app: 0|req: 129/256] 10.222.0.82 () {58 vars in 1439 bytes} [Tue Aug 2 10:01:28 2022] PUT /identity/v3/projects/f7db7cf528494fbb8836c08d79b752ef/users/e052f8df6a754a65b0c54a30f6794a67/roles/e1c04b19fc4543b8b54f58bbcffbc598 => generated 0 bytes in 495 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 02 10:01:29.147054 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfe8a584-7a28-4611-aa24-cb295bba478e None admin] Validating token access rules against request {{(pid=80378) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:01:29.158794 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f3e9812-07c7-4181-b899-f640b10c47b0 None None] REQUEST_METHOD: `GET` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:01:29.159289 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f3e9812-07c7-4181-b899-f640b10c47b0 None None] SCRIPT_NAME: `/identity` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:01:29.159947 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f3e9812-07c7-4181-b899-f640b10c47b0 None None] PATH_INFO: `/` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:01:29.162759 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: [pid: 80377|app: 0|req: 130/257] 10.222.0.82 () {54 vars in 926 bytes} [Tue Aug 2 10:01:29 2022] GET /identity => generated 270 bytes in 7 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 02 10:01:29.172077 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c43bf5f1-57c7-4479-9e4b-1a35d3931c7a None None] REQUEST_METHOD: `POST` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:01:29.172521 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c43bf5f1-57c7-4479-9e4b-1a35d3931c7a None None] SCRIPT_NAME: `/identity` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:01:29.173509 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c43bf5f1-57c7-4479-9e4b-1a35d3931c7a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:01:29.225573 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.auth.core [None req-c43bf5f1-57c7-4479-9e4b-1a35d3931c7a None None] MFA Rules not processed for user `891005fa5ff74041b75747d16cc66798`. Rule list: `[]` (Enabled: `True`). {{(pid=80377) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 02 10:01:29.227324 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.fernet_utils [None req-c43bf5f1-57c7-4479-9e4b-1a35d3931c7a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80377) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 10:01:29.281804 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da2356bd-db9f-4c28-ab68-2ba9b24bd98c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '891005fa5ff74041b75747d16cc66798', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80378) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:01:29.283515 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da2356bd-db9f-4c28-ab68-2ba9b24bd98c None admin] REQUEST_METHOD: `GET` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:01:29.283925 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da2356bd-db9f-4c28-ab68-2ba9b24bd98c None admin] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:01:29.284390 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da2356bd-db9f-4c28-ab68-2ba9b24bd98c None admin] PATH_INFO: `/v3/roles` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:01:29.285753 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da2356bd-db9f-4c28-ab68-2ba9b24bd98c None admin] RBAC: Adding query filter params (name=admin) {{(pid=80378) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 02 10:01:29.286322 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da2356bd-db9f-4c28-ab68-2ba9b24bd98c None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:01:29.288521 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da2356bd-db9f-4c28-ab68-2ba9b24bd98c None admin] RBAC: Authorization granted {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:01:29.320534 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 128/258] 10.222.0.82 () {56 vars in 1099 bytes} [Tue Aug 2 10:01:28 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 574 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 02 10:01:29.328188 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da2356bd-db9f-4c28-ab68-2ba9b24bd98c None admin] Authenticating user token {{(pid=80378) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:01:29.402398 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG dbcounter [-] [80378] Writing DB stats keystone:SELECT=100 {{(pid=80378) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 10:01:29.443325 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: [pid: 80377|app: 0|req: 131/259] 10.222.0.82 () {58 vars in 1023 bytes} [Tue Aug 2 10:01:29 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 273 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Aug 02 10:01:29.511246 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c43bf5f1-57c7-4479-9e4b-1a35d3931c7a None None] Authenticating user token {{(pid=80377) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:01:29.568524 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da2356bd-db9f-4c28-ab68-2ba9b24bd98c None admin] Validating token access rules against request {{(pid=80378) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:01:29.659337 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d7115fc-2764-4d3b-bf8a-84c68bdd0f6f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '891005fa5ff74041b75747d16cc66798', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80378) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:01:29.663115 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d7115fc-2764-4d3b-bf8a-84c68bdd0f6f None admin] REQUEST_METHOD: `GET` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:01:29.664059 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d7115fc-2764-4d3b-bf8a-84c68bdd0f6f None admin] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:01:29.664814 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d7115fc-2764-4d3b-bf8a-84c68bdd0f6f None admin] PATH_INFO: `/v3/users/4b50324ba62a45b892d69d01fa699f73` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:01:29.668501 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d7115fc-2764-4d3b-bf8a-84c68bdd0f6f None admin] RBAC: Authorizing `identity:get_user(user_id=4b50324ba62a45b892d69d01fa699f73)` {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:01:29.673177 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d7115fc-2764-4d3b-bf8a-84c68bdd0f6f None admin] RBAC: Authorization granted {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:01:29.678031 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 129/260] 10.222.0.82 () {56 vars in 1177 bytes} [Tue Aug 2 10:01:29 2022] GET /identity/v3/users/4b50324ba62a45b892d69d01fa699f73 => generated 283 bytes in 350 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 02 10:01:29.696681 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d7115fc-2764-4d3b-bf8a-84c68bdd0f6f None admin] Authenticating user token {{(pid=80378) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:01:29.846778 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c43bf5f1-57c7-4479-9e4b-1a35d3931c7a None None] Validating token access rules against request {{(pid=80377) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:01:29.951697 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG dbcounter [-] [80377] Writing DB stats keystone:INSERT=1,keystone:SELECT=99 {{(pid=80377) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 10:01:29.965597 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG dbcounter [-] [80378] Writing DB stats keystone:SELECT=100 {{(pid=80378) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 10:01:29.994395 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7d8c3d7-aa5d-496f-93c8-4e1b42cef05e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '891005fa5ff74041b75747d16cc66798', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80377) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:01:29.996053 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7d8c3d7-aa5d-496f-93c8-4e1b42cef05e None admin] REQUEST_METHOD: `GET` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:01:29.997219 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7d8c3d7-aa5d-496f-93c8-4e1b42cef05e None admin] SCRIPT_NAME: `/identity` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:01:29.998352 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7d8c3d7-aa5d-496f-93c8-4e1b42cef05e None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:01:30.013075 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d7115fc-2764-4d3b-bf8a-84c68bdd0f6f None admin] Validating token access rules against request {{(pid=80378) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:01:30.050953 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7d8c3d7-aa5d-496f-93c8-4e1b42cef05e None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 02 10:01:30.071448 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application [None req-f7d8c3d7-aa5d-496f-93c8-4e1b42cef05e None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Aug 02 10:01:30.071448 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 02 10:01:30.071448 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 02 10:01:30.071448 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 02 10:01:30.071448 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 02 10:01:30.071448 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 02 10:01:30.071448 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 02 10:01:30.071448 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 02 10:01:30.071448 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 02 10:01:30.071448 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 02 10:01:30.071448 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 02 10:01:30.071448 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 02 10:01:30.071448 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 02 10:01:30.071448 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 02 10:01:30.071448 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 02 10:01:30.071448 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 02 10:01:30.073218 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 02 10:01:30.073218 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 02 10:01:30.073218 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 02 10:01:30.073218 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 02 10:01:30.073218 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 02 10:01:30.073218 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 02 10:01:30.073218 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 02 10:01:30.073218 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 02 10:01:30.073218 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 02 10:01:30.073218 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application return self.get_or_create( Aug 02 10:01:30.073218 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 02 10:01:30.073218 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application with Lock( Aug 02 10:01:30.073218 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 02 10:01:30.073218 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application return self._enter() Aug 02 10:01:30.073218 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 02 10:01:30.073218 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 02 10:01:30.075018 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 02 10:01:30.075018 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application return self.creator() Aug 02 10:01:30.075018 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 02 10:01:30.075018 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application created_value = creator( Aug 02 10:01:30.075018 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 02 10:01:30.075018 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 02 10:01:30.075018 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 02 10:01:30.075018 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 02 10:01:30.075018 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 02 10:01:30.075018 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 02 10:01:30.075018 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Aug 02 10:01:30.075018 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application Aug 02 10:01:30.081849 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: [pid: 80377|app: 0|req: 132/261] 10.222.0.82 () {56 vars in 1114 bytes} [Tue Aug 2 10:01:29 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 572 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 02 10:01:30.091791 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7d8c3d7-aa5d-496f-93c8-4e1b42cef05e None admin] Authenticating user token {{(pid=80377) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:01:30.183836 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a341c8a-d0b4-4127-9447-b6ccab268f24 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '891005fa5ff74041b75747d16cc66798', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80378) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:01:30.186041 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a341c8a-d0b4-4127-9447-b6ccab268f24 None admin] REQUEST_METHOD: `GET` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:01:30.186831 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a341c8a-d0b4-4127-9447-b6ccab268f24 None admin] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:01:30.188383 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a341c8a-d0b4-4127-9447-b6ccab268f24 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:01:30.334110 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a341c8a-d0b4-4127-9447-b6ccab268f24 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:01:30.335804 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a341c8a-d0b4-4127-9447-b6ccab268f24 None admin] RBAC: Authorization granted {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:01:30.462306 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7d8c3d7-aa5d-496f-93c8-4e1b42cef05e None admin] Validating token access rules against request {{(pid=80377) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:01:30.482354 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG dbcounter [-] [80378] Writing DB stats keystone:SELECT=100 {{(pid=80378) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 10:01:30.504824 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 130/262] 10.222.0.82 () {58 vars in 1282 bytes} [Tue Aug 2 10:01:29 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 809 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 02 10:01:30.513383 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a341c8a-d0b4-4127-9447-b6ccab268f24 None admin] Authenticating user token {{(pid=80378) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:01:30.547039 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17a920cf-d84d-422b-8955-96dd3625519c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '891005fa5ff74041b75747d16cc66798', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80377) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:01:30.548869 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17a920cf-d84d-422b-8955-96dd3625519c None admin] REQUEST_METHOD: `GET` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:01:30.549388 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17a920cf-d84d-422b-8955-96dd3625519c None admin] SCRIPT_NAME: `/identity` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:01:30.549904 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17a920cf-d84d-422b-8955-96dd3625519c None admin] PATH_INFO: `/v3/roles` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:01:30.551675 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17a920cf-d84d-422b-8955-96dd3625519c None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=80377) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 02 10:01:30.552233 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17a920cf-d84d-422b-8955-96dd3625519c None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:01:30.559068 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17a920cf-d84d-422b-8955-96dd3625519c None admin] RBAC: Authorization granted {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:01:30.586721 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: [pid: 80377|app: 0|req: 133/263] 10.222.0.82 () {56 vars in 1111 bytes} [Tue Aug 2 10:01:30 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 496 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 02 10:01:30.596173 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17a920cf-d84d-422b-8955-96dd3625519c None admin] Authenticating user token {{(pid=80377) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:01:30.721779 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG dbcounter [-] [80377] Writing DB stats keystone:SELECT=100 {{(pid=80377) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 10:01:30.828764 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a341c8a-d0b4-4127-9447-b6ccab268f24 None admin] Validating token access rules against request {{(pid=80378) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:01:30.884923 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17a920cf-d84d-422b-8955-96dd3625519c None admin] Validating token access rules against request {{(pid=80377) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:01:30.947075 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fcc0f9dd-b7e4-4770-978c-0eaf20605b46 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '891005fa5ff74041b75747d16cc66798', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80378) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:01:30.948580 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcc0f9dd-b7e4-4770-978c-0eaf20605b46 None admin] REQUEST_METHOD: `GET` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:01:30.948742 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d2c2e6a-ed08-4c65-998c-14f06419ab52 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '891005fa5ff74041b75747d16cc66798', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80377) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:01:30.949357 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d2c2e6a-ed08-4c65-998c-14f06419ab52 None admin] REQUEST_METHOD: `GET` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:01:30.949639 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d2c2e6a-ed08-4c65-998c-14f06419ab52 None admin] SCRIPT_NAME: `/identity` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:01:30.949939 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d2c2e6a-ed08-4c65-998c-14f06419ab52 None admin] PATH_INFO: `/v3/users/4b50324ba62a45b892d69d01fa699f73` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:01:30.950451 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcc0f9dd-b7e4-4770-978c-0eaf20605b46 None admin] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:01:30.951223 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcc0f9dd-b7e4-4770-978c-0eaf20605b46 None admin] PATH_INFO: `/v3/projects/f7db7cf528494fbb8836c08d79b752ef` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:01:30.953779 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fcc0f9dd-b7e4-4770-978c-0eaf20605b46 None admin] RBAC: Authorizing `identity:get_project(project_id=f7db7cf528494fbb8836c08d79b752ef)` {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:01:30.954727 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d2c2e6a-ed08-4c65-998c-14f06419ab52 None admin] RBAC: Authorizing `identity:get_user(user_id=4b50324ba62a45b892d69d01fa699f73)` {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:01:30.956223 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d2c2e6a-ed08-4c65-998c-14f06419ab52 None admin] RBAC: Authorization granted {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:01:30.956709 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fcc0f9dd-b7e4-4770-978c-0eaf20605b46 None admin] RBAC: Authorization granted {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:01:30.957987 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: [pid: 80377|app: 0|req: 134/264] 10.222.0.82 () {56 vars in 1177 bytes} [Tue Aug 2 10:01:30 2022] GET /identity/v3/users/4b50324ba62a45b892d69d01fa699f73 => generated 283 bytes in 364 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 02 10:01:30.960893 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 131/265] 10.222.0.82 () {56 vars in 1186 bytes} [Tue Aug 2 10:01:30 2022] GET /identity/v3/projects/f7db7cf528494fbb8836c08d79b752ef => generated 302 bytes in 448 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 02 10:01:30.966532 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d2c2e6a-ed08-4c65-998c-14f06419ab52 None admin] Authenticating user token {{(pid=80377) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:01:30.971040 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fcc0f9dd-b7e4-4770-978c-0eaf20605b46 None admin] Authenticating user token {{(pid=80378) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:01:31.129467 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG dbcounter [-] [80378] Writing DB stats keystone:SELECT=100 {{(pid=80378) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 10:01:31.164140 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d2c2e6a-ed08-4c65-998c-14f06419ab52 None admin] Validating token access rules against request {{(pid=80377) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:01:31.218297 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG dbcounter [-] [80377] Writing DB stats keystone:SELECT=100 {{(pid=80377) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 10:01:31.284014 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fcc0f9dd-b7e4-4770-978c-0eaf20605b46 None admin] Validating token access rules against request {{(pid=80378) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:01:31.286982 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38edd491-7a25-4769-95a4-dca61d685180 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '891005fa5ff74041b75747d16cc66798', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80377) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:01:31.288047 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38edd491-7a25-4769-95a4-dca61d685180 None admin] REQUEST_METHOD: `GET` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:01:31.288553 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38edd491-7a25-4769-95a4-dca61d685180 None admin] SCRIPT_NAME: `/identity` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:01:31.289065 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38edd491-7a25-4769-95a4-dca61d685180 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:01:31.415343 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9470e830-6c06-4da3-be62-e76e799b1010 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '891005fa5ff74041b75747d16cc66798', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80378) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:01:31.416268 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9470e830-6c06-4da3-be62-e76e799b1010 None admin] REQUEST_METHOD: `GET` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:01:31.416728 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9470e830-6c06-4da3-be62-e76e799b1010 None admin] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:01:31.417181 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9470e830-6c06-4da3-be62-e76e799b1010 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:01:31.418618 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9470e830-6c06-4da3-be62-e76e799b1010 None admin] RBAC: Adding query filter params (role.id=237966442f05411191105bf336bc14f3, scope.project.id=f7db7cf528494fbb8836c08d79b752ef, user.id=4b50324ba62a45b892d69d01fa699f73) {{(pid=80378) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 02 10:01:31.419135 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9470e830-6c06-4da3-be62-e76e799b1010 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:01:31.421692 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9470e830-6c06-4da3-be62-e76e799b1010 None admin] RBAC: Authorization granted {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:01:31.446281 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 132/266] 10.222.0.82 () {56 vars in 1374 bytes} [Tue Aug 2 10:01:30 2022] GET /identity/v3/role_assignments?user.id=4b50324ba62a45b892d69d01fa699f73&scope.project.id=f7db7cf528494fbb8836c08d79b752ef&role.id=237966442f05411191105bf336bc14f3 => generated 608 bytes in 476 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 02 10:01:31.456548 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38edd491-7a25-4769-95a4-dca61d685180 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:01:31.459613 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38edd491-7a25-4769-95a4-dca61d685180 None admin] RBAC: Authorization granted {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:01:31.601419 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9e721b4-24f6-426c-bc04-dfa7c05e0be7 None None] REQUEST_METHOD: `GET` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:01:31.605979 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9e721b4-24f6-426c-bc04-dfa7c05e0be7 None None] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:01:31.606539 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9e721b4-24f6-426c-bc04-dfa7c05e0be7 None None] PATH_INFO: `/` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:01:31.609400 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 133/267] 10.222.0.82 () {54 vars in 926 bytes} [Tue Aug 2 10:01:31 2022] GET /identity => generated 270 bytes in 10 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 02 10:01:31.618421 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11a4f8ee-a075-40bb-80d1-4a9938081518 None None] REQUEST_METHOD: `POST` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:01:31.618875 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11a4f8ee-a075-40bb-80d1-4a9938081518 None None] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:01:31.619330 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11a4f8ee-a075-40bb-80d1-4a9938081518 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:01:31.663119 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: [pid: 80377|app: 0|req: 135/268] 10.222.0.82 () {58 vars in 1282 bytes} [Tue Aug 2 10:01:30 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 697 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 02 10:01:31.665576 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-283b6295-06c1-46cf-b1df-4d27f1b33e51 None None] REQUEST_METHOD: `GET` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:01:31.666065 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-283b6295-06c1-46cf-b1df-4d27f1b33e51 None None] SCRIPT_NAME: `/identity` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:01:31.666584 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-283b6295-06c1-46cf-b1df-4d27f1b33e51 None None] PATH_INFO: `/` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:01:31.669537 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: [pid: 80377|app: 0|req: 136/269] 10.222.0.82 () {54 vars in 926 bytes} [Tue Aug 2 10:01:31 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 02 10:01:31.671101 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-283b6295-06c1-46cf-b1df-4d27f1b33e51 None None] Authenticating user token {{(pid=80377) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:01:31.675466 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.auth.core [None req-11a4f8ee-a075-40bb-80d1-4a9938081518 None None] MFA Rules not processed for user `891005fa5ff74041b75747d16cc66798`. Rule list: `[]` (Enabled: `True`). {{(pid=80378) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 02 10:01:31.677592 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.fernet_utils [None req-11a4f8ee-a075-40bb-80d1-4a9938081518 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80378) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 10:01:31.739317 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG dbcounter [-] [80377] Writing DB stats keystone:SELECT=100 {{(pid=80377) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 10:01:31.833931 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 134/270] 10.222.0.82 () {58 vars in 1023 bytes} [Tue Aug 2 10:01:31 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 217 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 02 10:01:31.840660 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e607d2c-72ec-41b5-b82e-f4c912381fb3 None None] REQUEST_METHOD: `POST` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:01:31.841430 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e607d2c-72ec-41b5-b82e-f4c912381fb3 None None] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:01:31.844405 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e607d2c-72ec-41b5-b82e-f4c912381fb3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:01:31.891892 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.auth.core [None req-3e607d2c-72ec-41b5-b82e-f4c912381fb3 None None] MFA Rules not processed for user `891005fa5ff74041b75747d16cc66798`. Rule list: `[]` (Enabled: `True`). {{(pid=80378) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 02 10:01:31.894569 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.fernet_utils [None req-3e607d2c-72ec-41b5-b82e-f4c912381fb3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80378) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 10:01:31.983645 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-283b6295-06c1-46cf-b1df-4d27f1b33e51 None None] Validating token access rules against request {{(pid=80377) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:01:32.002650 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG dbcounter [-] [80378] Writing DB stats keystone:SELECT=100 {{(pid=80378) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 10:01:32.047655 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 135/271] 10.222.0.82 () {58 vars in 1023 bytes} [Tue Aug 2 10:01:31 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 209 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 02 10:01:32.050234 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e607d2c-72ec-41b5-b82e-f4c912381fb3 None None] Authenticating user token {{(pid=80378) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:01:32.052178 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5094f1c8-83aa-4eae-acc9-fd6f29bbd48c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '891005fa5ff74041b75747d16cc66798', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80377) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:01:32.053307 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5094f1c8-83aa-4eae-acc9-fd6f29bbd48c None admin] REQUEST_METHOD: `GET` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:01:32.055474 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5094f1c8-83aa-4eae-acc9-fd6f29bbd48c None admin] SCRIPT_NAME: `/identity` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:01:32.056051 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5094f1c8-83aa-4eae-acc9-fd6f29bbd48c None admin] PATH_INFO: `/v3/projects/f7db7cf528494fbb8836c08d79b752ef` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:01:32.058338 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5094f1c8-83aa-4eae-acc9-fd6f29bbd48c None admin] RBAC: Authorizing `identity:get_project(project_id=f7db7cf528494fbb8836c08d79b752ef)` {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:01:32.062473 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5094f1c8-83aa-4eae-acc9-fd6f29bbd48c None admin] RBAC: Authorization granted {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:01:32.067079 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: [pid: 80377|app: 0|req: 137/272] 10.222.0.82 () {56 vars in 1186 bytes} [Tue Aug 2 10:01:31 2022] GET /identity/v3/projects/f7db7cf528494fbb8836c08d79b752ef => generated 302 bytes in 397 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 02 10:01:32.075024 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5094f1c8-83aa-4eae-acc9-fd6f29bbd48c None admin] Authenticating user token {{(pid=80377) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:01:32.223699 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG dbcounter [-] [80377] Writing DB stats keystone:SELECT=100 {{(pid=80377) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 10:01:32.244880 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e607d2c-72ec-41b5-b82e-f4c912381fb3 None None] Validating token access rules against request {{(pid=80378) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:01:32.259826 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5094f1c8-83aa-4eae-acc9-fd6f29bbd48c None admin] Validating token access rules against request {{(pid=80377) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:01:32.297572 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77312444-53ce-45f1-adc0-c75c224ce9be None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '891005fa5ff74041b75747d16cc66798', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80378) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:01:32.298321 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77312444-53ce-45f1-adc0-c75c224ce9be None admin] REQUEST_METHOD: `GET` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:01:32.298590 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77312444-53ce-45f1-adc0-c75c224ce9be None admin] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:01:32.298861 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77312444-53ce-45f1-adc0-c75c224ce9be None admin] PATH_INFO: `/v3/domains/default` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:01:32.300065 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77312444-53ce-45f1-adc0-c75c224ce9be None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:01:32.301539 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77312444-53ce-45f1-adc0-c75c224ce9be None admin] RBAC: Authorization granted {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:01:32.303524 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 136/273] 10.222.0.82 () {56 vars in 1108 bytes} [Tue Aug 2 10:01:32 2022] GET /identity/v3/domains/default => generated 199 bytes in 254 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 02 10:01:32.304340 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77312444-53ce-45f1-adc0-c75c224ce9be None admin] Authenticating user token {{(pid=80378) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:01:32.328602 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a64dedc7-30a8-45de-8622-1cb4897c832d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '891005fa5ff74041b75747d16cc66798', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80377) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:01:32.331495 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a64dedc7-30a8-45de-8622-1cb4897c832d None admin] REQUEST_METHOD: `GET` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:01:32.332634 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a64dedc7-30a8-45de-8622-1cb4897c832d None admin] SCRIPT_NAME: `/identity` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:01:32.333393 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a64dedc7-30a8-45de-8622-1cb4897c832d None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:01:32.335367 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a64dedc7-30a8-45de-8622-1cb4897c832d None admin] RBAC: Adding query filter params (role.id=649993d933d44c50b268df7fdf336fbf, scope.project.id=f7db7cf528494fbb8836c08d79b752ef, user.id=4b50324ba62a45b892d69d01fa699f73) {{(pid=80377) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 02 10:01:32.336192 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a64dedc7-30a8-45de-8622-1cb4897c832d None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:01:32.339901 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a64dedc7-30a8-45de-8622-1cb4897c832d None admin] RBAC: Authorization granted {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:01:32.353154 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: [pid: 80377|app: 0|req: 138/274] 10.222.0.82 () {56 vars in 1374 bytes} [Tue Aug 2 10:01:32 2022] GET /identity/v3/role_assignments?user.id=4b50324ba62a45b892d69d01fa699f73&scope.project.id=f7db7cf528494fbb8836c08d79b752ef&role.id=649993d933d44c50b268df7fdf336fbf => generated 608 bytes in 279 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 02 10:01:32.355120 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a64dedc7-30a8-45de-8622-1cb4897c832d None admin] Authenticating user token {{(pid=80377) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:01:32.397361 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG dbcounter [-] [80378] Writing DB stats keystone:SELECT=100 {{(pid=80378) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 10:01:32.496455 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77312444-53ce-45f1-adc0-c75c224ce9be None admin] Validating token access rules against request {{(pid=80378) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:01:32.555211 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2c43663-fa0a-4baa-8b44-dc1435b5902b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '891005fa5ff74041b75747d16cc66798', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80378) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:01:32.556715 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2c43663-fa0a-4baa-8b44-dc1435b5902b None admin] REQUEST_METHOD: `GET` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:01:32.557325 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2c43663-fa0a-4baa-8b44-dc1435b5902b None admin] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:01:32.557933 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2c43663-fa0a-4baa-8b44-dc1435b5902b None admin] PATH_INFO: `/v3/roles/member` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:01:32.567829 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a64dedc7-30a8-45de-8622-1cb4897c832d None admin] Validating token access rules against request {{(pid=80377) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:01:32.576187 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2c43663-fa0a-4baa-8b44-dc1435b5902b None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 02 10:01:32.583702 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application [None req-b2c43663-fa0a-4baa-8b44-dc1435b5902b None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Aug 02 10:01:32.583702 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 02 10:01:32.583702 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 02 10:01:32.583702 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 02 10:01:32.583702 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 02 10:01:32.583702 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 02 10:01:32.583702 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 02 10:01:32.583702 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 02 10:01:32.583702 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 02 10:01:32.583702 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 02 10:01:32.583702 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 02 10:01:32.583702 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 02 10:01:32.583702 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 02 10:01:32.583702 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 02 10:01:32.583702 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 02 10:01:32.583702 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 02 10:01:32.585065 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 02 10:01:32.585065 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 02 10:01:32.585065 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 02 10:01:32.585065 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 02 10:01:32.585065 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 02 10:01:32.585065 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 02 10:01:32.585065 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 02 10:01:32.585065 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 02 10:01:32.585065 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 02 10:01:32.585065 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application return self.get_or_create( Aug 02 10:01:32.585065 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 02 10:01:32.585065 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application with Lock( Aug 02 10:01:32.585065 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 02 10:01:32.585065 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application return self._enter() Aug 02 10:01:32.585065 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 02 10:01:32.585065 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 02 10:01:32.586607 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 02 10:01:32.586607 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application return self.creator() Aug 02 10:01:32.586607 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 02 10:01:32.586607 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application created_value = creator( Aug 02 10:01:32.586607 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 02 10:01:32.586607 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 02 10:01:32.586607 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 02 10:01:32.586607 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 02 10:01:32.586607 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 02 10:01:32.586607 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 02 10:01:32.586607 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Aug 02 10:01:32.586607 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application Aug 02 10:01:32.590401 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 137/275] 10.222.0.82 () {56 vars in 1099 bytes} [Tue Aug 2 10:01:32 2022] GET /identity/v3/roles/member => generated 84 bytes in 286 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 02 10:01:32.592517 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25b6326d-46af-4af0-9bb2-2fd1d1ef9530 None None] REQUEST_METHOD: `GET` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:01:32.593352 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25b6326d-46af-4af0-9bb2-2fd1d1ef9530 None None] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:01:32.595507 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25b6326d-46af-4af0-9bb2-2fd1d1ef9530 None None] PATH_INFO: `/` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:01:32.597682 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 138/276] 10.222.0.82 () {54 vars in 926 bytes} [Tue Aug 2 10:01:32 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 02 10:01:32.599677 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25b6326d-46af-4af0-9bb2-2fd1d1ef9530 None None] Authenticating user token {{(pid=80378) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:01:32.629383 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2068050e-4bf0-40db-85ee-04997e75d50c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '891005fa5ff74041b75747d16cc66798', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80377) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:01:32.630461 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2068050e-4bf0-40db-85ee-04997e75d50c None admin] REQUEST_METHOD: `POST` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:01:32.630934 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2068050e-4bf0-40db-85ee-04997e75d50c None admin] SCRIPT_NAME: `/identity` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:01:32.631398 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2068050e-4bf0-40db-85ee-04997e75d50c None admin] PATH_INFO: `/v3/users` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:01:32.633379 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2068050e-4bf0-40db-85ee-04997e75d50c None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:01:32.635999 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2068050e-4bf0-40db-85ee-04997e75d50c None admin] RBAC: Authorization granted {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:01:32.681027 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: [pid: 80377|app: 0|req: 139/277] 10.222.0.82 () {60 vars in 1132 bytes} [Tue Aug 2 10:01:32 2022] POST /identity/v3/users => generated 293 bytes in 327 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 02 10:01:32.688872 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d1fb1e9-c8fa-4487-b0c9-f1be46f1775e None None] REQUEST_METHOD: `POST` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:01:32.689287 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d1fb1e9-c8fa-4487-b0c9-f1be46f1775e None None] SCRIPT_NAME: `/identity` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:01:32.689653 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d1fb1e9-c8fa-4487-b0c9-f1be46f1775e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:01:32.700416 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG dbcounter [-] [80377] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80377) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 10:01:32.724343 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.auth.core [None req-8d1fb1e9-c8fa-4487-b0c9-f1be46f1775e None None] MFA Rules not processed for user `891005fa5ff74041b75747d16cc66798`. Rule list: `[]` (Enabled: `True`). {{(pid=80377) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 02 10:01:32.726685 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.fernet_utils [None req-8d1fb1e9-c8fa-4487-b0c9-f1be46f1775e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80377) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 10:01:32.804370 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25b6326d-46af-4af0-9bb2-2fd1d1ef9530 None None] Validating token access rules against request {{(pid=80378) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:01:32.821740 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG dbcounter [-] [80378] Writing DB stats keystone:SELECT=100 {{(pid=80378) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 10:01:32.862204 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: [pid: 80377|app: 0|req: 140/278] 10.222.0.82 () {58 vars in 1023 bytes} [Tue Aug 2 10:01:32 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 180 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 02 10:01:32.872308 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8cbdbd90-1180-4994-a2aa-981030c8a857 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '891005fa5ff74041b75747d16cc66798', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80378) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:01:32.873395 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cbdbd90-1180-4994-a2aa-981030c8a857 None admin] REQUEST_METHOD: `GET` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:01:32.873912 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cbdbd90-1180-4994-a2aa-981030c8a857 None admin] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:01:32.874397 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cbdbd90-1180-4994-a2aa-981030c8a857 None admin] PATH_INFO: `/v3/roles` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:01:32.875397 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8cbdbd90-1180-4994-a2aa-981030c8a857 None admin] RBAC: Adding query filter params (name=member) {{(pid=80378) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 02 10:01:32.875982 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8cbdbd90-1180-4994-a2aa-981030c8a857 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:01:32.877995 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8cbdbd90-1180-4994-a2aa-981030c8a857 None admin] RBAC: Authorization granted {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:01:32.884892 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d1fb1e9-c8fa-4487-b0c9-f1be46f1775e None None] Authenticating user token {{(pid=80377) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:01:32.895772 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 139/279] 10.222.0.82 () {56 vars in 1101 bytes} [Tue Aug 2 10:01:32 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 296 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 02 10:01:32.901341 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8cbdbd90-1180-4994-a2aa-981030c8a857 None admin] Authenticating user token {{(pid=80378) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:01:33.056402 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d1fb1e9-c8fa-4487-b0c9-f1be46f1775e None None] Validating token access rules against request {{(pid=80377) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:01:33.080652 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8cbdbd90-1180-4994-a2aa-981030c8a857 None admin] Validating token access rules against request {{(pid=80378) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:01:33.116347 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG dbcounter [-] [80377] Writing DB stats keystone:SELECT=100 {{(pid=80377) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 10:01:33.121480 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c4d7efc-8eab-4c22-bd89-7a61dabd70b4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '891005fa5ff74041b75747d16cc66798', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80377) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:01:33.122174 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c4d7efc-8eab-4c22-bd89-7a61dabd70b4 None admin] REQUEST_METHOD: `GET` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:01:33.122460 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c4d7efc-8eab-4c22-bd89-7a61dabd70b4 None admin] SCRIPT_NAME: `/identity` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:01:33.124383 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c4d7efc-8eab-4c22-bd89-7a61dabd70b4 None admin] PATH_INFO: `/v3/roles/reader` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:01:33.139426 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce37e471-27bb-4e8a-8a53-59e439e471b7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '891005fa5ff74041b75747d16cc66798', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80378) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:01:33.140424 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce37e471-27bb-4e8a-8a53-59e439e471b7 None admin] REQUEST_METHOD: `GET` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:01:33.140706 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce37e471-27bb-4e8a-8a53-59e439e471b7 None admin] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:01:33.141007 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce37e471-27bb-4e8a-8a53-59e439e471b7 None admin] PATH_INFO: `/v3/users/a5edd0d9ceec465d852028e78855bbe3` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:01:33.143466 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c4d7efc-8eab-4c22-bd89-7a61dabd70b4 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 02 10:01:33.151024 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application [None req-1c4d7efc-8eab-4c22-bd89-7a61dabd70b4 None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Aug 02 10:01:33.151024 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 02 10:01:33.151024 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 02 10:01:33.151024 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 02 10:01:33.151024 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 02 10:01:33.151024 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 02 10:01:33.151024 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 02 10:01:33.151024 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 02 10:01:33.151024 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 02 10:01:33.151024 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 02 10:01:33.151024 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 02 10:01:33.151024 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 02 10:01:33.151024 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 02 10:01:33.151024 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 02 10:01:33.151024 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 02 10:01:33.151024 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 02 10:01:33.151973 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 02 10:01:33.151973 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 02 10:01:33.151973 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 02 10:01:33.151973 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 02 10:01:33.151973 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 02 10:01:33.151973 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 02 10:01:33.151973 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 02 10:01:33.151973 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 02 10:01:33.151973 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 02 10:01:33.151973 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application return self.get_or_create( Aug 02 10:01:33.151973 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 02 10:01:33.151973 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application with Lock( Aug 02 10:01:33.151973 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 02 10:01:33.151973 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application return self._enter() Aug 02 10:01:33.151973 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 02 10:01:33.151973 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 02 10:01:33.152889 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 02 10:01:33.152889 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application return self.creator() Aug 02 10:01:33.152889 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 02 10:01:33.152889 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application created_value = creator( Aug 02 10:01:33.152889 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 02 10:01:33.152889 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 02 10:01:33.152889 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 02 10:01:33.152889 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 02 10:01:33.152889 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 02 10:01:33.152889 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 02 10:01:33.152889 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Aug 02 10:01:33.152889 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application Aug 02 10:01:33.156148 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: [pid: 80377|app: 0|req: 141/280] 10.222.0.82 () {56 vars in 1099 bytes} [Tue Aug 2 10:01:32 2022] GET /identity/v3/roles/reader => generated 84 bytes in 272 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 02 10:01:33.164057 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c4d7efc-8eab-4c22-bd89-7a61dabd70b4 None admin] Authenticating user token {{(pid=80377) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:01:33.166970 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce37e471-27bb-4e8a-8a53-59e439e471b7 None admin] RBAC: Authorizing `identity:get_user(user_id=a5edd0d9ceec465d852028e78855bbe3)` {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:01:33.168648 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce37e471-27bb-4e8a-8a53-59e439e471b7 None admin] RBAC: Authorization granted {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:01:33.170773 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 140/281] 10.222.0.82 () {56 vars in 1177 bytes} [Tue Aug 2 10:01:32 2022] GET /identity/v3/users/a5edd0d9ceec465d852028e78855bbe3 => generated 293 bytes in 270 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 02 10:01:33.178444 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce37e471-27bb-4e8a-8a53-59e439e471b7 None admin] Authenticating user token {{(pid=80378) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:01:33.217594 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG dbcounter [-] [80378] Writing DB stats keystone:SELECT=100 {{(pid=80378) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 10:01:33.345403 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c4d7efc-8eab-4c22-bd89-7a61dabd70b4 None admin] Validating token access rules against request {{(pid=80377) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:01:33.350707 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce37e471-27bb-4e8a-8a53-59e439e471b7 None admin] Validating token access rules against request {{(pid=80378) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:01:33.397122 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68c5ac51-16d3-466e-be12-cfc4fea87a2c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '891005fa5ff74041b75747d16cc66798', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80378) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:01:33.397761 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68c5ac51-16d3-466e-be12-cfc4fea87a2c None admin] REQUEST_METHOD: `GET` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:01:33.398066 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68c5ac51-16d3-466e-be12-cfc4fea87a2c None admin] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:01:33.398490 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68c5ac51-16d3-466e-be12-cfc4fea87a2c None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:01:33.399304 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68c5ac51-16d3-466e-be12-cfc4fea87a2c None admin] RBAC: Adding query filter params (role.id=e1c04b19fc4543b8b54f58bbcffbc598, user.id=a5edd0d9ceec465d852028e78855bbe3, scope.system=all) {{(pid=80378) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 02 10:01:33.399616 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68c5ac51-16d3-466e-be12-cfc4fea87a2c None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:01:33.401154 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68c5ac51-16d3-466e-be12-cfc4fea87a2c None admin] RBAC: Authorization granted {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:01:33.408078 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 141/282] 10.222.0.82 () {56 vars in 1308 bytes} [Tue Aug 2 10:01:33 2022] GET /identity/v3/role_assignments?user.id=a5edd0d9ceec465d852028e78855bbe3&scope.system=all&role.id=e1c04b19fc4543b8b54f58bbcffbc598 => generated 226 bytes in 230 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 02 10:01:33.434247 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fbeeee4-cb2f-4b43-868d-7b61a866f01e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '891005fa5ff74041b75747d16cc66798', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80377) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:01:33.434888 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fbeeee4-cb2f-4b43-868d-7b61a866f01e None admin] REQUEST_METHOD: `GET` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:01:33.435172 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fbeeee4-cb2f-4b43-868d-7b61a866f01e None admin] SCRIPT_NAME: `/identity` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:01:33.435452 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fbeeee4-cb2f-4b43-868d-7b61a866f01e None admin] PATH_INFO: `/v3/roles` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:01:33.436184 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4fbeeee4-cb2f-4b43-868d-7b61a866f01e None admin] RBAC: Adding query filter params (name=reader) {{(pid=80377) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 02 10:01:33.436491 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4fbeeee4-cb2f-4b43-868d-7b61a866f01e None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:01:33.438046 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4fbeeee4-cb2f-4b43-868d-7b61a866f01e None admin] RBAC: Authorization granted {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:01:33.452524 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: [pid: 80377|app: 0|req: 142/283] 10.222.0.82 () {56 vars in 1101 bytes} [Tue Aug 2 10:01:33 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 290 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 02 10:01:33.461924 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68c5ac51-16d3-466e-be12-cfc4fea87a2c None admin] Authenticating user token {{(pid=80378) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:01:33.593840 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG dbcounter [-] [80378] Writing DB stats keystone:SELECT=100 {{(pid=80378) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 10:01:33.607220 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68c5ac51-16d3-466e-be12-cfc4fea87a2c None admin] Validating token access rules against request {{(pid=80378) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:01:33.655910 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3202ebc-5530-458d-b6a9-85215ab624ce None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '891005fa5ff74041b75747d16cc66798', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80378) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:01:33.656593 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3202ebc-5530-458d-b6a9-85215ab624ce None admin] REQUEST_METHOD: `GET` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:01:33.656884 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3202ebc-5530-458d-b6a9-85215ab624ce None admin] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:01:33.662516 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3202ebc-5530-458d-b6a9-85215ab624ce None admin] PATH_INFO: `/v3/users/b5222f0dc11f4c6998b83c676ee8e6b8` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:01:33.664560 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3202ebc-5530-458d-b6a9-85215ab624ce None admin] RBAC: Authorizing `identity:get_user(user_id=b5222f0dc11f4c6998b83c676ee8e6b8)` {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:01:33.666171 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3202ebc-5530-458d-b6a9-85215ab624ce None admin] RBAC: Authorization granted {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:01:33.668135 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 142/284] 10.222.0.82 () {56 vars in 1177 bytes} [Tue Aug 2 10:01:33 2022] GET /identity/v3/users/b5222f0dc11f4c6998b83c676ee8e6b8 => generated 297 bytes in 207 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 02 10:01:33.670808 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-691f6504-fbf0-44d1-ade0-2667d7bdf139 None None] REQUEST_METHOD: `GET` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:01:33.671085 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-691f6504-fbf0-44d1-ade0-2667d7bdf139 None None] SCRIPT_NAME: `/identity` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:01:33.671372 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-691f6504-fbf0-44d1-ade0-2667d7bdf139 None None] PATH_INFO: `/` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:01:33.673269 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: [pid: 80377|app: 0|req: 143/285] 10.222.0.82 () {54 vars in 926 bytes} [Tue Aug 2 10:01:33 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 02 10:01:33.677360 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3202ebc-5530-458d-b6a9-85215ab624ce None admin] Authenticating user token {{(pid=80378) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:01:33.678943 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92d72077-18cc-43b8-8da9-f3c85b8394e2 None None] REQUEST_METHOD: `POST` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:01:33.679243 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92d72077-18cc-43b8-8da9-f3c85b8394e2 None None] SCRIPT_NAME: `/identity` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:01:33.679594 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92d72077-18cc-43b8-8da9-f3c85b8394e2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:01:33.751558 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.auth.core [None req-92d72077-18cc-43b8-8da9-f3c85b8394e2 None None] MFA Rules not processed for user `891005fa5ff74041b75747d16cc66798`. Rule list: `[]` (Enabled: `True`). {{(pid=80377) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 02 10:01:33.753610 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.fernet_utils [None req-92d72077-18cc-43b8-8da9-f3c85b8394e2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80377) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 10:01:33.854996 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3202ebc-5530-458d-b6a9-85215ab624ce None admin] Validating token access rules against request {{(pid=80378) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:01:33.884495 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG dbcounter [-] [80377] Writing DB stats keystone:SELECT=100 {{(pid=80377) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 10:01:33.936814 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: [pid: 80377|app: 0|req: 144/286] 10.222.0.82 () {58 vars in 1023 bytes} [Tue Aug 2 10:01:33 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 259 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Aug 02 10:01:33.954543 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c08b8a8-833a-4b6b-8116-c7a529a3ca80 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '891005fa5ff74041b75747d16cc66798', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80378) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:01:33.957705 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c08b8a8-833a-4b6b-8116-c7a529a3ca80 None admin] REQUEST_METHOD: `GET` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:01:33.958516 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c08b8a8-833a-4b6b-8116-c7a529a3ca80 None admin] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:01:33.959283 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c08b8a8-833a-4b6b-8116-c7a529a3ca80 None admin] PATH_INFO: `/v3/projects/f7db7cf528494fbb8836c08d79b752ef` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:01:33.962074 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c08b8a8-833a-4b6b-8116-c7a529a3ca80 None admin] RBAC: Authorizing `identity:get_project(project_id=f7db7cf528494fbb8836c08d79b752ef)` {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:01:33.965170 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c08b8a8-833a-4b6b-8116-c7a529a3ca80 None admin] RBAC: Authorization granted {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:01:33.969168 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 143/287] 10.222.0.82 () {56 vars in 1186 bytes} [Tue Aug 2 10:01:33 2022] GET /identity/v3/projects/f7db7cf528494fbb8836c08d79b752ef => generated 302 bytes in 292 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 02 10:01:33.978839 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92d72077-18cc-43b8-8da9-f3c85b8394e2 None None] Authenticating user token {{(pid=80377) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:01:33.987771 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c08b8a8-833a-4b6b-8116-c7a529a3ca80 None admin] Authenticating user token {{(pid=80378) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:01:34.038905 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG dbcounter [-] [80378] Writing DB stats keystone:SELECT=100 {{(pid=80378) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 10:01:34.159458 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c08b8a8-833a-4b6b-8116-c7a529a3ca80 None admin] Validating token access rules against request {{(pid=80378) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:01:34.227779 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3791654d-137c-4a82-a015-d77d82f9c0bc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '891005fa5ff74041b75747d16cc66798', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80378) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:01:34.228694 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3791654d-137c-4a82-a015-d77d82f9c0bc None admin] REQUEST_METHOD: `GET` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:01:34.229086 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3791654d-137c-4a82-a015-d77d82f9c0bc None admin] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:01:34.229474 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3791654d-137c-4a82-a015-d77d82f9c0bc None admin] PATH_INFO: `/v3/roles/member` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:01:34.257179 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3791654d-137c-4a82-a015-d77d82f9c0bc None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 02 10:01:34.282662 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application [None req-3791654d-137c-4a82-a015-d77d82f9c0bc None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Aug 02 10:01:34.282662 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 02 10:01:34.282662 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 02 10:01:34.282662 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 02 10:01:34.282662 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 02 10:01:34.282662 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 02 10:01:34.282662 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 02 10:01:34.282662 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 02 10:01:34.282662 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 02 10:01:34.282662 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 02 10:01:34.282662 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 02 10:01:34.282662 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 02 10:01:34.282662 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 02 10:01:34.282662 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 02 10:01:34.282662 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 02 10:01:34.282662 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 02 10:01:34.284362 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 02 10:01:34.284362 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 02 10:01:34.284362 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 02 10:01:34.284362 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 02 10:01:34.284362 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 02 10:01:34.284362 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 02 10:01:34.284362 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 02 10:01:34.284362 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 02 10:01:34.284362 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 02 10:01:34.284362 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application return self.get_or_create( Aug 02 10:01:34.284362 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 02 10:01:34.284362 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application with Lock( Aug 02 10:01:34.284362 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 02 10:01:34.284362 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application return self._enter() Aug 02 10:01:34.284362 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 02 10:01:34.284362 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 02 10:01:34.285911 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 02 10:01:34.285911 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application return self.creator() Aug 02 10:01:34.285911 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 02 10:01:34.285911 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application created_value = creator( Aug 02 10:01:34.285911 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 02 10:01:34.285911 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 02 10:01:34.285911 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 02 10:01:34.285911 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 02 10:01:34.285911 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 02 10:01:34.285911 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 02 10:01:34.285911 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Aug 02 10:01:34.285911 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application Aug 02 10:01:34.294938 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 144/288] 10.222.0.82 () {56 vars in 1099 bytes} [Tue Aug 2 10:01:33 2022] GET /identity/v3/roles/member => generated 84 bytes in 306 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 02 10:01:34.299694 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3791654d-137c-4a82-a015-d77d82f9c0bc None admin] Authenticating user token {{(pid=80378) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:01:34.326040 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92d72077-18cc-43b8-8da9-f3c85b8394e2 None None] Validating token access rules against request {{(pid=80377) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:01:34.405464 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-354098d9-764a-42ee-a386-e7be2d134c40 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '891005fa5ff74041b75747d16cc66798', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80377) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:01:34.406968 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-354098d9-764a-42ee-a386-e7be2d134c40 None admin] REQUEST_METHOD: `PUT` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:01:34.408090 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-354098d9-764a-42ee-a386-e7be2d134c40 None admin] SCRIPT_NAME: `/identity` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:01:34.408521 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-354098d9-764a-42ee-a386-e7be2d134c40 None admin] PATH_INFO: `/v3/projects/f7db7cf528494fbb8836c08d79b752ef/users/b5222f0dc11f4c6998b83c676ee8e6b8/roles/444e768aa5fb44bbab6c20afad6a0218` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:01:34.419257 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-354098d9-764a-42ee-a386-e7be2d134c40 None admin] RBAC: Authorizing `identity:create_grant(project_id=f7db7cf528494fbb8836c08d79b752ef, user_id=b5222f0dc11f4c6998b83c676ee8e6b8, role_id=444e768aa5fb44bbab6c20afad6a0218)` {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:01:34.421428 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-354098d9-764a-42ee-a386-e7be2d134c40 None admin] RBAC: Authorization granted {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:01:34.445210 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: [pid: 80377|app: 0|req: 145/289] 10.222.0.82 () {58 vars in 1439 bytes} [Tue Aug 2 10:01:33 2022] PUT /identity/v3/projects/f7db7cf528494fbb8836c08d79b752ef/users/b5222f0dc11f4c6998b83c676ee8e6b8/roles/444e768aa5fb44bbab6c20afad6a0218 => generated 0 bytes in 467 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 02 10:01:34.470614 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG dbcounter [-] [80378] Writing DB stats keystone:SELECT=100 {{(pid=80378) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 10:01:34.494520 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3791654d-137c-4a82-a015-d77d82f9c0bc None admin] Validating token access rules against request {{(pid=80378) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:01:34.548468 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-536bda53-485c-4b15-ba34-cec1d319f8c7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '891005fa5ff74041b75747d16cc66798', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80378) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:01:34.549481 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-536bda53-485c-4b15-ba34-cec1d319f8c7 None admin] REQUEST_METHOD: `GET` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:01:34.549964 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-536bda53-485c-4b15-ba34-cec1d319f8c7 None admin] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:01:34.550518 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-536bda53-485c-4b15-ba34-cec1d319f8c7 None admin] PATH_INFO: `/v3/roles` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:01:34.551840 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-536bda53-485c-4b15-ba34-cec1d319f8c7 None admin] RBAC: Adding query filter params (name=member) {{(pid=80378) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 02 10:01:34.552333 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-536bda53-485c-4b15-ba34-cec1d319f8c7 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:01:34.554915 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-536bda53-485c-4b15-ba34-cec1d319f8c7 None admin] RBAC: Authorization granted {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:01:34.576446 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 145/290] 10.222.0.82 () {56 vars in 1101 bytes} [Tue Aug 2 10:01:34 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 278 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 02 10:01:34.585937 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-354098d9-764a-42ee-a386-e7be2d134c40 None admin] Authenticating user token {{(pid=80377) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:01:34.692682 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG dbcounter [-] [80377] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80377) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 10:01:34.803478 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-354098d9-764a-42ee-a386-e7be2d134c40 None admin] Validating token access rules against request {{(pid=80377) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:01:34.859964 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b70c56f3-57d2-4734-997f-599cec307d3f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '891005fa5ff74041b75747d16cc66798', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80377) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:01:34.859964 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b70c56f3-57d2-4734-997f-599cec307d3f None admin] REQUEST_METHOD: `GET` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:01:34.859964 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b70c56f3-57d2-4734-997f-599cec307d3f None admin] SCRIPT_NAME: `/identity` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:01:34.859964 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b70c56f3-57d2-4734-997f-599cec307d3f None admin] PATH_INFO: `/v3/users/e052f8df6a754a65b0c54a30f6794a67` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:01:34.861664 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b70c56f3-57d2-4734-997f-599cec307d3f None admin] RBAC: Authorizing `identity:get_user(user_id=e052f8df6a754a65b0c54a30f6794a67)` {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:01:34.863309 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b70c56f3-57d2-4734-997f-599cec307d3f None admin] RBAC: Authorization granted {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:01:34.865123 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: [pid: 80377|app: 0|req: 146/291] 10.222.0.82 () {56 vars in 1177 bytes} [Tue Aug 2 10:01:34 2022] GET /identity/v3/users/e052f8df6a754a65b0c54a30f6794a67 => generated 297 bytes in 280 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 02 10:01:34.871882 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-536bda53-485c-4b15-ba34-cec1d319f8c7 None admin] Authenticating user token {{(pid=80378) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:01:35.084981 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-536bda53-485c-4b15-ba34-cec1d319f8c7 None admin] Validating token access rules against request {{(pid=80378) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:01:35.152955 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e50c06b5-3c91-4621-8615-794ac9310d55 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '891005fa5ff74041b75747d16cc66798', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80378) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:01:35.154070 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e50c06b5-3c91-4621-8615-794ac9310d55 None admin] REQUEST_METHOD: `GET` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:01:35.154529 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e50c06b5-3c91-4621-8615-794ac9310d55 None admin] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:01:35.154932 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e50c06b5-3c91-4621-8615-794ac9310d55 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:01:35.199420 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG dbcounter [-] [80378] Writing DB stats keystone:SELECT=100 {{(pid=80378) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 10:01:35.277569 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e50c06b5-3c91-4621-8615-794ac9310d55 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:01:35.279095 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e50c06b5-3c91-4621-8615-794ac9310d55 None admin] RBAC: Authorization granted {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:01:35.466993 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 146/292] 10.222.0.82 () {58 vars in 1282 bytes} [Tue Aug 2 10:01:34 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 596 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 02 10:01:35.475811 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b70c56f3-57d2-4734-997f-599cec307d3f None admin] Authenticating user token {{(pid=80377) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:01:35.701910 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b70c56f3-57d2-4734-997f-599cec307d3f None admin] Validating token access rules against request {{(pid=80377) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:01:35.721343 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG dbcounter [-] [80377] Writing DB stats keystone:SELECT=100 {{(pid=80377) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 10:01:35.770866 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e391f2a-54fd-43dc-8b32-4b6b2d5d273c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '891005fa5ff74041b75747d16cc66798', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80377) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:01:35.772009 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e391f2a-54fd-43dc-8b32-4b6b2d5d273c None admin] REQUEST_METHOD: `GET` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:01:35.772517 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e391f2a-54fd-43dc-8b32-4b6b2d5d273c None admin] SCRIPT_NAME: `/identity` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:01:35.773027 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e391f2a-54fd-43dc-8b32-4b6b2d5d273c None admin] PATH_INFO: `/v3/projects/f7db7cf528494fbb8836c08d79b752ef` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:01:35.779243 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e391f2a-54fd-43dc-8b32-4b6b2d5d273c None admin] RBAC: Authorizing `identity:get_project(project_id=f7db7cf528494fbb8836c08d79b752ef)` {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:01:35.782792 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e391f2a-54fd-43dc-8b32-4b6b2d5d273c None admin] RBAC: Authorization granted {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:01:35.785663 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: [pid: 80377|app: 0|req: 147/293] 10.222.0.82 () {56 vars in 1186 bytes} [Tue Aug 2 10:01:35 2022] GET /identity/v3/projects/f7db7cf528494fbb8836c08d79b752ef => generated 302 bytes in 311 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 02 10:01:35.794423 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e50c06b5-3c91-4621-8615-794ac9310d55 None admin] Authenticating user token {{(pid=80378) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:01:35.902835 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG dbcounter [-] [80378] Writing DB stats keystone:SELECT=100 {{(pid=80378) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 10:01:36.025066 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e50c06b5-3c91-4621-8615-794ac9310d55 None admin] Validating token access rules against request {{(pid=80378) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:01:36.083822 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8891be1-2efd-4bcc-9769-3a21c8e4ecfc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '891005fa5ff74041b75747d16cc66798', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80378) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:01:36.084617 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8891be1-2efd-4bcc-9769-3a21c8e4ecfc None admin] REQUEST_METHOD: `GET` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:01:36.084899 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8891be1-2efd-4bcc-9769-3a21c8e4ecfc None admin] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:01:36.085173 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8891be1-2efd-4bcc-9769-3a21c8e4ecfc None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:01:36.088308 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8891be1-2efd-4bcc-9769-3a21c8e4ecfc None admin] RBAC: Adding query filter params (role.id=e1c04b19fc4543b8b54f58bbcffbc598, scope.project.id=f7db7cf528494fbb8836c08d79b752ef, user.id=e052f8df6a754a65b0c54a30f6794a67) {{(pid=80378) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 02 10:01:36.097174 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8891be1-2efd-4bcc-9769-3a21c8e4ecfc None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:01:36.097174 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8891be1-2efd-4bcc-9769-3a21c8e4ecfc None admin] RBAC: Authorization granted {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:01:36.103763 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 147/294] 10.222.0.82 () {56 vars in 1374 bytes} [Tue Aug 2 10:01:35 2022] GET /identity/v3/role_assignments?user.id=e052f8df6a754a65b0c54a30f6794a67&scope.project.id=f7db7cf528494fbb8836c08d79b752ef&role.id=e1c04b19fc4543b8b54f58bbcffbc598 => generated 608 bytes in 310 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 02 10:01:36.467839 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73612bc4-c968-4436-ac2f-bb2ed08540ef None None] REQUEST_METHOD: `GET` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:01:36.468754 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73612bc4-c968-4436-ac2f-bb2ed08540ef None None] SCRIPT_NAME: `/identity` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:01:36.469443 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73612bc4-c968-4436-ac2f-bb2ed08540ef None None] PATH_INFO: `/` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:01:36.472478 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: [pid: 80377|app: 0|req: 148/295] 10.222.0.82 () {54 vars in 926 bytes} [Tue Aug 2 10:01:36 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 02 10:01:36.479035 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25bedf0d-2d3f-4735-9003-be6ef551e504 None None] REQUEST_METHOD: `POST` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:01:36.479880 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25bedf0d-2d3f-4735-9003-be6ef551e504 None None] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:01:36.480624 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25bedf0d-2d3f-4735-9003-be6ef551e504 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:01:36.516355 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.auth.core [None req-25bedf0d-2d3f-4735-9003-be6ef551e504 None None] MFA Rules not processed for user `891005fa5ff74041b75747d16cc66798`. Rule list: `[]` (Enabled: `True`). {{(pid=80378) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 02 10:01:36.518674 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.fernet_utils [None req-25bedf0d-2d3f-4735-9003-be6ef551e504 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80378) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 10:01:36.639963 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f294c4eb-2ab0-421e-8a0d-957cc776db33 None None] REQUEST_METHOD: `GET` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:01:36.640983 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f294c4eb-2ab0-421e-8a0d-957cc776db33 None None] SCRIPT_NAME: `/identity` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:01:36.641459 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f294c4eb-2ab0-421e-8a0d-957cc776db33 None None] PATH_INFO: `/` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:01:36.644490 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: [pid: 80377|app: 0|req: 149/296] 10.222.0.82 () {54 vars in 926 bytes} [Tue Aug 2 10:01:36 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 02 10:01:36.650340 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 148/297] 10.222.0.82 () {58 vars in 1023 bytes} [Tue Aug 2 10:01:36 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 172 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 02 10:01:36.652679 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54fc2c52-4b35-4a33-a3f5-d50f8a7dc3f4 None None] REQUEST_METHOD: `POST` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:01:36.653363 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54fc2c52-4b35-4a33-a3f5-d50f8a7dc3f4 None None] SCRIPT_NAME: `/identity` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:01:36.653934 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54fc2c52-4b35-4a33-a3f5-d50f8a7dc3f4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:01:36.684726 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.auth.core [None req-54fc2c52-4b35-4a33-a3f5-d50f8a7dc3f4 None None] MFA Rules not processed for user `891005fa5ff74041b75747d16cc66798`. Rule list: `[]` (Enabled: `True`). {{(pid=80377) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 02 10:01:36.686401 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.fernet_utils [None req-54fc2c52-4b35-4a33-a3f5-d50f8a7dc3f4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80377) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 10:01:36.694261 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25bedf0d-2d3f-4735-9003-be6ef551e504 None None] Authenticating user token {{(pid=80378) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:01:36.787769 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG dbcounter [-] [80378] Writing DB stats keystone:SELECT=100 {{(pid=80378) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 10:01:36.870899 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: [pid: 80377|app: 0|req: 150/298] 10.222.0.82 () {58 vars in 1023 bytes} [Tue Aug 2 10:01:36 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 219 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Aug 02 10:01:36.913358 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54fc2c52-4b35-4a33-a3f5-d50f8a7dc3f4 None None] Authenticating user token {{(pid=80377) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:01:36.941093 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25bedf0d-2d3f-4735-9003-be6ef551e504 None None] Validating token access rules against request {{(pid=80378) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:01:37.010007 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-260c82dd-9aaf-4049-84cf-7fcd5a8c4bc0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '891005fa5ff74041b75747d16cc66798', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80378) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:01:37.010996 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-260c82dd-9aaf-4049-84cf-7fcd5a8c4bc0 None admin] REQUEST_METHOD: `GET` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:01:37.011447 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-260c82dd-9aaf-4049-84cf-7fcd5a8c4bc0 None admin] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:01:37.012604 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-260c82dd-9aaf-4049-84cf-7fcd5a8c4bc0 None admin] PATH_INFO: `/v3/roles/reader` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:01:37.036959 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-260c82dd-9aaf-4049-84cf-7fcd5a8c4bc0 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 02 10:01:37.045442 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application [None req-260c82dd-9aaf-4049-84cf-7fcd5a8c4bc0 None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Aug 02 10:01:37.045442 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 02 10:01:37.045442 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 02 10:01:37.045442 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 02 10:01:37.045442 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 02 10:01:37.045442 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 02 10:01:37.045442 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 02 10:01:37.045442 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 02 10:01:37.045442 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 02 10:01:37.045442 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 02 10:01:37.045442 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 02 10:01:37.045442 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 02 10:01:37.045442 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 02 10:01:37.045442 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 02 10:01:37.045442 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 02 10:01:37.045442 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 02 10:01:37.046440 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 02 10:01:37.046440 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 02 10:01:37.046440 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 02 10:01:37.046440 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 02 10:01:37.046440 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 02 10:01:37.046440 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 02 10:01:37.046440 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 02 10:01:37.046440 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 02 10:01:37.046440 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 02 10:01:37.046440 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application return self.get_or_create( Aug 02 10:01:37.046440 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 02 10:01:37.046440 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application with Lock( Aug 02 10:01:37.046440 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 02 10:01:37.046440 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application return self._enter() Aug 02 10:01:37.046440 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 02 10:01:37.046440 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 02 10:01:37.049073 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 02 10:01:37.049073 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application return self.creator() Aug 02 10:01:37.049073 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 02 10:01:37.049073 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application created_value = creator( Aug 02 10:01:37.049073 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 02 10:01:37.049073 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 02 10:01:37.049073 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 02 10:01:37.049073 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 02 10:01:37.049073 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 02 10:01:37.049073 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 02 10:01:37.049073 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Aug 02 10:01:37.049073 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application Aug 02 10:01:37.049073 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 149/299] 10.222.0.82 () {56 vars in 1099 bytes} [Tue Aug 2 10:01:36 2022] GET /identity/v3/roles/reader => generated 84 bytes in 353 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 02 10:01:37.055195 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-260c82dd-9aaf-4049-84cf-7fcd5a8c4bc0 None admin] Authenticating user token {{(pid=80378) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:01:37.100138 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54fc2c52-4b35-4a33-a3f5-d50f8a7dc3f4 None None] Validating token access rules against request {{(pid=80377) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:01:37.122288 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG dbcounter [-] [80377] Writing DB stats keystone:SELECT=100 {{(pid=80377) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 10:01:37.162854 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7338254c-218f-415c-860d-84e3ef485584 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '891005fa5ff74041b75747d16cc66798', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80377) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:01:37.163519 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7338254c-218f-415c-860d-84e3ef485584 None admin] REQUEST_METHOD: `GET` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:01:37.163873 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7338254c-218f-415c-860d-84e3ef485584 None admin] SCRIPT_NAME: `/identity` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:01:37.164172 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7338254c-218f-415c-860d-84e3ef485584 None admin] PATH_INFO: `/v3/domains/default` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:01:37.165503 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7338254c-218f-415c-860d-84e3ef485584 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:01:37.167028 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7338254c-218f-415c-860d-84e3ef485584 None admin] RBAC: Authorization granted {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:01:37.169810 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: [pid: 80377|app: 0|req: 151/300] 10.222.0.82 () {56 vars in 1108 bytes} [Tue Aug 2 10:01:36 2022] GET /identity/v3/domains/default => generated 199 bytes in 256 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 02 10:01:37.170111 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6aa81eb4-b286-436a-bb87-fb9bbd2fcdbb None None] REQUEST_METHOD: `GET` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:01:37.170347 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6aa81eb4-b286-436a-bb87-fb9bbd2fcdbb None None] SCRIPT_NAME: `/identity` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:01:37.170783 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6aa81eb4-b286-436a-bb87-fb9bbd2fcdbb None None] PATH_INFO: `/` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:01:37.174035 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: [pid: 80377|app: 0|req: 152/301] 10.222.0.82 () {54 vars in 926 bytes} [Tue Aug 2 10:01:37 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 02 10:01:37.178925 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6aa81eb4-b286-436a-bb87-fb9bbd2fcdbb None None] Authenticating user token {{(pid=80377) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:01:37.245107 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-260c82dd-9aaf-4049-84cf-7fcd5a8c4bc0 None admin] Validating token access rules against request {{(pid=80378) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:01:37.247899 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG dbcounter [-] [80378] Writing DB stats keystone:SELECT=100 {{(pid=80378) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 10:01:37.323423 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-201d99e2-7d91-47da-b55f-7f4762395da2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '891005fa5ff74041b75747d16cc66798', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80378) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:01:37.323423 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-201d99e2-7d91-47da-b55f-7f4762395da2 None admin] REQUEST_METHOD: `GET` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:01:37.323423 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-201d99e2-7d91-47da-b55f-7f4762395da2 None admin] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:01:37.323423 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-201d99e2-7d91-47da-b55f-7f4762395da2 None admin] PATH_INFO: `/v3/roles` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:01:37.324708 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-201d99e2-7d91-47da-b55f-7f4762395da2 None admin] RBAC: Adding query filter params (name=reader) {{(pid=80378) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 02 10:01:37.324708 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-201d99e2-7d91-47da-b55f-7f4762395da2 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:01:37.326320 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-201d99e2-7d91-47da-b55f-7f4762395da2 None admin] RBAC: Authorization granted {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:01:37.340928 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 150/302] 10.222.0.82 () {56 vars in 1101 bytes} [Tue Aug 2 10:01:37 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 286 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 02 10:01:37.350640 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36aa0244-264a-47ca-b0d7-ba6e55159d63 None None] REQUEST_METHOD: `POST` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:01:37.350640 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36aa0244-264a-47ca-b0d7-ba6e55159d63 None None] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:01:37.351190 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36aa0244-264a-47ca-b0d7-ba6e55159d63 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:01:37.351313 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6aa81eb4-b286-436a-bb87-fb9bbd2fcdbb None None] Validating token access rules against request {{(pid=80377) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:01:37.383566 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.auth.core [None req-36aa0244-264a-47ca-b0d7-ba6e55159d63 None None] MFA Rules not processed for user `891005fa5ff74041b75747d16cc66798`. Rule list: `[]` (Enabled: `True`). {{(pid=80378) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 02 10:01:37.385533 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.fernet_utils [None req-36aa0244-264a-47ca-b0d7-ba6e55159d63 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80378) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 10:01:37.403111 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7acfe7cd-39a8-46d4-b8a4-2e90effd81d3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '891005fa5ff74041b75747d16cc66798', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80377) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:01:37.404020 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7acfe7cd-39a8-46d4-b8a4-2e90effd81d3 None admin] REQUEST_METHOD: `POST` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:01:37.404455 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7acfe7cd-39a8-46d4-b8a4-2e90effd81d3 None admin] SCRIPT_NAME: `/identity` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:01:37.404924 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7acfe7cd-39a8-46d4-b8a4-2e90effd81d3 None admin] PATH_INFO: `/v3/groups` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:01:37.406092 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7acfe7cd-39a8-46d4-b8a4-2e90effd81d3 None admin] RBAC: Authorizing `identity:create_group()` {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:01:37.408729 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7acfe7cd-39a8-46d4-b8a4-2e90effd81d3 None admin] RBAC: Authorization granted {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:01:37.437825 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: [pid: 80377|app: 0|req: 153/303] 10.222.0.82 () {60 vars in 1134 bytes} [Tue Aug 2 10:01:37 2022] POST /identity/v3/groups => generated 229 bytes in 252 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 02 10:01:37.437825 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7acfe7cd-39a8-46d4-b8a4-2e90effd81d3 None admin] Authenticating user token {{(pid=80377) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:01:37.514034 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 151/304] 10.222.0.82 () {58 vars in 1023 bytes} [Tue Aug 2 10:01:37 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 165 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 02 10:01:37.528975 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG dbcounter [-] [80377] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80377) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 10:01:37.544039 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36aa0244-264a-47ca-b0d7-ba6e55159d63 None None] Authenticating user token {{(pid=80378) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:01:37.629638 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7acfe7cd-39a8-46d4-b8a4-2e90effd81d3 None admin] Validating token access rules against request {{(pid=80377) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:01:37.696743 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b3b5309-af41-4f14-87da-b51923133b65 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '891005fa5ff74041b75747d16cc66798', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80377) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:01:37.697950 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b3b5309-af41-4f14-87da-b51923133b65 None admin] REQUEST_METHOD: `GET` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:01:37.698368 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b3b5309-af41-4f14-87da-b51923133b65 None admin] SCRIPT_NAME: `/identity` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:01:37.698888 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b3b5309-af41-4f14-87da-b51923133b65 None admin] PATH_INFO: `/v3/users/8a9b9d020f064f5ab9f664ee649e8389` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:01:37.722408 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b3b5309-af41-4f14-87da-b51923133b65 None admin] RBAC: Authorizing `identity:get_user(user_id=8a9b9d020f064f5ab9f664ee649e8389)` {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:01:37.724669 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b3b5309-af41-4f14-87da-b51923133b65 None admin] RBAC: Authorization granted {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:01:37.727177 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: [pid: 80377|app: 0|req: 154/305] 10.222.0.82 () {56 vars in 1177 bytes} [Tue Aug 2 10:01:37 2022] GET /identity/v3/users/8a9b9d020f064f5ab9f664ee649e8389 => generated 293 bytes in 297 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 02 10:01:37.738254 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b3b5309-af41-4f14-87da-b51923133b65 None admin] Authenticating user token {{(pid=80377) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:01:37.749973 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG dbcounter [-] [80378] Writing DB stats keystone:SELECT=100 {{(pid=80378) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 10:01:37.764152 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36aa0244-264a-47ca-b0d7-ba6e55159d63 None None] Validating token access rules against request {{(pid=80378) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:01:37.842131 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f83a0e7a-7e80-4595-b6dc-96e4c87be839 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '891005fa5ff74041b75747d16cc66798', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80378) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:01:37.844107 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f83a0e7a-7e80-4595-b6dc-96e4c87be839 None admin] REQUEST_METHOD: `GET` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:01:37.845452 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f83a0e7a-7e80-4595-b6dc-96e4c87be839 None admin] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:01:37.846221 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f83a0e7a-7e80-4595-b6dc-96e4c87be839 None admin] PATH_INFO: `/v3/roles/member` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:01:37.871386 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f83a0e7a-7e80-4595-b6dc-96e4c87be839 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 02 10:01:37.883053 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application [None req-f83a0e7a-7e80-4595-b6dc-96e4c87be839 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Aug 02 10:01:37.883053 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 02 10:01:37.883053 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 02 10:01:37.883053 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 02 10:01:37.883053 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 02 10:01:37.883053 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 02 10:01:37.883053 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 02 10:01:37.883053 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 02 10:01:37.883053 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 02 10:01:37.883053 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 02 10:01:37.883053 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 02 10:01:37.883053 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 02 10:01:37.883053 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 02 10:01:37.883053 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 02 10:01:37.883053 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 02 10:01:37.883053 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 02 10:01:37.884591 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 02 10:01:37.884591 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 02 10:01:37.884591 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 02 10:01:37.884591 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 02 10:01:37.884591 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 02 10:01:37.884591 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 02 10:01:37.884591 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 02 10:01:37.884591 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 02 10:01:37.884591 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 02 10:01:37.884591 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application return self.get_or_create( Aug 02 10:01:37.884591 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 02 10:01:37.884591 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application with Lock( Aug 02 10:01:37.884591 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 02 10:01:37.884591 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application return self._enter() Aug 02 10:01:37.884591 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 02 10:01:37.884591 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 02 10:01:37.885608 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 02 10:01:37.885608 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application return self.creator() Aug 02 10:01:37.885608 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 02 10:01:37.885608 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application created_value = creator( Aug 02 10:01:37.885608 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 02 10:01:37.885608 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 02 10:01:37.885608 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 02 10:01:37.885608 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 02 10:01:37.885608 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 02 10:01:37.885608 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 02 10:01:37.885608 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Aug 02 10:01:37.885608 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application Aug 02 10:01:37.888798 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 152/306] 10.222.0.82 () {56 vars in 1099 bytes} [Tue Aug 2 10:01:37 2022] GET /identity/v3/roles/member => generated 84 bytes in 345 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 02 10:01:37.898386 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f83a0e7a-7e80-4595-b6dc-96e4c87be839 None admin] Authenticating user token {{(pid=80378) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:01:37.941353 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG dbcounter [-] [80377] Writing DB stats keystone:SELECT=100 {{(pid=80377) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 10:01:37.953627 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b3b5309-af41-4f14-87da-b51923133b65 None admin] Validating token access rules against request {{(pid=80377) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:01:38.022481 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1324b8c6-f38e-40b2-ba9e-231405826f12 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '891005fa5ff74041b75747d16cc66798', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80377) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:01:38.024147 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1324b8c6-f38e-40b2-ba9e-231405826f12 None admin] REQUEST_METHOD: `GET` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:01:38.026004 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1324b8c6-f38e-40b2-ba9e-231405826f12 None admin] SCRIPT_NAME: `/identity` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:01:38.026541 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1324b8c6-f38e-40b2-ba9e-231405826f12 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:01:38.027943 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1324b8c6-f38e-40b2-ba9e-231405826f12 None admin] RBAC: Adding query filter params (role.id=444e768aa5fb44bbab6c20afad6a0218, user.id=8a9b9d020f064f5ab9f664ee649e8389, scope.system=all) {{(pid=80377) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 02 10:01:38.028432 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1324b8c6-f38e-40b2-ba9e-231405826f12 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:01:38.030201 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1324b8c6-f38e-40b2-ba9e-231405826f12 None admin] RBAC: Authorization granted {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:01:38.040715 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: [pid: 80377|app: 0|req: 155/307] 10.222.0.82 () {56 vars in 1308 bytes} [Tue Aug 2 10:01:37 2022] GET /identity/v3/role_assignments?user.id=8a9b9d020f064f5ab9f664ee649e8389&scope.system=all&role.id=444e768aa5fb44bbab6c20afad6a0218 => generated 226 bytes in 304 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 02 10:01:38.050871 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9df80c32-cb18-481e-b11c-755569732096 None None] REQUEST_METHOD: `GET` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:01:38.052982 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9df80c32-cb18-481e-b11c-755569732096 None None] SCRIPT_NAME: `/identity` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:01:38.053761 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9df80c32-cb18-481e-b11c-755569732096 None None] PATH_INFO: `/` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:01:38.056873 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: [pid: 80377|app: 0|req: 156/308] 10.222.0.82 () {54 vars in 926 bytes} [Tue Aug 2 10:01:38 2022] GET /identity => generated 270 bytes in 9 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 02 10:01:38.065812 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cba0f8b3-1296-40f4-9b8b-45fe7c354b02 None None] REQUEST_METHOD: `POST` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:01:38.067955 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cba0f8b3-1296-40f4-9b8b-45fe7c354b02 None None] SCRIPT_NAME: `/identity` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:01:38.068552 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cba0f8b3-1296-40f4-9b8b-45fe7c354b02 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:01:38.090213 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f83a0e7a-7e80-4595-b6dc-96e4c87be839 None admin] Validating token access rules against request {{(pid=80378) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:01:38.099527 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.auth.core [None req-cba0f8b3-1296-40f4-9b8b-45fe7c354b02 None None] MFA Rules not processed for user `891005fa5ff74041b75747d16cc66798`. Rule list: `[]` (Enabled: `True`). {{(pid=80377) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 02 10:01:38.101707 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.fernet_utils [None req-cba0f8b3-1296-40f4-9b8b-45fe7c354b02 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80377) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 10:01:38.170552 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd1e8e48-a45d-411c-81c2-0acf9149f9bd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '891005fa5ff74041b75747d16cc66798', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80378) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:01:38.170552 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd1e8e48-a45d-411c-81c2-0acf9149f9bd None admin] REQUEST_METHOD: `GET` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:01:38.173105 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd1e8e48-a45d-411c-81c2-0acf9149f9bd None admin] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:01:38.173105 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd1e8e48-a45d-411c-81c2-0acf9149f9bd None admin] PATH_INFO: `/v3/roles` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:01:38.174104 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd1e8e48-a45d-411c-81c2-0acf9149f9bd None admin] RBAC: Adding query filter params (name=member) {{(pid=80378) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 02 10:01:38.174271 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd1e8e48-a45d-411c-81c2-0acf9149f9bd None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:01:38.176657 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd1e8e48-a45d-411c-81c2-0acf9149f9bd None admin] RBAC: Authorization granted {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:01:38.202758 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 153/309] 10.222.0.82 () {56 vars in 1101 bytes} [Tue Aug 2 10:01:37 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 301 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 02 10:01:38.202758 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd1e8e48-a45d-411c-81c2-0acf9149f9bd None admin] Authenticating user token {{(pid=80378) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:01:38.235352 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG dbcounter [-] [80378] Writing DB stats keystone:SELECT=100 {{(pid=80378) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 10:01:38.284928 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: [pid: 80377|app: 0|req: 157/310] 10.222.0.82 () {58 vars in 1023 bytes} [Tue Aug 2 10:01:38 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 221 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Aug 02 10:01:38.325301 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cba0f8b3-1296-40f4-9b8b-45fe7c354b02 None None] Authenticating user token {{(pid=80377) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:01:38.396601 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd1e8e48-a45d-411c-81c2-0acf9149f9bd None admin] Validating token access rules against request {{(pid=80378) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:01:38.455894 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-980239f5-ad09-4282-8f1e-7f0978825beb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '891005fa5ff74041b75747d16cc66798', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80378) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:01:38.456520 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-980239f5-ad09-4282-8f1e-7f0978825beb None admin] REQUEST_METHOD: `GET` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:01:38.456807 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-980239f5-ad09-4282-8f1e-7f0978825beb None admin] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:01:38.457084 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-980239f5-ad09-4282-8f1e-7f0978825beb None admin] PATH_INFO: `/v3/users/a5edd0d9ceec465d852028e78855bbe3` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:01:38.459617 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-980239f5-ad09-4282-8f1e-7f0978825beb None admin] RBAC: Authorizing `identity:get_user(user_id=a5edd0d9ceec465d852028e78855bbe3)` {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:01:38.461223 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-980239f5-ad09-4282-8f1e-7f0978825beb None admin] RBAC: Authorization granted {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:01:38.464745 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 154/311] 10.222.0.82 () {56 vars in 1177 bytes} [Tue Aug 2 10:01:38 2022] GET /identity/v3/users/a5edd0d9ceec465d852028e78855bbe3 => generated 293 bytes in 265 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 02 10:01:38.471185 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-980239f5-ad09-4282-8f1e-7f0978825beb None admin] Authenticating user token {{(pid=80378) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:01:38.504476 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG dbcounter [-] [80377] Writing DB stats keystone:SELECT=100 {{(pid=80377) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 10:01:38.530222 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cba0f8b3-1296-40f4-9b8b-45fe7c354b02 None None] Validating token access rules against request {{(pid=80377) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:01:38.610064 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8526a95-b3f1-4329-99e0-3473dd4b94d5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '891005fa5ff74041b75747d16cc66798', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80377) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:01:38.611134 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8526a95-b3f1-4329-99e0-3473dd4b94d5 None admin] REQUEST_METHOD: `GET` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:01:38.611420 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8526a95-b3f1-4329-99e0-3473dd4b94d5 None admin] SCRIPT_NAME: `/identity` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:01:38.611692 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8526a95-b3f1-4329-99e0-3473dd4b94d5 None admin] PATH_INFO: `/v3/roles/reader` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:01:38.638488 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8526a95-b3f1-4329-99e0-3473dd4b94d5 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 02 10:01:38.645221 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application [None req-c8526a95-b3f1-4329-99e0-3473dd4b94d5 None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Aug 02 10:01:38.645221 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 02 10:01:38.645221 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 02 10:01:38.645221 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 02 10:01:38.645221 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 02 10:01:38.645221 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 02 10:01:38.645221 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 02 10:01:38.645221 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 02 10:01:38.645221 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 02 10:01:38.645221 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 02 10:01:38.645221 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 02 10:01:38.645221 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 02 10:01:38.645221 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 02 10:01:38.645221 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 02 10:01:38.645221 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 02 10:01:38.645221 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 02 10:01:38.647003 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 02 10:01:38.647003 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 02 10:01:38.647003 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 02 10:01:38.647003 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 02 10:01:38.647003 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 02 10:01:38.647003 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 02 10:01:38.647003 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 02 10:01:38.647003 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 02 10:01:38.647003 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 02 10:01:38.647003 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application return self.get_or_create( Aug 02 10:01:38.647003 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 02 10:01:38.647003 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application with Lock( Aug 02 10:01:38.647003 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 02 10:01:38.647003 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application return self._enter() Aug 02 10:01:38.647003 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 02 10:01:38.647003 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 02 10:01:38.648594 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 02 10:01:38.648594 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application return self.creator() Aug 02 10:01:38.648594 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 02 10:01:38.648594 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application created_value = creator( Aug 02 10:01:38.648594 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 02 10:01:38.648594 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 02 10:01:38.648594 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 02 10:01:38.648594 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 02 10:01:38.648594 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 02 10:01:38.648594 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 02 10:01:38.648594 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Aug 02 10:01:38.648594 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application Aug 02 10:01:38.648594 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: [pid: 80377|app: 0|req: 158/312] 10.222.0.82 () {56 vars in 1099 bytes} [Tue Aug 2 10:01:38 2022] GET /identity/v3/roles/reader => generated 84 bytes in 323 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 02 10:01:38.654705 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG dbcounter [-] [80378] Writing DB stats keystone:SELECT=100 {{(pid=80378) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 10:01:38.658155 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8526a95-b3f1-4329-99e0-3473dd4b94d5 None admin] Authenticating user token {{(pid=80377) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:01:38.681401 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-980239f5-ad09-4282-8f1e-7f0978825beb None admin] Validating token access rules against request {{(pid=80378) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:01:38.738469 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd6a0c8e-01d9-4cee-a20b-320777e367c6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '891005fa5ff74041b75747d16cc66798', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80378) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:01:38.740020 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd6a0c8e-01d9-4cee-a20b-320777e367c6 None admin] REQUEST_METHOD: `PUT` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:01:38.740808 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd6a0c8e-01d9-4cee-a20b-320777e367c6 None admin] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:01:38.741535 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd6a0c8e-01d9-4cee-a20b-320777e367c6 None admin] PATH_INFO: `/v3/system/users/a5edd0d9ceec465d852028e78855bbe3/roles/e1c04b19fc4543b8b54f58bbcffbc598` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:01:38.745129 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd6a0c8e-01d9-4cee-a20b-320777e367c6 None admin] RBAC: Authorizing `identity:create_system_grant_for_user(user_id=a5edd0d9ceec465d852028e78855bbe3, role_id=e1c04b19fc4543b8b54f58bbcffbc598)` {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:01:38.748342 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd6a0c8e-01d9-4cee-a20b-320777e367c6 None admin] RBAC: Authorization granted {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:01:38.769897 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 155/313] 10.222.0.82 () {58 vars in 1334 bytes} [Tue Aug 2 10:01:38 2022] PUT /identity/v3/system/users/a5edd0d9ceec465d852028e78855bbe3/roles/e1c04b19fc4543b8b54f58bbcffbc598 => generated 0 bytes in 300 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 02 10:01:38.867828 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8526a95-b3f1-4329-99e0-3473dd4b94d5 None admin] Validating token access rules against request {{(pid=80377) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:01:38.927446 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd82cb46-deda-4a9f-8685-598c3daec752 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '891005fa5ff74041b75747d16cc66798', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80377) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:01:38.928497 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd82cb46-deda-4a9f-8685-598c3daec752 None admin] REQUEST_METHOD: `GET` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:01:38.928996 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd82cb46-deda-4a9f-8685-598c3daec752 None admin] SCRIPT_NAME: `/identity` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:01:38.929479 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd82cb46-deda-4a9f-8685-598c3daec752 None admin] PATH_INFO: `/v3/roles` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:01:38.930873 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd82cb46-deda-4a9f-8685-598c3daec752 None admin] RBAC: Adding query filter params (name=reader) {{(pid=80377) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 02 10:01:38.931430 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd82cb46-deda-4a9f-8685-598c3daec752 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:01:38.934134 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd82cb46-deda-4a9f-8685-598c3daec752 None admin] RBAC: Authorization granted {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:01:38.952413 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: [pid: 80377|app: 0|req: 159/314] 10.222.0.82 () {56 vars in 1101 bytes} [Tue Aug 2 10:01:38 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 296 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 02 10:01:38.961892 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd6a0c8e-01d9-4cee-a20b-320777e367c6 None admin] Authenticating user token {{(pid=80378) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:01:39.185606 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd6a0c8e-01d9-4cee-a20b-320777e367c6 None admin] Validating token access rules against request {{(pid=80378) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:01:39.243161 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1f4c6b2-a6c2-4110-ad5c-415c1862242a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '891005fa5ff74041b75747d16cc66798', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80378) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:01:39.244075 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1f4c6b2-a6c2-4110-ad5c-415c1862242a None admin] REQUEST_METHOD: `GET` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:01:39.244363 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1f4c6b2-a6c2-4110-ad5c-415c1862242a None admin] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:01:39.244640 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1f4c6b2-a6c2-4110-ad5c-415c1862242a None admin] PATH_INFO: `/v3/users/b5222f0dc11f4c6998b83c676ee8e6b8` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:01:39.246628 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1f4c6b2-a6c2-4110-ad5c-415c1862242a None admin] RBAC: Authorizing `identity:get_user(user_id=b5222f0dc11f4c6998b83c676ee8e6b8)` {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:01:39.249431 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1f4c6b2-a6c2-4110-ad5c-415c1862242a None admin] RBAC: Authorization granted {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:01:39.252729 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 156/315] 10.222.0.82 () {56 vars in 1177 bytes} [Tue Aug 2 10:01:38 2022] GET /identity/v3/users/b5222f0dc11f4c6998b83c676ee8e6b8 => generated 297 bytes in 292 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 02 10:01:39.259696 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd82cb46-deda-4a9f-8685-598c3daec752 None admin] Authenticating user token {{(pid=80377) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:01:39.282188 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG dbcounter [-] [80377] Writing DB stats keystone:SELECT=100 {{(pid=80377) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 10:01:39.428730 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd82cb46-deda-4a9f-8685-598c3daec752 None admin] Validating token access rules against request {{(pid=80377) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:01:39.492860 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d1591ff-c1a2-4631-9770-9e6348a61ed7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '891005fa5ff74041b75747d16cc66798', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80377) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:01:39.493557 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d1591ff-c1a2-4631-9770-9e6348a61ed7 None admin] REQUEST_METHOD: `GET` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:01:39.493868 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d1591ff-c1a2-4631-9770-9e6348a61ed7 None admin] SCRIPT_NAME: `/identity` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:01:39.494150 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d1591ff-c1a2-4631-9770-9e6348a61ed7 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:01:39.585017 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d1591ff-c1a2-4631-9770-9e6348a61ed7 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:01:39.586929 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d1591ff-c1a2-4631-9770-9e6348a61ed7 None admin] RBAC: Authorization granted {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:01:39.605185 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG dbcounter [-] [80377] Writing DB stats keystone:SELECT=100 {{(pid=80377) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 10:01:39.710532 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: [pid: 80377|app: 0|req: 160/316] 10.222.0.82 () {58 vars in 1282 bytes} [Tue Aug 2 10:01:39 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 452 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 02 10:01:39.716857 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1f4c6b2-a6c2-4110-ad5c-415c1862242a None admin] Authenticating user token {{(pid=80378) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:01:39.746493 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG dbcounter [-] [80378] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80378) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 10:01:39.879842 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1f4c6b2-a6c2-4110-ad5c-415c1862242a None admin] Validating token access rules against request {{(pid=80378) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:01:39.945255 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-184f7629-6df3-4bbe-8a02-2b09b5d48671 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '891005fa5ff74041b75747d16cc66798', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80378) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:01:39.946377 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-184f7629-6df3-4bbe-8a02-2b09b5d48671 None admin] REQUEST_METHOD: `GET` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:01:39.946877 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-184f7629-6df3-4bbe-8a02-2b09b5d48671 None admin] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:01:39.947370 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-184f7629-6df3-4bbe-8a02-2b09b5d48671 None admin] PATH_INFO: `/v3/projects/f7db7cf528494fbb8836c08d79b752ef` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:01:39.949514 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-184f7629-6df3-4bbe-8a02-2b09b5d48671 None admin] RBAC: Authorizing `identity:get_project(project_id=f7db7cf528494fbb8836c08d79b752ef)` {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:01:39.952330 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-184f7629-6df3-4bbe-8a02-2b09b5d48671 None admin] RBAC: Authorization granted {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:01:39.956609 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 157/317] 10.222.0.82 () {56 vars in 1186 bytes} [Tue Aug 2 10:01:39 2022] GET /identity/v3/projects/f7db7cf528494fbb8836c08d79b752ef => generated 302 bytes in 240 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 02 10:01:39.963073 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d1591ff-c1a2-4631-9770-9e6348a61ed7 None admin] Authenticating user token {{(pid=80377) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:01:40.145757 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d1591ff-c1a2-4631-9770-9e6348a61ed7 None admin] Validating token access rules against request {{(pid=80377) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:01:40.174630 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG dbcounter [-] [80377] Writing DB stats keystone:SELECT=100 {{(pid=80377) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 10:01:40.199335 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fef6d6c3-d7e9-4b87-a2a8-300ec0a63be2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '891005fa5ff74041b75747d16cc66798', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80377) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:01:40.199961 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fef6d6c3-d7e9-4b87-a2a8-300ec0a63be2 None admin] REQUEST_METHOD: `GET` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:01:40.200244 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fef6d6c3-d7e9-4b87-a2a8-300ec0a63be2 None admin] SCRIPT_NAME: `/identity` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:01:40.200519 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fef6d6c3-d7e9-4b87-a2a8-300ec0a63be2 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:01:40.201318 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fef6d6c3-d7e9-4b87-a2a8-300ec0a63be2 None admin] RBAC: Adding query filter params (role.id=444e768aa5fb44bbab6c20afad6a0218, scope.project.id=f7db7cf528494fbb8836c08d79b752ef, user.id=b5222f0dc11f4c6998b83c676ee8e6b8) {{(pid=80377) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 02 10:01:40.201628 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fef6d6c3-d7e9-4b87-a2a8-300ec0a63be2 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:01:40.203110 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fef6d6c3-d7e9-4b87-a2a8-300ec0a63be2 None admin] RBAC: Authorization granted {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:01:40.210481 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: [pid: 80377|app: 0|req: 161/318] 10.222.0.82 () {56 vars in 1374 bytes} [Tue Aug 2 10:01:39 2022] GET /identity/v3/role_assignments?user.id=b5222f0dc11f4c6998b83c676ee8e6b8&scope.project.id=f7db7cf528494fbb8836c08d79b752ef&role.id=444e768aa5fb44bbab6c20afad6a0218 => generated 608 bytes in 248 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 02 10:01:40.571498 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0bdb273-eac1-433f-b74b-8d56311d27b0 None None] REQUEST_METHOD: `GET` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:01:40.572212 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0bdb273-eac1-433f-b74b-8d56311d27b0 None None] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:01:40.572581 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0bdb273-eac1-433f-b74b-8d56311d27b0 None None] PATH_INFO: `/` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:01:40.575164 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 158/319] 10.222.0.82 () {54 vars in 926 bytes} [Tue Aug 2 10:01:40 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 02 10:01:40.583204 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcb316d8-6df4-4bad-9730-ffef99b48cf2 None None] REQUEST_METHOD: `POST` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:01:40.583572 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcb316d8-6df4-4bad-9730-ffef99b48cf2 None None] SCRIPT_NAME: `/identity` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:01:40.583872 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcb316d8-6df4-4bad-9730-ffef99b48cf2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:01:40.606772 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.auth.core [None req-dcb316d8-6df4-4bad-9730-ffef99b48cf2 None None] MFA Rules not processed for user `891005fa5ff74041b75747d16cc66798`. Rule list: `[]` (Enabled: `True`). {{(pid=80377) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 02 10:01:40.607990 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.fernet_utils [None req-dcb316d8-6df4-4bad-9730-ffef99b48cf2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80377) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 10:01:40.723346 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: [pid: 80377|app: 0|req: 162/320] 10.222.0.82 () {58 vars in 1023 bytes} [Tue Aug 2 10:01:40 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 142 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Aug 02 10:01:40.766621 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0bdb273-eac1-433f-b74b-8d56311d27b0 None None] Authenticating user token {{(pid=80378) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:01:40.775673 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef072fc1-0145-4067-9e6f-3018d1577dcd None None] REQUEST_METHOD: `GET` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:01:40.776914 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef072fc1-0145-4067-9e6f-3018d1577dcd None None] SCRIPT_NAME: `/identity` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:01:40.777217 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef072fc1-0145-4067-9e6f-3018d1577dcd None None] PATH_INFO: `/` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:01:40.778773 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: [pid: 80377|app: 0|req: 163/321] 10.222.0.82 () {54 vars in 926 bytes} [Tue Aug 2 10:01:40 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 02 10:01:40.784723 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8eab0ec-9aa8-477c-8a0f-7d762a6617b6 None None] REQUEST_METHOD: `POST` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:01:40.785046 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8eab0ec-9aa8-477c-8a0f-7d762a6617b6 None None] SCRIPT_NAME: `/identity` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:01:40.785286 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8eab0ec-9aa8-477c-8a0f-7d762a6617b6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:01:40.815485 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.auth.core [None req-e8eab0ec-9aa8-477c-8a0f-7d762a6617b6 None None] MFA Rules not processed for user `891005fa5ff74041b75747d16cc66798`. Rule list: `[]` (Enabled: `True`). {{(pid=80377) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 02 10:01:40.817233 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.fernet_utils [None req-e8eab0ec-9aa8-477c-8a0f-7d762a6617b6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80377) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 10:01:40.949430 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG dbcounter [-] [80378] Writing DB stats keystone:SELECT=100 {{(pid=80378) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 10:01:40.957854 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: [pid: 80377|app: 0|req: 164/322] 10.222.0.82 () {58 vars in 1023 bytes} [Tue Aug 2 10:01:40 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 171 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Aug 02 10:01:40.975369 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0bdb273-eac1-433f-b74b-8d56311d27b0 None None] Validating token access rules against request {{(pid=80378) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:01:40.986830 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8eab0ec-9aa8-477c-8a0f-7d762a6617b6 None None] Authenticating user token {{(pid=80377) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:01:41.049441 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-965c4279-0c84-4304-bb41-2cb625d4a6bf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '891005fa5ff74041b75747d16cc66798', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80378) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:01:41.050580 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-965c4279-0c84-4304-bb41-2cb625d4a6bf None admin] REQUEST_METHOD: `GET` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:01:41.051060 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-965c4279-0c84-4304-bb41-2cb625d4a6bf None admin] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:01:41.051537 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-965c4279-0c84-4304-bb41-2cb625d4a6bf None admin] PATH_INFO: `/v3/roles/reader` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:01:41.055486 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG dbcounter [-] [80377] Writing DB stats keystone:SELECT=100 {{(pid=80377) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 10:01:41.075224 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-965c4279-0c84-4304-bb41-2cb625d4a6bf None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 02 10:01:41.085764 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application [None req-965c4279-0c84-4304-bb41-2cb625d4a6bf None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Aug 02 10:01:41.085764 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 02 10:01:41.085764 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 02 10:01:41.085764 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 02 10:01:41.085764 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 02 10:01:41.085764 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 02 10:01:41.085764 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 02 10:01:41.085764 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 02 10:01:41.085764 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 02 10:01:41.085764 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 02 10:01:41.085764 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 02 10:01:41.085764 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 02 10:01:41.085764 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 02 10:01:41.085764 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 02 10:01:41.085764 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 02 10:01:41.085764 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 02 10:01:41.087137 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 02 10:01:41.087137 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 02 10:01:41.087137 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 02 10:01:41.087137 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 02 10:01:41.087137 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 02 10:01:41.087137 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 02 10:01:41.087137 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 02 10:01:41.087137 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 02 10:01:41.087137 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 02 10:01:41.087137 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application return self.get_or_create( Aug 02 10:01:41.087137 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 02 10:01:41.087137 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application with Lock( Aug 02 10:01:41.087137 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 02 10:01:41.087137 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application return self._enter() Aug 02 10:01:41.087137 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 02 10:01:41.087137 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 02 10:01:41.088669 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 02 10:01:41.088669 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application return self.creator() Aug 02 10:01:41.088669 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 02 10:01:41.088669 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application created_value = creator( Aug 02 10:01:41.088669 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 02 10:01:41.088669 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 02 10:01:41.088669 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 02 10:01:41.088669 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 02 10:01:41.088669 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 02 10:01:41.088669 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 02 10:01:41.088669 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Aug 02 10:01:41.088669 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application Aug 02 10:01:41.089642 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 159/323] 10.222.0.82 () {56 vars in 1099 bytes} [Tue Aug 2 10:01:40 2022] GET /identity/v3/roles/reader => generated 84 bytes in 324 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 02 10:01:41.098813 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-965c4279-0c84-4304-bb41-2cb625d4a6bf None admin] Authenticating user token {{(pid=80378) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:01:41.222188 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8eab0ec-9aa8-477c-8a0f-7d762a6617b6 None None] Validating token access rules against request {{(pid=80377) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:01:41.289353 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-965c4279-0c84-4304-bb41-2cb625d4a6bf None admin] Validating token access rules against request {{(pid=80378) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:01:41.290434 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74589e4d-5992-4a25-964f-6f40add98822 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '891005fa5ff74041b75747d16cc66798', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80377) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:01:41.291529 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74589e4d-5992-4a25-964f-6f40add98822 None admin] REQUEST_METHOD: `GET` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:01:41.292027 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74589e4d-5992-4a25-964f-6f40add98822 None admin] SCRIPT_NAME: `/identity` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:01:41.292590 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74589e4d-5992-4a25-964f-6f40add98822 None admin] PATH_INFO: `/v3/domains/default` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:01:41.294662 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74589e4d-5992-4a25-964f-6f40add98822 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:01:41.296357 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74589e4d-5992-4a25-964f-6f40add98822 None admin] RBAC: Authorization granted {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:01:41.308577 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: [pid: 80377|app: 0|req: 165/324] 10.222.0.82 () {56 vars in 1108 bytes} [Tue Aug 2 10:01:40 2022] GET /identity/v3/domains/default => generated 199 bytes in 314 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 02 10:01:41.310334 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74589e4d-5992-4a25-964f-6f40add98822 None admin] Authenticating user token {{(pid=80377) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:01:41.354182 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3a6792f-fb8b-4a5b-bd75-ca761524d0c5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '891005fa5ff74041b75747d16cc66798', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80378) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:01:41.355237 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3a6792f-fb8b-4a5b-bd75-ca761524d0c5 None admin] REQUEST_METHOD: `GET` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:01:41.355754 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3a6792f-fb8b-4a5b-bd75-ca761524d0c5 None admin] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:01:41.356351 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3a6792f-fb8b-4a5b-bd75-ca761524d0c5 None admin] PATH_INFO: `/v3/roles` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:01:41.357648 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3a6792f-fb8b-4a5b-bd75-ca761524d0c5 None admin] RBAC: Adding query filter params (name=reader) {{(pid=80378) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 02 10:01:41.358245 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3a6792f-fb8b-4a5b-bd75-ca761524d0c5 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:01:41.361515 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3a6792f-fb8b-4a5b-bd75-ca761524d0c5 None admin] RBAC: Authorization granted {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:01:41.377030 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 160/325] 10.222.0.82 () {56 vars in 1101 bytes} [Tue Aug 2 10:01:41 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 280 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 02 10:01:41.384791 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3a6792f-fb8b-4a5b-bd75-ca761524d0c5 None admin] Authenticating user token {{(pid=80378) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:01:41.400303 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG dbcounter [-] [80378] Writing DB stats keystone:SELECT=100 {{(pid=80378) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 10:01:41.479402 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74589e4d-5992-4a25-964f-6f40add98822 None admin] Validating token access rules against request {{(pid=80377) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:01:41.483389 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG dbcounter [-] [80377] Writing DB stats keystone:SELECT=100 {{(pid=80377) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 10:01:41.534716 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3a6792f-fb8b-4a5b-bd75-ca761524d0c5 None admin] Validating token access rules against request {{(pid=80378) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:01:41.544849 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed97697c-b61f-4030-949b-da3d8b400d1a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '891005fa5ff74041b75747d16cc66798', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80377) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:01:41.545972 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed97697c-b61f-4030-949b-da3d8b400d1a None admin] REQUEST_METHOD: `POST` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:01:41.546480 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed97697c-b61f-4030-949b-da3d8b400d1a None admin] SCRIPT_NAME: `/identity` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:01:41.546986 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed97697c-b61f-4030-949b-da3d8b400d1a None admin] PATH_INFO: `/v3/groups` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:01:41.548814 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed97697c-b61f-4030-949b-da3d8b400d1a None admin] RBAC: Authorizing `identity:create_group()` {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:01:41.551592 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed97697c-b61f-4030-949b-da3d8b400d1a None admin] RBAC: Authorization granted {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:01:41.583624 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: [pid: 80377|app: 0|req: 166/326] 10.222.0.82 () {60 vars in 1134 bytes} [Tue Aug 2 10:01:41 2022] POST /identity/v3/groups => generated 226 bytes in 267 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 02 10:01:41.595907 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60add194-d572-4f60-b6c4-18a2e07bea30 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '891005fa5ff74041b75747d16cc66798', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80378) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:01:41.597061 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60add194-d572-4f60-b6c4-18a2e07bea30 None admin] REQUEST_METHOD: `GET` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:01:41.597582 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60add194-d572-4f60-b6c4-18a2e07bea30 None admin] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:01:41.598131 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60add194-d572-4f60-b6c4-18a2e07bea30 None admin] PATH_INFO: `/v3/users/8a9b9d020f064f5ab9f664ee649e8389` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:01:41.600459 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60add194-d572-4f60-b6c4-18a2e07bea30 None admin] RBAC: Authorizing `identity:get_user(user_id=8a9b9d020f064f5ab9f664ee649e8389)` {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:01:41.603427 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60add194-d572-4f60-b6c4-18a2e07bea30 None admin] RBAC: Authorization granted {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:01:41.606881 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 161/327] 10.222.0.82 () {56 vars in 1177 bytes} [Tue Aug 2 10:01:41 2022] GET /identity/v3/users/8a9b9d020f064f5ab9f664ee649e8389 => generated 293 bytes in 223 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 02 10:01:41.616361 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed97697c-b61f-4030-949b-da3d8b400d1a None admin] Authenticating user token {{(pid=80377) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:01:41.810969 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c3da066-b10f-4580-998a-51c23ada393e None None] REQUEST_METHOD: `GET` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:01:41.811494 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c3da066-b10f-4580-998a-51c23ada393e None None] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:01:41.811827 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c3da066-b10f-4580-998a-51c23ada393e None None] PATH_INFO: `/` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:01:41.814417 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 162/328] 10.222.0.82 () {54 vars in 926 bytes} [Tue Aug 2 10:01:41 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 02 10:01:41.818055 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed97697c-b61f-4030-949b-da3d8b400d1a None admin] Validating token access rules against request {{(pid=80377) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:01:41.823202 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c5a05d3-ad35-4251-8911-a49e9890b620 None None] REQUEST_METHOD: `POST` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:01:41.823686 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c5a05d3-ad35-4251-8911-a49e9890b620 None None] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:01:41.824168 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c5a05d3-ad35-4251-8911-a49e9890b620 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:01:41.848232 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.auth.core [None req-1c5a05d3-ad35-4251-8911-a49e9890b620 None None] MFA Rules not processed for user `891005fa5ff74041b75747d16cc66798`. Rule list: `[]` (Enabled: `True`). {{(pid=80378) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 02 10:01:41.849761 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.fernet_utils [None req-1c5a05d3-ad35-4251-8911-a49e9890b620 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80378) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 10:01:41.882503 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-909ef118-e871-4eef-be2d-464814e17ae9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '891005fa5ff74041b75747d16cc66798', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80377) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:01:41.883558 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-909ef118-e871-4eef-be2d-464814e17ae9 None admin] REQUEST_METHOD: `PUT` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:01:41.884009 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-909ef118-e871-4eef-be2d-464814e17ae9 None admin] SCRIPT_NAME: `/identity` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:01:41.884460 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-909ef118-e871-4eef-be2d-464814e17ae9 None admin] PATH_INFO: `/v3/system/users/8a9b9d020f064f5ab9f664ee649e8389/roles/444e768aa5fb44bbab6c20afad6a0218` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:01:41.887945 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-909ef118-e871-4eef-be2d-464814e17ae9 None admin] RBAC: Authorizing `identity:create_system_grant_for_user(user_id=8a9b9d020f064f5ab9f664ee649e8389, role_id=444e768aa5fb44bbab6c20afad6a0218)` {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:01:41.889334 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-909ef118-e871-4eef-be2d-464814e17ae9 None admin] RBAC: Authorization granted {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:01:41.913410 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: [pid: 80377|app: 0|req: 167/329] 10.222.0.82 () {58 vars in 1334 bytes} [Tue Aug 2 10:01:41 2022] PUT /identity/v3/system/users/8a9b9d020f064f5ab9f664ee649e8389/roles/444e768aa5fb44bbab6c20afad6a0218 => generated 0 bytes in 298 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 02 10:01:41.981088 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 163/330] 10.222.0.82 () {58 vars in 1023 bytes} [Tue Aug 2 10:01:41 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 159 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Aug 02 10:01:42.021624 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-909ef118-e871-4eef-be2d-464814e17ae9 None admin] Authenticating user token {{(pid=80377) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:01:42.069952 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG dbcounter [-] [80377] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80377) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 10:01:42.197694 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-909ef118-e871-4eef-be2d-464814e17ae9 None admin] Validating token access rules against request {{(pid=80377) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:01:42.263157 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-281c33b6-4548-412a-8660-659b140cda82 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '891005fa5ff74041b75747d16cc66798', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80377) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:01:42.264018 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-281c33b6-4548-412a-8660-659b140cda82 None admin] REQUEST_METHOD: `GET` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:01:42.264362 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-281c33b6-4548-412a-8660-659b140cda82 None admin] SCRIPT_NAME: `/identity` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:01:42.264704 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-281c33b6-4548-412a-8660-659b140cda82 None admin] PATH_INFO: `/v3/roles/member` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:01:42.278057 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-281c33b6-4548-412a-8660-659b140cda82 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 02 10:01:42.285169 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application [None req-281c33b6-4548-412a-8660-659b140cda82 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Aug 02 10:01:42.285169 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 02 10:01:42.285169 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 02 10:01:42.285169 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 02 10:01:42.285169 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 02 10:01:42.285169 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 02 10:01:42.285169 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 02 10:01:42.285169 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 02 10:01:42.285169 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 02 10:01:42.285169 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 02 10:01:42.285169 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 02 10:01:42.285169 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 02 10:01:42.285169 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 02 10:01:42.285169 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 02 10:01:42.285169 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 02 10:01:42.285169 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 02 10:01:42.286633 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 02 10:01:42.286633 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 02 10:01:42.286633 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 02 10:01:42.286633 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 02 10:01:42.286633 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 02 10:01:42.286633 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 02 10:01:42.286633 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 02 10:01:42.286633 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 02 10:01:42.286633 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 02 10:01:42.286633 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application return self.get_or_create( Aug 02 10:01:42.286633 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 02 10:01:42.286633 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application with Lock( Aug 02 10:01:42.286633 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 02 10:01:42.286633 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application return self._enter() Aug 02 10:01:42.286633 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 02 10:01:42.286633 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 02 10:01:42.288169 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 02 10:01:42.288169 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application return self.creator() Aug 02 10:01:42.288169 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 02 10:01:42.288169 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application created_value = creator( Aug 02 10:01:42.288169 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 02 10:01:42.288169 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 02 10:01:42.288169 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 02 10:01:42.288169 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 02 10:01:42.288169 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 02 10:01:42.288169 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 02 10:01:42.288169 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Aug 02 10:01:42.288169 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application Aug 02 10:01:42.288169 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: [pid: 80377|app: 0|req: 168/331] 10.222.0.82 () {56 vars in 1099 bytes} [Tue Aug 2 10:01:42 2022] GET /identity/v3/roles/member => generated 84 bytes in 267 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 02 10:01:42.296053 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c5a05d3-ad35-4251-8911-a49e9890b620 None None] Authenticating user token {{(pid=80378) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:01:42.307089 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG dbcounter [-] [80378] Writing DB stats keystone:SELECT=100 {{(pid=80378) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 10:01:42.453188 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c5a05d3-ad35-4251-8911-a49e9890b620 None None] Validating token access rules against request {{(pid=80378) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:01:42.499947 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74bf555f-183f-4f42-9710-ef7c7e38e2d5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '891005fa5ff74041b75747d16cc66798', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80378) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:01:42.500916 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74bf555f-183f-4f42-9710-ef7c7e38e2d5 None admin] REQUEST_METHOD: `GET` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:01:42.501733 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74bf555f-183f-4f42-9710-ef7c7e38e2d5 None admin] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:01:42.502063 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74bf555f-183f-4f42-9710-ef7c7e38e2d5 None admin] PATH_INFO: `/v3/roles` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:01:42.502785 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74bf555f-183f-4f42-9710-ef7c7e38e2d5 None admin] RBAC: Adding query filter params (name=member) {{(pid=80378) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 02 10:01:42.503092 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74bf555f-183f-4f42-9710-ef7c7e38e2d5 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:01:42.504836 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74bf555f-183f-4f42-9710-ef7c7e38e2d5 None admin] RBAC: Authorization granted {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:01:42.520802 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 164/332] 10.222.0.82 () {56 vars in 1101 bytes} [Tue Aug 2 10:01:42 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 226 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 02 10:01:42.526800 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-281c33b6-4548-412a-8660-659b140cda82 None admin] Authenticating user token {{(pid=80377) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:01:42.694089 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG dbcounter [-] [80377] Writing DB stats keystone:SELECT=100 {{(pid=80377) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 10:01:42.721658 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-281c33b6-4548-412a-8660-659b140cda82 None admin] Validating token access rules against request {{(pid=80377) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:01:42.788442 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb3c587d-b843-40ed-b5b4-56cb27c911d8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '891005fa5ff74041b75747d16cc66798', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80377) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:01:42.789400 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb3c587d-b843-40ed-b5b4-56cb27c911d8 None admin] REQUEST_METHOD: `GET` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:01:42.789843 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb3c587d-b843-40ed-b5b4-56cb27c911d8 None admin] SCRIPT_NAME: `/identity` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:01:42.790250 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb3c587d-b843-40ed-b5b4-56cb27c911d8 None admin] PATH_INFO: `/v3/users/a5edd0d9ceec465d852028e78855bbe3` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:01:42.792087 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb3c587d-b843-40ed-b5b4-56cb27c911d8 None admin] RBAC: Authorizing `identity:get_user(user_id=a5edd0d9ceec465d852028e78855bbe3)` {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:01:42.794372 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb3c587d-b843-40ed-b5b4-56cb27c911d8 None admin] RBAC: Authorization granted {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:01:42.801053 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: [pid: 80377|app: 0|req: 169/333] 10.222.0.82 () {56 vars in 1177 bytes} [Tue Aug 2 10:01:42 2022] GET /identity/v3/users/a5edd0d9ceec465d852028e78855bbe3 => generated 293 bytes in 272 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 02 10:01:42.806512 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74bf555f-183f-4f42-9710-ef7c7e38e2d5 None admin] Authenticating user token {{(pid=80378) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:01:42.948593 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG dbcounter [-] [80378] Writing DB stats keystone:SELECT=100 {{(pid=80378) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 10:01:43.002955 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74bf555f-183f-4f42-9710-ef7c7e38e2d5 None admin] Validating token access rules against request {{(pid=80378) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:01:43.057902 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26e36c1e-9dde-4ffc-8b81-e73a5a1cff39 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '891005fa5ff74041b75747d16cc66798', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80378) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:01:43.058873 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26e36c1e-9dde-4ffc-8b81-e73a5a1cff39 None admin] REQUEST_METHOD: `GET` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:01:43.059159 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26e36c1e-9dde-4ffc-8b81-e73a5a1cff39 None admin] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:01:43.059536 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26e36c1e-9dde-4ffc-8b81-e73a5a1cff39 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:01:43.060433 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26e36c1e-9dde-4ffc-8b81-e73a5a1cff39 None admin] RBAC: Adding query filter params (role.id=e1c04b19fc4543b8b54f58bbcffbc598, user.id=a5edd0d9ceec465d852028e78855bbe3, scope.system=all) {{(pid=80378) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 02 10:01:43.060748 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26e36c1e-9dde-4ffc-8b81-e73a5a1cff39 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:01:43.062296 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26e36c1e-9dde-4ffc-8b81-e73a5a1cff39 None admin] RBAC: Authorization granted {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:01:43.083679 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 165/334] 10.222.0.82 () {56 vars in 1308 bytes} [Tue Aug 2 10:01:42 2022] GET /identity/v3/role_assignments?user.id=a5edd0d9ceec465d852028e78855bbe3&scope.system=all&role.id=e1c04b19fc4543b8b54f58bbcffbc598 => generated 510 bytes in 278 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 02 10:01:45.069093 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d22cdd71-72ba-454f-8110-6e1a3f42e44e None None] REQUEST_METHOD: `GET` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:01:45.069093 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d22cdd71-72ba-454f-8110-6e1a3f42e44e None None] SCRIPT_NAME: `/identity` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:01:45.070639 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d22cdd71-72ba-454f-8110-6e1a3f42e44e None None] PATH_INFO: `/` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:01:45.072573 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: [pid: 80377|app: 0|req: 170/335] 10.222.0.82 () {54 vars in 926 bytes} [Tue Aug 2 10:01:45 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 02 10:01:45.074936 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72421177-a6d2-4df4-aa7b-a22175e26a1f None None] REQUEST_METHOD: `GET` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:01:45.076032 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72421177-a6d2-4df4-aa7b-a22175e26a1f None None] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:01:45.081282 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72421177-a6d2-4df4-aa7b-a22175e26a1f None None] PATH_INFO: `/` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:01:45.083799 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 166/336] 10.222.0.82 () {54 vars in 926 bytes} [Tue Aug 2 10:01:45 2022] GET /identity => generated 270 bytes in 10 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 02 10:01:45.087190 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66dbc843-c7a8-4fad-b1a5-2f796a3b01ac None None] REQUEST_METHOD: `POST` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:01:45.087862 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66dbc843-c7a8-4fad-b1a5-2f796a3b01ac None None] SCRIPT_NAME: `/identity` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:01:45.088499 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66dbc843-c7a8-4fad-b1a5-2f796a3b01ac None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:01:45.093563 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a2533ce-d886-46c7-9a77-5eb7b9210db1 None None] REQUEST_METHOD: `POST` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:01:45.094123 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a2533ce-d886-46c7-9a77-5eb7b9210db1 None None] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:01:45.095183 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a2533ce-d886-46c7-9a77-5eb7b9210db1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:01:45.116620 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.auth.core [None req-66dbc843-c7a8-4fad-b1a5-2f796a3b01ac None None] MFA Rules not processed for user `891005fa5ff74041b75747d16cc66798`. Rule list: `[]` (Enabled: `True`). {{(pid=80377) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 02 10:01:45.118159 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.fernet_utils [None req-66dbc843-c7a8-4fad-b1a5-2f796a3b01ac None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80377) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 10:01:45.130715 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.auth.core [None req-7a2533ce-d886-46c7-9a77-5eb7b9210db1 None None] MFA Rules not processed for user `891005fa5ff74041b75747d16cc66798`. Rule list: `[]` (Enabled: `True`). {{(pid=80378) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 02 10:01:45.132933 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.fernet_utils [None req-7a2533ce-d886-46c7-9a77-5eb7b9210db1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80378) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 10:01:45.257490 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: [pid: 80377|app: 0|req: 171/337] 10.222.0.82 () {58 vars in 1023 bytes} [Tue Aug 2 10:01:45 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 172 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 02 10:01:45.287040 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 167/338] 10.222.0.82 () {58 vars in 1023 bytes} [Tue Aug 2 10:01:45 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 194 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 02 10:01:45.327275 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66dbc843-c7a8-4fad-b1a5-2f796a3b01ac None None] Authenticating user token {{(pid=80377) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:01:45.344444 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a2533ce-d886-46c7-9a77-5eb7b9210db1 None None] Authenticating user token {{(pid=80378) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:01:45.539367 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG dbcounter [-] [80378] Writing DB stats keystone:SELECT=100 {{(pid=80378) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 10:01:45.549555 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG dbcounter [-] [80377] Writing DB stats keystone:SELECT=100 {{(pid=80377) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 10:01:45.573942 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66dbc843-c7a8-4fad-b1a5-2f796a3b01ac None None] Validating token access rules against request {{(pid=80377) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:01:45.635100 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9ed7181-c35e-4519-8a15-c23d60fd2ce2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '891005fa5ff74041b75747d16cc66798', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80377) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:01:45.636129 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9ed7181-c35e-4519-8a15-c23d60fd2ce2 None admin] REQUEST_METHOD: `GET` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:01:45.636608 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9ed7181-c35e-4519-8a15-c23d60fd2ce2 None admin] SCRIPT_NAME: `/identity` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:01:45.637075 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9ed7181-c35e-4519-8a15-c23d60fd2ce2 None admin] PATH_INFO: `/v3/roles/member` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:01:45.660106 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9ed7181-c35e-4519-8a15-c23d60fd2ce2 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 02 10:01:45.670228 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application [None req-d9ed7181-c35e-4519-8a15-c23d60fd2ce2 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Aug 02 10:01:45.670228 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 02 10:01:45.670228 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 02 10:01:45.670228 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 02 10:01:45.670228 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 02 10:01:45.670228 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 02 10:01:45.670228 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 02 10:01:45.670228 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 02 10:01:45.670228 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 02 10:01:45.670228 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 02 10:01:45.670228 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 02 10:01:45.670228 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 02 10:01:45.670228 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 02 10:01:45.670228 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 02 10:01:45.670228 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 02 10:01:45.670228 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 02 10:01:45.671175 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 02 10:01:45.671175 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 02 10:01:45.671175 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 02 10:01:45.671175 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 02 10:01:45.671175 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 02 10:01:45.671175 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 02 10:01:45.671175 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 02 10:01:45.671175 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 02 10:01:45.671175 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 02 10:01:45.671175 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application return self.get_or_create( Aug 02 10:01:45.671175 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 02 10:01:45.671175 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application with Lock( Aug 02 10:01:45.671175 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 02 10:01:45.671175 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application return self._enter() Aug 02 10:01:45.671175 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 02 10:01:45.671175 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 02 10:01:45.672337 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 02 10:01:45.672337 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application return self.creator() Aug 02 10:01:45.672337 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 02 10:01:45.672337 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application created_value = creator( Aug 02 10:01:45.672337 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 02 10:01:45.672337 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 02 10:01:45.672337 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 02 10:01:45.672337 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 02 10:01:45.672337 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 02 10:01:45.672337 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 02 10:01:45.672337 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Aug 02 10:01:45.672337 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application Aug 02 10:01:45.675553 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: [pid: 80377|app: 0|req: 172/339] 10.222.0.82 () {56 vars in 1099 bytes} [Tue Aug 2 10:01:45 2022] GET /identity/v3/roles/member => generated 84 bytes in 349 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 02 10:01:45.682897 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9ed7181-c35e-4519-8a15-c23d60fd2ce2 None admin] Authenticating user token {{(pid=80377) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:01:45.725833 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a2533ce-d886-46c7-9a77-5eb7b9210db1 None None] Validating token access rules against request {{(pid=80378) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:01:45.819665 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00692e8f-96b3-4864-b495-53cfde1d295b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '891005fa5ff74041b75747d16cc66798', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80378) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:01:45.820622 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00692e8f-96b3-4864-b495-53cfde1d295b None admin] REQUEST_METHOD: `GET` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:01:45.821165 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00692e8f-96b3-4864-b495-53cfde1d295b None admin] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:01:45.821664 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00692e8f-96b3-4864-b495-53cfde1d295b None admin] PATH_INFO: `/v3/roles/reader` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:01:45.863458 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00692e8f-96b3-4864-b495-53cfde1d295b None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 02 10:01:45.872910 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9ed7181-c35e-4519-8a15-c23d60fd2ce2 None admin] Validating token access rules against request {{(pid=80377) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:01:45.873631 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application [None req-00692e8f-96b3-4864-b495-53cfde1d295b None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Aug 02 10:01:45.873631 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 02 10:01:45.873631 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 02 10:01:45.873631 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 02 10:01:45.873631 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 02 10:01:45.873631 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 02 10:01:45.873631 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 02 10:01:45.873631 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 02 10:01:45.873631 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 02 10:01:45.873631 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 02 10:01:45.873631 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 02 10:01:45.873631 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 02 10:01:45.873631 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 02 10:01:45.873631 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 02 10:01:45.873631 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 02 10:01:45.873631 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 02 10:01:45.875200 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 02 10:01:45.875200 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 02 10:01:45.875200 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 02 10:01:45.875200 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 02 10:01:45.875200 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 02 10:01:45.875200 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 02 10:01:45.875200 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 02 10:01:45.875200 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 02 10:01:45.875200 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 02 10:01:45.875200 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application return self.get_or_create( Aug 02 10:01:45.875200 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 02 10:01:45.875200 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application with Lock( Aug 02 10:01:45.875200 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 02 10:01:45.875200 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application return self._enter() Aug 02 10:01:45.875200 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 02 10:01:45.875200 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 02 10:01:45.877154 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 02 10:01:45.877154 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application return self.creator() Aug 02 10:01:45.877154 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 02 10:01:45.877154 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application created_value = creator( Aug 02 10:01:45.877154 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 02 10:01:45.877154 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 02 10:01:45.877154 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 02 10:01:45.877154 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 02 10:01:45.877154 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 02 10:01:45.877154 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 02 10:01:45.877154 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Aug 02 10:01:45.877154 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application Aug 02 10:01:45.880258 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 168/340] 10.222.0.82 () {56 vars in 1099 bytes} [Tue Aug 2 10:01:45 2022] GET /identity/v3/roles/reader => generated 84 bytes in 537 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 02 10:01:45.895552 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00692e8f-96b3-4864-b495-53cfde1d295b None admin] Authenticating user token {{(pid=80378) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:01:45.953850 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80a818f1-2aa6-4abc-8a7b-81b9e1808384 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '891005fa5ff74041b75747d16cc66798', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80377) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:01:45.953850 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80a818f1-2aa6-4abc-8a7b-81b9e1808384 None admin] REQUEST_METHOD: `GET` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:01:45.953850 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80a818f1-2aa6-4abc-8a7b-81b9e1808384 None admin] SCRIPT_NAME: `/identity` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:01:45.953850 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80a818f1-2aa6-4abc-8a7b-81b9e1808384 None admin] PATH_INFO: `/v3/roles` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:01:45.955058 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80a818f1-2aa6-4abc-8a7b-81b9e1808384 None admin] RBAC: Adding query filter params (name=member) {{(pid=80377) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 02 10:01:45.955058 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80a818f1-2aa6-4abc-8a7b-81b9e1808384 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:01:45.957037 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80a818f1-2aa6-4abc-8a7b-81b9e1808384 None admin] RBAC: Authorization granted {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:01:45.980878 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: [pid: 80377|app: 0|req: 173/341] 10.222.0.82 () {56 vars in 1101 bytes} [Tue Aug 2 10:01:45 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 293 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 02 10:01:45.984028 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80a818f1-2aa6-4abc-8a7b-81b9e1808384 None admin] Authenticating user token {{(pid=80377) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:01:46.002788 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG dbcounter [-] [80377] Writing DB stats keystone:SELECT=100 {{(pid=80377) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 10:01:46.181410 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80a818f1-2aa6-4abc-8a7b-81b9e1808384 None admin] Validating token access rules against request {{(pid=80377) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:01:46.203095 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00692e8f-96b3-4864-b495-53cfde1d295b None admin] Validating token access rules against request {{(pid=80378) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:01:46.240586 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1891853-5876-4610-8350-7166b577fe6f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '891005fa5ff74041b75747d16cc66798', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80377) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:01:46.241572 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1891853-5876-4610-8350-7166b577fe6f None admin] REQUEST_METHOD: `GET` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:01:46.242007 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1891853-5876-4610-8350-7166b577fe6f None admin] SCRIPT_NAME: `/identity` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:01:46.242471 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1891853-5876-4610-8350-7166b577fe6f None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:01:46.248360 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG dbcounter [-] [80378] Writing DB stats keystone:SELECT=100 {{(pid=80378) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 10:01:46.333991 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f215eac4-6538-495c-bb7f-495e7ac36ebc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '891005fa5ff74041b75747d16cc66798', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80378) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:01:46.335096 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f215eac4-6538-495c-bb7f-495e7ac36ebc None admin] REQUEST_METHOD: `GET` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:01:46.335642 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f215eac4-6538-495c-bb7f-495e7ac36ebc None admin] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:01:46.336124 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f215eac4-6538-495c-bb7f-495e7ac36ebc None admin] PATH_INFO: `/v3/roles` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:01:46.337454 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f215eac4-6538-495c-bb7f-495e7ac36ebc None admin] RBAC: Adding query filter params (name=reader) {{(pid=80378) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 02 10:01:46.342193 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f215eac4-6538-495c-bb7f-495e7ac36ebc None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:01:46.345579 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f215eac4-6538-495c-bb7f-495e7ac36ebc None admin] RBAC: Authorization granted {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:01:46.380759 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 169/342] 10.222.0.82 () {56 vars in 1101 bytes} [Tue Aug 2 10:01:45 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 486 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 02 10:01:46.383548 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d45276bd-723a-44dc-bd28-9cf04feda42d None None] REQUEST_METHOD: `GET` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:01:46.383692 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d45276bd-723a-44dc-bd28-9cf04feda42d None None] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:01:46.384671 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d45276bd-723a-44dc-bd28-9cf04feda42d None None] PATH_INFO: `/` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:01:46.387392 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 170/343] 10.222.0.82 () {54 vars in 926 bytes} [Tue Aug 2 10:01:46 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 02 10:01:46.389008 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d45276bd-723a-44dc-bd28-9cf04feda42d None None] Authenticating user token {{(pid=80378) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:01:46.442461 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1891853-5876-4610-8350-7166b577fe6f None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:01:46.445110 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1891853-5876-4610-8350-7166b577fe6f None admin] RBAC: Authorization granted {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:01:46.492745 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG dbcounter [-] [80377] Writing DB stats keystone:SELECT=100 {{(pid=80377) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 10:01:46.698115 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: [pid: 80377|app: 0|req: 174/344] 10.222.0.82 () {58 vars in 1282 bytes} [Tue Aug 2 10:01:45 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 715 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 02 10:01:46.698547 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d45276bd-723a-44dc-bd28-9cf04feda42d None None] Validating token access rules against request {{(pid=80378) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:01:46.700005 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-640eedb8-cbdf-4b0b-a9d9-ee5f35723156 None None] REQUEST_METHOD: `POST` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:01:46.700411 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-640eedb8-cbdf-4b0b-a9d9-ee5f35723156 None None] SCRIPT_NAME: `/identity` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:01:46.700789 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-640eedb8-cbdf-4b0b-a9d9-ee5f35723156 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:01:46.737625 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.auth.core [None req-640eedb8-cbdf-4b0b-a9d9-ee5f35723156 None None] MFA Rules not processed for user `891005fa5ff74041b75747d16cc66798`. Rule list: `[]` (Enabled: `True`). {{(pid=80377) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 02 10:01:46.739506 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.fernet_utils [None req-640eedb8-cbdf-4b0b-a9d9-ee5f35723156 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80377) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 10:01:46.769058 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34bd49b7-9bcc-4a5d-b5a0-c3378c7fced6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '891005fa5ff74041b75747d16cc66798', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80378) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:01:46.770227 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34bd49b7-9bcc-4a5d-b5a0-c3378c7fced6 None admin] REQUEST_METHOD: `GET` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:01:46.770723 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34bd49b7-9bcc-4a5d-b5a0-c3378c7fced6 None admin] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:01:46.771215 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34bd49b7-9bcc-4a5d-b5a0-c3378c7fced6 None admin] PATH_INFO: `/v3/users/8a9b9d020f064f5ab9f664ee649e8389` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:01:46.773370 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34bd49b7-9bcc-4a5d-b5a0-c3378c7fced6 None admin] RBAC: Authorizing `identity:get_user(user_id=8a9b9d020f064f5ab9f664ee649e8389)` {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:01:46.776075 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34bd49b7-9bcc-4a5d-b5a0-c3378c7fced6 None admin] RBAC: Authorization granted {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:01:46.779546 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 171/345] 10.222.0.82 () {56 vars in 1177 bytes} [Tue Aug 2 10:01:46 2022] GET /identity/v3/users/8a9b9d020f064f5ab9f664ee649e8389 => generated 293 bytes in 392 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 02 10:01:46.780958 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34bd49b7-9bcc-4a5d-b5a0-c3378c7fced6 None admin] Authenticating user token {{(pid=80378) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:01:46.854972 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: [pid: 80377|app: 0|req: 175/346] 10.222.0.82 () {58 vars in 1023 bytes} [Tue Aug 2 10:01:46 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 156 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 02 10:01:46.856118 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-640eedb8-cbdf-4b0b-a9d9-ee5f35723156 None None] Authenticating user token {{(pid=80377) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:01:46.884643 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG dbcounter [-] [80378] Writing DB stats keystone:SELECT=100 {{(pid=80378) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 10:01:46.964738 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG dbcounter [-] [80377] Writing DB stats keystone:SELECT=100 {{(pid=80377) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 10:01:47.046900 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34bd49b7-9bcc-4a5d-b5a0-c3378c7fced6 None admin] Validating token access rules against request {{(pid=80378) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:01:47.093199 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-640eedb8-cbdf-4b0b-a9d9-ee5f35723156 None None] Validating token access rules against request {{(pid=80377) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:01:47.131373 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80aa5b40-fc1b-4c9a-8219-58ca4d4b2ae6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '891005fa5ff74041b75747d16cc66798', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80378) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:01:47.132767 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80aa5b40-fc1b-4c9a-8219-58ca4d4b2ae6 None admin] REQUEST_METHOD: `GET` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:01:47.133476 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80aa5b40-fc1b-4c9a-8219-58ca4d4b2ae6 None admin] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:01:47.134774 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80aa5b40-fc1b-4c9a-8219-58ca4d4b2ae6 None admin] PATH_INFO: `/v3/projects/c1809c9e380b45ec803130fe38eeaf24` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:01:47.137007 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80aa5b40-fc1b-4c9a-8219-58ca4d4b2ae6 None admin] RBAC: Authorizing `identity:get_project(project_id=c1809c9e380b45ec803130fe38eeaf24)` {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:01:47.139810 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80aa5b40-fc1b-4c9a-8219-58ca4d4b2ae6 None admin] RBAC: Authorization granted {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:01:47.143672 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 172/347] 10.222.0.82 () {56 vars in 1186 bytes} [Tue Aug 2 10:01:46 2022] GET /identity/v3/projects/c1809c9e380b45ec803130fe38eeaf24 => generated 298 bytes in 363 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 02 10:01:47.148868 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80aa5b40-fc1b-4c9a-8219-58ca4d4b2ae6 None admin] Authenticating user token {{(pid=80378) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:01:47.174331 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e6b66de-ce0e-46b6-9578-7a6ffd9485b2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '891005fa5ff74041b75747d16cc66798', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80377) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:01:47.175676 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e6b66de-ce0e-46b6-9578-7a6ffd9485b2 None admin] REQUEST_METHOD: `GET` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:01:47.176406 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e6b66de-ce0e-46b6-9578-7a6ffd9485b2 None admin] SCRIPT_NAME: `/identity` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:01:47.177244 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e6b66de-ce0e-46b6-9578-7a6ffd9485b2 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:01:47.178757 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e6b66de-ce0e-46b6-9578-7a6ffd9485b2 None admin] RBAC: Adding query filter params (role.id=444e768aa5fb44bbab6c20afad6a0218, user.id=8a9b9d020f064f5ab9f664ee649e8389, scope.system=all) {{(pid=80377) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 02 10:01:47.179567 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e6b66de-ce0e-46b6-9578-7a6ffd9485b2 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:01:47.182331 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e6b66de-ce0e-46b6-9578-7a6ffd9485b2 None admin] RBAC: Authorization granted {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:01:47.202404 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: [pid: 80377|app: 0|req: 176/348] 10.222.0.82 () {56 vars in 1308 bytes} [Tue Aug 2 10:01:46 2022] GET /identity/v3/role_assignments?user.id=8a9b9d020f064f5ab9f664ee649e8389&scope.system=all&role.id=444e768aa5fb44bbab6c20afad6a0218 => generated 510 bytes in 347 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 02 10:01:47.205527 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53fd1cdb-7b46-47e7-8967-847c5730837f None None] REQUEST_METHOD: `GET` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:01:47.206613 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53fd1cdb-7b46-47e7-8967-847c5730837f None None] SCRIPT_NAME: `/identity` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:01:47.207266 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53fd1cdb-7b46-47e7-8967-847c5730837f None None] PATH_INFO: `/` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:01:47.213997 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: [pid: 80377|app: 0|req: 177/349] 10.222.0.82 () {54 vars in 926 bytes} [Tue Aug 2 10:01:47 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 02 10:01:47.215405 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53fd1cdb-7b46-47e7-8967-847c5730837f None None] Authenticating user token {{(pid=80377) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:01:47.388018 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG dbcounter [-] [80377] Writing DB stats keystone:SELECT=100 {{(pid=80377) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 10:01:47.398399 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53fd1cdb-7b46-47e7-8967-847c5730837f None None] Validating token access rules against request {{(pid=80377) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:01:47.404507 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80aa5b40-fc1b-4c9a-8219-58ca4d4b2ae6 None admin] Validating token access rules against request {{(pid=80378) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:01:47.423141 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG dbcounter [-] [80378] Writing DB stats keystone:SELECT=100 {{(pid=80378) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 10:01:47.454333 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b24bcef0-b8db-4c90-b4e6-ff11fb4939b4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '891005fa5ff74041b75747d16cc66798', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80377) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:01:47.454333 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b24bcef0-b8db-4c90-b4e6-ff11fb4939b4 None admin] REQUEST_METHOD: `GET` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:01:47.454333 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b24bcef0-b8db-4c90-b4e6-ff11fb4939b4 None admin] SCRIPT_NAME: `/identity` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:01:47.454333 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b24bcef0-b8db-4c90-b4e6-ff11fb4939b4 None admin] PATH_INFO: `/v3/groups/7a7f06a893c2484aafc43a46704b6e2a` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:01:47.466058 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b24bcef0-b8db-4c90-b4e6-ff11fb4939b4 None admin] RBAC: Authorizing `identity:get_group(group_id=7a7f06a893c2484aafc43a46704b6e2a)` {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:01:47.469008 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b24bcef0-b8db-4c90-b4e6-ff11fb4939b4 None admin] RBAC: Authorization granted {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:01:47.470544 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7f26912-7b86-4b49-864b-0ae8f454716f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '891005fa5ff74041b75747d16cc66798', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80378) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:01:47.472055 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7f26912-7b86-4b49-864b-0ae8f454716f None admin] REQUEST_METHOD: `GET` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:01:47.472813 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7f26912-7b86-4b49-864b-0ae8f454716f None admin] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:01:47.473503 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7f26912-7b86-4b49-864b-0ae8f454716f None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:01:47.474464 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: [pid: 80377|app: 0|req: 178/350] 10.222.0.82 () {56 vars in 1180 bytes} [Tue Aug 2 10:01:47 2022] GET /identity/v3/groups/7a7f06a893c2484aafc43a46704b6e2a => generated 226 bytes in 260 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 02 10:01:47.476894 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd8094a0-070e-4d1d-a287-b0e7eb14f59a None None] REQUEST_METHOD: `POST` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:01:47.477388 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd8094a0-070e-4d1d-a287-b0e7eb14f59a None None] SCRIPT_NAME: `/identity` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:01:47.477879 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd8094a0-070e-4d1d-a287-b0e7eb14f59a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:01:47.494953 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7f26912-7b86-4b49-864b-0ae8f454716f None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 02 10:01:47.508121 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application [None req-e7f26912-7b86-4b49-864b-0ae8f454716f None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Aug 02 10:01:47.508121 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 02 10:01:47.508121 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 02 10:01:47.508121 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 02 10:01:47.508121 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 02 10:01:47.508121 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 02 10:01:47.508121 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 02 10:01:47.508121 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 02 10:01:47.508121 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 02 10:01:47.508121 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 02 10:01:47.508121 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 02 10:01:47.508121 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 02 10:01:47.508121 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 02 10:01:47.508121 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 02 10:01:47.508121 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 02 10:01:47.508121 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 02 10:01:47.509945 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 02 10:01:47.509945 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 02 10:01:47.509945 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 02 10:01:47.509945 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 02 10:01:47.509945 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 02 10:01:47.509945 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 02 10:01:47.509945 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 02 10:01:47.509945 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 02 10:01:47.509945 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 02 10:01:47.509945 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application return self.get_or_create( Aug 02 10:01:47.509945 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 02 10:01:47.509945 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application with Lock( Aug 02 10:01:47.509945 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 02 10:01:47.509945 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application return self._enter() Aug 02 10:01:47.509945 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 02 10:01:47.509945 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 02 10:01:47.511501 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 02 10:01:47.511501 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application return self.creator() Aug 02 10:01:47.511501 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 02 10:01:47.511501 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application created_value = creator( Aug 02 10:01:47.511501 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 02 10:01:47.511501 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 02 10:01:47.511501 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 02 10:01:47.511501 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 02 10:01:47.511501 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 02 10:01:47.511501 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 02 10:01:47.511501 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Aug 02 10:01:47.511501 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application Aug 02 10:01:47.512525 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.auth.core [None req-bd8094a0-070e-4d1d-a287-b0e7eb14f59a None None] MFA Rules not processed for user `891005fa5ff74041b75747d16cc66798`. Rule list: `[]` (Enabled: `True`). {{(pid=80377) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 02 10:01:47.512525 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.fernet_utils [None req-bd8094a0-070e-4d1d-a287-b0e7eb14f59a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80377) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 10:01:47.516516 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 173/351] 10.222.0.82 () {56 vars in 1114 bytes} [Tue Aug 2 10:01:47 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 372 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 02 10:01:47.518772 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7f26912-7b86-4b49-864b-0ae8f454716f None admin] Authenticating user token {{(pid=80378) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:01:47.629150 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: [pid: 80377|app: 0|req: 179/352] 10.222.0.82 () {58 vars in 1023 bytes} [Tue Aug 2 10:01:47 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 154 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 02 10:01:47.634791 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd8094a0-070e-4d1d-a287-b0e7eb14f59a None None] Authenticating user token {{(pid=80377) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:01:47.775288 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7f26912-7b86-4b49-864b-0ae8f454716f None admin] Validating token access rules against request {{(pid=80378) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:01:47.829323 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG dbcounter [-] [80377] Writing DB stats keystone:SELECT=100 {{(pid=80377) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 10:01:47.839761 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f12f248-8d57-418d-a168-b4eea15f66de None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '891005fa5ff74041b75747d16cc66798', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80378) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:01:47.840577 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f12f248-8d57-418d-a168-b4eea15f66de None admin] REQUEST_METHOD: `GET` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:01:47.840861 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f12f248-8d57-418d-a168-b4eea15f66de None admin] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:01:47.841501 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f12f248-8d57-418d-a168-b4eea15f66de None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:01:47.842466 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f12f248-8d57-418d-a168-b4eea15f66de None admin] RBAC: Adding query filter params (group.id=7a7f06a893c2484aafc43a46704b6e2a, role.id=e1c04b19fc4543b8b54f58bbcffbc598, scope.project.id=c1809c9e380b45ec803130fe38eeaf24) {{(pid=80378) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 02 10:01:47.842786 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f12f248-8d57-418d-a168-b4eea15f66de None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:01:47.844379 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f12f248-8d57-418d-a168-b4eea15f66de None admin] RBAC: Authorization granted {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:01:47.844939 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd8094a0-070e-4d1d-a287-b0e7eb14f59a None None] Validating token access rules against request {{(pid=80377) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:01:47.853154 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 174/353] 10.222.0.82 () {56 vars in 1376 bytes} [Tue Aug 2 10:01:47 2022] GET /identity/v3/role_assignments?group.id=7a7f06a893c2484aafc43a46704b6e2a&scope.project.id=c1809c9e380b45ec803130fe38eeaf24&role.id=e1c04b19fc4543b8b54f58bbcffbc598 => generated 260 bytes in 335 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 02 10:01:47.854012 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-214872fa-84a5-4f73-a585-d99c5e252723 None None] REQUEST_METHOD: `GET` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:01:47.854461 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-214872fa-84a5-4f73-a585-d99c5e252723 None None] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:01:47.854746 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-214872fa-84a5-4f73-a585-d99c5e252723 None None] PATH_INFO: `/` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:01:47.856572 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 175/354] 10.222.0.82 () {54 vars in 926 bytes} [Tue Aug 2 10:01:47 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 02 10:01:47.861594 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-214872fa-84a5-4f73-a585-d99c5e252723 None None] Authenticating user token {{(pid=80378) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:01:47.916171 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b733530-1468-4b70-9487-60db748d463b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '891005fa5ff74041b75747d16cc66798', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80377) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:01:47.916941 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b733530-1468-4b70-9487-60db748d463b None admin] REQUEST_METHOD: `GET` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:01:47.917298 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b733530-1468-4b70-9487-60db748d463b None admin] SCRIPT_NAME: `/identity` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:01:47.917583 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b733530-1468-4b70-9487-60db748d463b None admin] PATH_INFO: `/v3/roles` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:01:47.918597 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b733530-1468-4b70-9487-60db748d463b None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=80377) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 02 10:01:47.919047 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b733530-1468-4b70-9487-60db748d463b None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:01:47.920920 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b733530-1468-4b70-9487-60db748d463b None admin] RBAC: Authorization granted {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:01:47.936092 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: [pid: 80377|app: 0|req: 180/355] 10.222.0.82 () {56 vars in 1111 bytes} [Tue Aug 2 10:01:47 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 306 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 02 10:01:47.937674 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f47b9b04-6be2-4a0d-aa2b-81b4f5402855 None None] REQUEST_METHOD: `POST` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:01:47.938116 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f47b9b04-6be2-4a0d-aa2b-81b4f5402855 None None] SCRIPT_NAME: `/identity` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:01:47.938934 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f47b9b04-6be2-4a0d-aa2b-81b4f5402855 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:01:47.955860 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG dbcounter [-] [80378] Writing DB stats keystone:SELECT=100 {{(pid=80378) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 10:01:47.968981 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.auth.core [None req-f47b9b04-6be2-4a0d-aa2b-81b4f5402855 None None] MFA Rules not processed for user `891005fa5ff74041b75747d16cc66798`. Rule list: `[]` (Enabled: `True`). {{(pid=80377) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 02 10:01:47.971916 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.fernet_utils [None req-f47b9b04-6be2-4a0d-aa2b-81b4f5402855 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80377) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 10:01:48.093876 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-214872fa-84a5-4f73-a585-d99c5e252723 None None] Validating token access rules against request {{(pid=80378) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:01:48.138034 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: [pid: 80377|app: 0|req: 181/356] 10.222.0.82 () {58 vars in 1023 bytes} [Tue Aug 2 10:01:47 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 201 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 02 10:01:48.140051 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f47b9b04-6be2-4a0d-aa2b-81b4f5402855 None None] Authenticating user token {{(pid=80377) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:01:48.179191 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-244a7a8e-fae7-47b0-86ff-b5ce2818dacc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '891005fa5ff74041b75747d16cc66798', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80378) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:01:48.180298 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-244a7a8e-fae7-47b0-86ff-b5ce2818dacc None admin] REQUEST_METHOD: `GET` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:01:48.181244 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-244a7a8e-fae7-47b0-86ff-b5ce2818dacc None admin] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:01:48.182108 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-244a7a8e-fae7-47b0-86ff-b5ce2818dacc None admin] PATH_INFO: `/v3/roles/member` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:01:48.203746 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-244a7a8e-fae7-47b0-86ff-b5ce2818dacc None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 02 10:01:48.215369 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application [None req-244a7a8e-fae7-47b0-86ff-b5ce2818dacc None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Aug 02 10:01:48.215369 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 02 10:01:48.215369 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 02 10:01:48.215369 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 02 10:01:48.215369 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 02 10:01:48.215369 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 02 10:01:48.215369 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 02 10:01:48.215369 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 02 10:01:48.215369 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 02 10:01:48.215369 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 02 10:01:48.215369 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 02 10:01:48.215369 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 02 10:01:48.215369 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 02 10:01:48.215369 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 02 10:01:48.215369 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 02 10:01:48.215369 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 02 10:01:48.216823 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 02 10:01:48.216823 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 02 10:01:48.216823 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 02 10:01:48.216823 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 02 10:01:48.216823 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 02 10:01:48.216823 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 02 10:01:48.216823 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 02 10:01:48.216823 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 02 10:01:48.216823 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 02 10:01:48.216823 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application return self.get_or_create( Aug 02 10:01:48.216823 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 02 10:01:48.216823 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application with Lock( Aug 02 10:01:48.216823 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 02 10:01:48.216823 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application return self._enter() Aug 02 10:01:48.216823 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 02 10:01:48.216823 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 02 10:01:48.218105 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 02 10:01:48.218105 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application return self.creator() Aug 02 10:01:48.218105 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 02 10:01:48.218105 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application created_value = creator( Aug 02 10:01:48.218105 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 02 10:01:48.218105 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 02 10:01:48.218105 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 02 10:01:48.218105 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 02 10:01:48.218105 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 02 10:01:48.218105 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 02 10:01:48.218105 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Aug 02 10:01:48.218105 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application Aug 02 10:01:48.218105 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 176/357] 10.222.0.82 () {56 vars in 1099 bytes} [Tue Aug 2 10:01:47 2022] GET /identity/v3/roles/member => generated 84 bytes in 361 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 02 10:01:48.227037 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-244a7a8e-fae7-47b0-86ff-b5ce2818dacc None admin] Authenticating user token {{(pid=80378) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:01:48.274038 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG dbcounter [-] [80377] Writing DB stats keystone:SELECT=100 {{(pid=80377) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 10:01:48.309500 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f47b9b04-6be2-4a0d-aa2b-81b4f5402855 None None] Validating token access rules against request {{(pid=80377) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:01:48.366506 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6aa6b1c-d445-42b2-bda2-9b188ff7c918 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '891005fa5ff74041b75747d16cc66798', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80377) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:01:48.367122 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6aa6b1c-d445-42b2-bda2-9b188ff7c918 None admin] REQUEST_METHOD: `GET` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:01:48.367387 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6aa6b1c-d445-42b2-bda2-9b188ff7c918 None admin] SCRIPT_NAME: `/identity` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:01:48.367663 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6aa6b1c-d445-42b2-bda2-9b188ff7c918 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:01:48.432967 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG dbcounter [-] [80378] Writing DB stats keystone:SELECT=100 {{(pid=80378) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 10:01:48.440735 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-244a7a8e-fae7-47b0-86ff-b5ce2818dacc None admin] Validating token access rules against request {{(pid=80378) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:01:48.475957 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6aa6b1c-d445-42b2-bda2-9b188ff7c918 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:01:48.478915 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6aa6b1c-d445-42b2-bda2-9b188ff7c918 None admin] RBAC: Authorization granted {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:01:48.497823 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0ed2d4c-9b70-49b4-b57f-93864640e278 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '891005fa5ff74041b75747d16cc66798', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80378) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:01:48.497823 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0ed2d4c-9b70-49b4-b57f-93864640e278 None admin] REQUEST_METHOD: `GET` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:01:48.497823 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0ed2d4c-9b70-49b4-b57f-93864640e278 None admin] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:01:48.497823 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0ed2d4c-9b70-49b4-b57f-93864640e278 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:01:48.515306 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0ed2d4c-9b70-49b4-b57f-93864640e278 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 02 10:01:48.521683 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application [None req-f0ed2d4c-9b70-49b4-b57f-93864640e278 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Aug 02 10:01:48.521683 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 02 10:01:48.521683 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 02 10:01:48.521683 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 02 10:01:48.521683 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 02 10:01:48.521683 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 02 10:01:48.521683 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 02 10:01:48.521683 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 02 10:01:48.521683 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 02 10:01:48.521683 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 02 10:01:48.521683 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 02 10:01:48.521683 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 02 10:01:48.521683 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 02 10:01:48.521683 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 02 10:01:48.521683 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 02 10:01:48.521683 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 02 10:01:48.523053 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 02 10:01:48.523053 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 02 10:01:48.523053 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 02 10:01:48.523053 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 02 10:01:48.523053 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 02 10:01:48.523053 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 02 10:01:48.523053 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 02 10:01:48.523053 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 02 10:01:48.523053 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 02 10:01:48.523053 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application return self.get_or_create( Aug 02 10:01:48.523053 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 02 10:01:48.523053 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application with Lock( Aug 02 10:01:48.523053 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 02 10:01:48.523053 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application return self._enter() Aug 02 10:01:48.523053 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 02 10:01:48.523053 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 02 10:01:48.524737 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 02 10:01:48.524737 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application return self.creator() Aug 02 10:01:48.524737 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 02 10:01:48.524737 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application created_value = creator( Aug 02 10:01:48.524737 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 02 10:01:48.524737 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 02 10:01:48.524737 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 02 10:01:48.524737 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 02 10:01:48.524737 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 02 10:01:48.524737 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 02 10:01:48.524737 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Aug 02 10:01:48.524737 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application Aug 02 10:01:48.524737 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 177/358] 10.222.0.82 () {56 vars in 1114 bytes} [Tue Aug 2 10:01:48 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 304 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 02 10:01:48.524737 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0ed2d4c-9b70-49b4-b57f-93864640e278 None admin] Authenticating user token {{(pid=80378) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:01:48.645020 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG dbcounter [-] [80377] Writing DB stats keystone:SELECT=100 {{(pid=80377) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 10:01:48.674968 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: [pid: 80377|app: 0|req: 182/359] 10.222.0.82 () {58 vars in 1282 bytes} [Tue Aug 2 10:01:48 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 536 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 02 10:01:48.676304 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-223939b7-3cab-46e2-a07d-6c620b1613f3 None None] REQUEST_METHOD: `GET` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:01:48.676562 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-223939b7-3cab-46e2-a07d-6c620b1613f3 None None] SCRIPT_NAME: `/identity` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:01:48.676817 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-223939b7-3cab-46e2-a07d-6c620b1613f3 None None] PATH_INFO: `/` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:01:48.678203 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: [pid: 80377|app: 0|req: 183/360] 10.222.0.82 () {54 vars in 926 bytes} [Tue Aug 2 10:01:48 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 02 10:01:48.678909 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-223939b7-3cab-46e2-a07d-6c620b1613f3 None None] Authenticating user token {{(pid=80377) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:01:48.721564 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0ed2d4c-9b70-49b4-b57f-93864640e278 None admin] Validating token access rules against request {{(pid=80378) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:01:48.791122 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67611314-18a6-4662-8731-fc56816e31c4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '891005fa5ff74041b75747d16cc66798', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80378) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:01:48.792024 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67611314-18a6-4662-8731-fc56816e31c4 None admin] REQUEST_METHOD: `GET` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:01:48.792602 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67611314-18a6-4662-8731-fc56816e31c4 None admin] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:01:48.793025 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67611314-18a6-4662-8731-fc56816e31c4 None admin] PATH_INFO: `/v3/roles` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:01:48.794347 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67611314-18a6-4662-8731-fc56816e31c4 None admin] RBAC: Adding query filter params (name=member) {{(pid=80378) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 02 10:01:48.794753 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67611314-18a6-4662-8731-fc56816e31c4 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:01:48.796656 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67611314-18a6-4662-8731-fc56816e31c4 None admin] RBAC: Authorization granted {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:01:48.809894 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 178/361] 10.222.0.82 () {56 vars in 1101 bytes} [Tue Aug 2 10:01:48 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 286 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 02 10:01:48.810172 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-223939b7-3cab-46e2-a07d-6c620b1613f3 None None] Validating token access rules against request {{(pid=80377) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:01:48.811417 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67611314-18a6-4662-8731-fc56816e31c4 None admin] Authenticating user token {{(pid=80378) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:01:48.841432 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG dbcounter [-] [80378] Writing DB stats keystone:SELECT=100 {{(pid=80378) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 10:01:48.865456 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39623668-6a93-4422-9c31-ca7c84f9bb1a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '891005fa5ff74041b75747d16cc66798', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80377) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:01:48.866752 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39623668-6a93-4422-9c31-ca7c84f9bb1a None admin] REQUEST_METHOD: `GET` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:01:48.867253 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39623668-6a93-4422-9c31-ca7c84f9bb1a None admin] SCRIPT_NAME: `/identity` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:01:48.867751 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39623668-6a93-4422-9c31-ca7c84f9bb1a None admin] PATH_INFO: `/v3/roles` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:01:48.869165 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39623668-6a93-4422-9c31-ca7c84f9bb1a None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=80377) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 02 10:01:48.869719 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39623668-6a93-4422-9c31-ca7c84f9bb1a None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:01:48.872118 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39623668-6a93-4422-9c31-ca7c84f9bb1a None admin] RBAC: Authorization granted {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:01:48.884095 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: [pid: 80377|app: 0|req: 184/362] 10.222.0.82 () {56 vars in 1111 bytes} [Tue Aug 2 10:01:48 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 205 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 02 10:01:48.889818 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09238ebb-dc05-4a25-b281-068c85906aa9 None None] REQUEST_METHOD: `POST` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:01:48.889818 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09238ebb-dc05-4a25-b281-068c85906aa9 None None] SCRIPT_NAME: `/identity` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:01:48.889818 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09238ebb-dc05-4a25-b281-068c85906aa9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:01:48.911255 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.auth.core [None req-09238ebb-dc05-4a25-b281-068c85906aa9 None None] MFA Rules not processed for user `891005fa5ff74041b75747d16cc66798`. Rule list: `[]` (Enabled: `True`). {{(pid=80377) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 02 10:01:48.913142 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.fernet_utils [None req-09238ebb-dc05-4a25-b281-068c85906aa9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80377) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 10:01:48.971341 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG dbcounter [-] [80377] Writing DB stats keystone:SELECT=100 {{(pid=80377) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 10:01:48.977056 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67611314-18a6-4662-8731-fc56816e31c4 None admin] Validating token access rules against request {{(pid=80378) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:01:49.019074 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: [pid: 80377|app: 0|req: 185/363] 10.222.0.82 () {58 vars in 1023 bytes} [Tue Aug 2 10:01:48 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 134 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 02 10:01:49.019275 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1387bc3-c3fc-41d9-ae58-cd7d178a35bb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '891005fa5ff74041b75747d16cc66798', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80378) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:01:49.019852 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1387bc3-c3fc-41d9-ae58-cd7d178a35bb None admin] REQUEST_METHOD: `GET` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:01:49.020140 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1387bc3-c3fc-41d9-ae58-cd7d178a35bb None admin] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:01:49.020425 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1387bc3-c3fc-41d9-ae58-cd7d178a35bb None admin] PATH_INFO: `/v3/projects/c1809c9e380b45ec803130fe38eeaf24` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:01:49.022966 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1387bc3-c3fc-41d9-ae58-cd7d178a35bb None admin] RBAC: Authorizing `identity:get_project(project_id=c1809c9e380b45ec803130fe38eeaf24)` {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:01:49.023128 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09238ebb-dc05-4a25-b281-068c85906aa9 None None] Authenticating user token {{(pid=80377) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:01:49.023461 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1387bc3-c3fc-41d9-ae58-cd7d178a35bb None admin] RBAC: Authorization granted {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:01:49.025306 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 179/364] 10.222.0.82 () {56 vars in 1186 bytes} [Tue Aug 2 10:01:48 2022] GET /identity/v3/projects/c1809c9e380b45ec803130fe38eeaf24 => generated 298 bytes in 215 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 02 10:01:49.026177 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1387bc3-c3fc-41d9-ae58-cd7d178a35bb None admin] Authenticating user token {{(pid=80378) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:01:49.168648 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG dbcounter [-] [80378] Writing DB stats keystone:SELECT=100 {{(pid=80378) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 10:01:49.190120 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1387bc3-c3fc-41d9-ae58-cd7d178a35bb None admin] Validating token access rules against request {{(pid=80378) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:01:49.217412 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09238ebb-dc05-4a25-b281-068c85906aa9 None None] Validating token access rules against request {{(pid=80377) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:01:49.256197 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef2ebf43-0eb0-48bf-adc0-c472ef2f138f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '891005fa5ff74041b75747d16cc66798', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80378) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:01:49.257152 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef2ebf43-0eb0-48bf-adc0-c472ef2f138f None admin] REQUEST_METHOD: `GET` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:01:49.257846 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef2ebf43-0eb0-48bf-adc0-c472ef2f138f None admin] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:01:49.258261 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef2ebf43-0eb0-48bf-adc0-c472ef2f138f None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:01:49.269884 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb0ab7b7-efb2-4659-a321-59e2dbe99f06 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '891005fa5ff74041b75747d16cc66798', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80377) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:01:49.270605 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb0ab7b7-efb2-4659-a321-59e2dbe99f06 None admin] REQUEST_METHOD: `GET` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:01:49.270884 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb0ab7b7-efb2-4659-a321-59e2dbe99f06 None admin] SCRIPT_NAME: `/identity` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:01:49.271154 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb0ab7b7-efb2-4659-a321-59e2dbe99f06 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:01:49.352726 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG dbcounter [-] [80377] Writing DB stats keystone:SELECT=100 {{(pid=80377) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 10:01:49.368716 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef2ebf43-0eb0-48bf-adc0-c472ef2f138f None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:01:49.370275 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef2ebf43-0eb0-48bf-adc0-c472ef2f138f None admin] RBAC: Authorization granted {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:01:49.403294 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb0ab7b7-efb2-4659-a321-59e2dbe99f06 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:01:49.405976 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb0ab7b7-efb2-4659-a321-59e2dbe99f06 None admin] RBAC: Authorization granted {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:01:49.495554 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG dbcounter [-] [80378] Writing DB stats keystone:SELECT=100 {{(pid=80378) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 10:01:49.521884 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 180/365] 10.222.0.82 () {58 vars in 1282 bytes} [Tue Aug 2 10:01:49 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 496 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 02 10:01:49.522737 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef2ebf43-0eb0-48bf-adc0-c472ef2f138f None admin] Authenticating user token {{(pid=80378) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:01:49.552704 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: [pid: 80377|app: 0|req: 186/366] 10.222.0.82 () {58 vars in 1282 bytes} [Tue Aug 2 10:01:49 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 532 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 02 10:01:49.558734 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb0ab7b7-efb2-4659-a321-59e2dbe99f06 None admin] Authenticating user token {{(pid=80377) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:01:49.713272 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef2ebf43-0eb0-48bf-adc0-c472ef2f138f None admin] Validating token access rules against request {{(pid=80378) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:01:49.713572 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG dbcounter [-] [80377] Writing DB stats keystone:SELECT=100 {{(pid=80377) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 10:01:49.762537 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb0ab7b7-efb2-4659-a321-59e2dbe99f06 None admin] Validating token access rules against request {{(pid=80377) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:01:49.797479 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d140e885-2329-4b89-bfab-8ad0808de9c6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '891005fa5ff74041b75747d16cc66798', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80378) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:01:49.798766 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d140e885-2329-4b89-bfab-8ad0808de9c6 None admin] REQUEST_METHOD: `GET` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:01:49.799045 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d140e885-2329-4b89-bfab-8ad0808de9c6 None admin] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:01:49.799319 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d140e885-2329-4b89-bfab-8ad0808de9c6 None admin] PATH_INFO: `/v3/groups/7a7f06a893c2484aafc43a46704b6e2a` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:01:49.800709 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d140e885-2329-4b89-bfab-8ad0808de9c6 None admin] RBAC: Authorizing `identity:get_group(group_id=7a7f06a893c2484aafc43a46704b6e2a)` {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:01:49.804181 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d140e885-2329-4b89-bfab-8ad0808de9c6 None admin] RBAC: Authorization granted {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:01:49.807186 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 181/367] 10.222.0.82 () {56 vars in 1180 bytes} [Tue Aug 2 10:01:49 2022] GET /identity/v3/groups/7a7f06a893c2484aafc43a46704b6e2a => generated 226 bytes in 285 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 02 10:01:49.808415 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d140e885-2329-4b89-bfab-8ad0808de9c6 None admin] Authenticating user token {{(pid=80378) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:01:49.834458 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29110c46-2b7a-41d1-b857-b8f48b68746f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '891005fa5ff74041b75747d16cc66798', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80377) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:01:49.835983 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29110c46-2b7a-41d1-b857-b8f48b68746f None admin] REQUEST_METHOD: `GET` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:01:49.836625 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29110c46-2b7a-41d1-b857-b8f48b68746f None admin] SCRIPT_NAME: `/identity` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:01:49.837306 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29110c46-2b7a-41d1-b857-b8f48b68746f None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:01:49.869166 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29110c46-2b7a-41d1-b857-b8f48b68746f None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 02 10:01:49.881388 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application [None req-29110c46-2b7a-41d1-b857-b8f48b68746f None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 02 10:01:49.881388 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 02 10:01:49.881388 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 02 10:01:49.881388 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 02 10:01:49.881388 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 02 10:01:49.881388 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 02 10:01:49.881388 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 02 10:01:49.881388 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 02 10:01:49.881388 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 02 10:01:49.881388 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 02 10:01:49.881388 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 02 10:01:49.881388 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 02 10:01:49.881388 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 02 10:01:49.881388 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 02 10:01:49.881388 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 02 10:01:49.881388 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 02 10:01:49.883363 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 02 10:01:49.883363 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 02 10:01:49.883363 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 02 10:01:49.883363 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 02 10:01:49.883363 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 02 10:01:49.883363 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 02 10:01:49.883363 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 02 10:01:49.883363 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 02 10:01:49.883363 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 02 10:01:49.883363 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application return self.get_or_create( Aug 02 10:01:49.883363 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 02 10:01:49.883363 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application with Lock( Aug 02 10:01:49.883363 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 02 10:01:49.883363 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application return self._enter() Aug 02 10:01:49.883363 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 02 10:01:49.883363 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 02 10:01:49.885032 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 02 10:01:49.885032 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application return self.creator() Aug 02 10:01:49.885032 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 02 10:01:49.885032 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application created_value = creator( Aug 02 10:01:49.885032 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 02 10:01:49.885032 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 02 10:01:49.885032 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 02 10:01:49.885032 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 02 10:01:49.885032 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 02 10:01:49.885032 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 02 10:01:49.885032 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 02 10:01:49.885032 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application Aug 02 10:01:49.885032 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: [pid: 80377|app: 0|req: 187/368] 10.222.0.82 () {56 vars in 1096 bytes} [Tue Aug 2 10:01:49 2022] GET /identity/v3/roles/admin => generated 83 bytes in 326 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 02 10:01:49.885032 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29110c46-2b7a-41d1-b857-b8f48b68746f None admin] Authenticating user token {{(pid=80377) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:01:49.939766 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG dbcounter [-] [80378] Writing DB stats keystone:SELECT=100 {{(pid=80378) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 10:01:50.023303 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d140e885-2329-4b89-bfab-8ad0808de9c6 None admin] Validating token access rules against request {{(pid=80378) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:01:50.081491 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29110c46-2b7a-41d1-b857-b8f48b68746f None admin] Validating token access rules against request {{(pid=80377) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:01:50.094510 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-186b48a0-ee05-44a8-81da-c08d60508f2b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '891005fa5ff74041b75747d16cc66798', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80378) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:01:50.095934 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-186b48a0-ee05-44a8-81da-c08d60508f2b None admin] REQUEST_METHOD: `GET` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:01:50.096240 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-186b48a0-ee05-44a8-81da-c08d60508f2b None admin] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:01:50.096645 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-186b48a0-ee05-44a8-81da-c08d60508f2b None admin] PATH_INFO: `/v3/projects/f7db7cf528494fbb8836c08d79b752ef` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:01:50.099050 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-186b48a0-ee05-44a8-81da-c08d60508f2b None admin] RBAC: Authorizing `identity:get_project(project_id=f7db7cf528494fbb8836c08d79b752ef)` {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:01:50.101622 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-186b48a0-ee05-44a8-81da-c08d60508f2b None admin] RBAC: Authorization granted {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:01:50.104299 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 182/369] 10.222.0.82 () {56 vars in 1186 bytes} [Tue Aug 2 10:01:49 2022] GET /identity/v3/projects/f7db7cf528494fbb8836c08d79b752ef => generated 302 bytes in 297 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 02 10:01:50.105672 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-186b48a0-ee05-44a8-81da-c08d60508f2b None admin] Authenticating user token {{(pid=80378) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:01:50.149224 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93940f43-58e1-4b91-8399-c97f93567eed None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '891005fa5ff74041b75747d16cc66798', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80377) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:01:50.149949 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93940f43-58e1-4b91-8399-c97f93567eed None admin] REQUEST_METHOD: `GET` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:01:50.150244 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93940f43-58e1-4b91-8399-c97f93567eed None admin] SCRIPT_NAME: `/identity` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:01:50.150519 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93940f43-58e1-4b91-8399-c97f93567eed None admin] PATH_INFO: `/v3/projects/f7db7cf528494fbb8836c08d79b752ef` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:01:50.153246 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93940f43-58e1-4b91-8399-c97f93567eed None admin] RBAC: Authorizing `identity:get_project(project_id=f7db7cf528494fbb8836c08d79b752ef)` {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:01:50.153486 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93940f43-58e1-4b91-8399-c97f93567eed None admin] RBAC: Authorization granted {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:01:50.155414 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: [pid: 80377|app: 0|req: 188/370] 10.222.0.82 () {56 vars in 1186 bytes} [Tue Aug 2 10:01:49 2022] GET /identity/v3/projects/f7db7cf528494fbb8836c08d79b752ef => generated 302 bytes in 271 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 02 10:01:50.155992 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93940f43-58e1-4b91-8399-c97f93567eed None admin] Authenticating user token {{(pid=80377) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:01:50.160355 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG dbcounter [-] [80377] Writing DB stats keystone:SELECT=100 {{(pid=80377) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 10:01:50.297907 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-186b48a0-ee05-44a8-81da-c08d60508f2b None admin] Validating token access rules against request {{(pid=80378) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:01:50.343140 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG dbcounter [-] [80378] Writing DB stats keystone:SELECT=100 {{(pid=80378) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 10:01:50.355850 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93940f43-58e1-4b91-8399-c97f93567eed None admin] Validating token access rules against request {{(pid=80377) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:01:50.368293 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-121992e7-e545-4781-8b6e-46e87e8a74f1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '891005fa5ff74041b75747d16cc66798', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80378) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:01:50.369385 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-121992e7-e545-4781-8b6e-46e87e8a74f1 None admin] REQUEST_METHOD: `GET` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:01:50.369917 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-121992e7-e545-4781-8b6e-46e87e8a74f1 None admin] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:01:50.370867 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-121992e7-e545-4781-8b6e-46e87e8a74f1 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:01:50.371796 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-121992e7-e545-4781-8b6e-46e87e8a74f1 None admin] RBAC: Adding query filter params (group.id=7a7f06a893c2484aafc43a46704b6e2a, role.id=649993d933d44c50b268df7fdf336fbf, scope.project.id=c1809c9e380b45ec803130fe38eeaf24) {{(pid=80378) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 02 10:01:50.372280 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-121992e7-e545-4781-8b6e-46e87e8a74f1 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:01:50.374673 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-121992e7-e545-4781-8b6e-46e87e8a74f1 None admin] RBAC: Authorization granted {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:01:50.397825 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 183/371] 10.222.0.82 () {56 vars in 1376 bytes} [Tue Aug 2 10:01:50 2022] GET /identity/v3/role_assignments?group.id=7a7f06a893c2484aafc43a46704b6e2a&scope.project.id=c1809c9e380b45ec803130fe38eeaf24&role.id=649993d933d44c50b268df7fdf336fbf => generated 260 bytes in 282 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 02 10:01:50.397825 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-121992e7-e545-4781-8b6e-46e87e8a74f1 None admin] Authenticating user token {{(pid=80378) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:01:50.424218 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ed3c9cf-13a8-444d-941f-aeeb0fbe482f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '891005fa5ff74041b75747d16cc66798', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80377) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:01:50.425308 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ed3c9cf-13a8-444d-941f-aeeb0fbe482f None admin] REQUEST_METHOD: `GET` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:01:50.425821 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ed3c9cf-13a8-444d-941f-aeeb0fbe482f None admin] SCRIPT_NAME: `/identity` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:01:50.426313 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ed3c9cf-13a8-444d-941f-aeeb0fbe482f None admin] PATH_INFO: `/v3/roles` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:01:50.427563 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ed3c9cf-13a8-444d-941f-aeeb0fbe482f None admin] RBAC: Adding query filter params (name=admin) {{(pid=80377) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 02 10:01:50.428086 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ed3c9cf-13a8-444d-941f-aeeb0fbe482f None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:01:50.430686 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ed3c9cf-13a8-444d-941f-aeeb0fbe482f None admin] RBAC: Authorization granted {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:01:50.447689 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: [pid: 80377|app: 0|req: 189/372] 10.222.0.82 () {56 vars in 1099 bytes} [Tue Aug 2 10:01:50 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 292 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 02 10:01:50.449864 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ed3c9cf-13a8-444d-941f-aeeb0fbe482f None admin] Authenticating user token {{(pid=80377) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:01:50.561520 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG dbcounter [-] [80377] Writing DB stats keystone:SELECT=100 {{(pid=80377) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 10:01:50.598328 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-121992e7-e545-4781-8b6e-46e87e8a74f1 None admin] Validating token access rules against request {{(pid=80378) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:01:50.641212 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ed3c9cf-13a8-444d-941f-aeeb0fbe482f None admin] Validating token access rules against request {{(pid=80377) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:01:50.647633 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39ba32f4-7204-485a-94d8-ef70ffa6033e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '891005fa5ff74041b75747d16cc66798', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80378) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:01:50.648485 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39ba32f4-7204-485a-94d8-ef70ffa6033e None admin] REQUEST_METHOD: `GET` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:01:50.648771 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39ba32f4-7204-485a-94d8-ef70ffa6033e None admin] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:01:50.649046 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39ba32f4-7204-485a-94d8-ef70ffa6033e None admin] PATH_INFO: `/v3/groups/7a7f06a893c2484aafc43a46704b6e2a` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:01:50.650558 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39ba32f4-7204-485a-94d8-ef70ffa6033e None admin] RBAC: Authorizing `identity:get_group(group_id=7a7f06a893c2484aafc43a46704b6e2a)` {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:01:50.652112 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39ba32f4-7204-485a-94d8-ef70ffa6033e None admin] RBAC: Authorization granted {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:01:50.653957 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 184/373] 10.222.0.82 () {56 vars in 1180 bytes} [Tue Aug 2 10:01:50 2022] GET /identity/v3/groups/7a7f06a893c2484aafc43a46704b6e2a => generated 226 bytes in 266 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 02 10:01:50.654898 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39ba32f4-7204-485a-94d8-ef70ffa6033e None admin] Authenticating user token {{(pid=80378) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:01:50.712501 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-add8bef6-9206-4abc-8267-526e4c9b9fd1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '891005fa5ff74041b75747d16cc66798', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80377) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:01:50.713170 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-add8bef6-9206-4abc-8267-526e4c9b9fd1 None admin] REQUEST_METHOD: `GET` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:01:50.713452 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-add8bef6-9206-4abc-8267-526e4c9b9fd1 None admin] SCRIPT_NAME: `/identity` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:01:50.713725 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-add8bef6-9206-4abc-8267-526e4c9b9fd1 None admin] PATH_INFO: `/v3/groups/7a7f06a893c2484aafc43a46704b6e2a` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:01:50.715489 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-add8bef6-9206-4abc-8267-526e4c9b9fd1 None admin] RBAC: Authorizing `identity:get_group(group_id=7a7f06a893c2484aafc43a46704b6e2a)` {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:01:50.717965 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-add8bef6-9206-4abc-8267-526e4c9b9fd1 None admin] RBAC: Authorization granted {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:01:50.720430 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: [pid: 80377|app: 0|req: 190/374] 10.222.0.82 () {56 vars in 1180 bytes} [Tue Aug 2 10:01:50 2022] GET /identity/v3/groups/7a7f06a893c2484aafc43a46704b6e2a => generated 226 bytes in 272 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 02 10:01:50.723681 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-648da564-da16-4015-a62a-1582cf851ecd None None] REQUEST_METHOD: `GET` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:01:50.724094 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-648da564-da16-4015-a62a-1582cf851ecd None None] SCRIPT_NAME: `/identity` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:01:50.725073 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-648da564-da16-4015-a62a-1582cf851ecd None None] PATH_INFO: `/` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:01:50.726855 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: [pid: 80377|app: 0|req: 191/375] 10.222.0.82 () {54 vars in 926 bytes} [Tue Aug 2 10:01:50 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 02 10:01:50.728229 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-648da564-da16-4015-a62a-1582cf851ecd None None] Authenticating user token {{(pid=80377) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:01:50.747135 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG dbcounter [-] [80378] Writing DB stats keystone:SELECT=100 {{(pid=80378) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 10:01:50.808021 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39ba32f4-7204-485a-94d8-ef70ffa6033e None admin] Validating token access rules against request {{(pid=80378) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:01:50.861210 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0c5448f-522f-44ed-9775-641f4b0910fe None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '891005fa5ff74041b75747d16cc66798', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80378) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:01:50.862296 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0c5448f-522f-44ed-9775-641f4b0910fe None admin] REQUEST_METHOD: `GET` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:01:50.862296 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0c5448f-522f-44ed-9775-641f4b0910fe None admin] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:01:50.862619 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0c5448f-522f-44ed-9775-641f4b0910fe None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:01:50.895128 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-648da564-da16-4015-a62a-1582cf851ecd None None] Validating token access rules against request {{(pid=80377) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:01:50.959117 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG dbcounter [-] [80377] Writing DB stats keystone:SELECT=100 {{(pid=80377) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 10:01:50.965472 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b94daeb2-fc64-4e13-9946-0560de3bd6b1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '891005fa5ff74041b75747d16cc66798', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80377) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:01:50.966158 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b94daeb2-fc64-4e13-9946-0560de3bd6b1 None admin] REQUEST_METHOD: `GET` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:01:50.966535 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b94daeb2-fc64-4e13-9946-0560de3bd6b1 None admin] SCRIPT_NAME: `/identity` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:01:50.967110 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b94daeb2-fc64-4e13-9946-0560de3bd6b1 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:01:50.968421 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b94daeb2-fc64-4e13-9946-0560de3bd6b1 None admin] RBAC: Adding query filter params (group.id=7a7f06a893c2484aafc43a46704b6e2a, role.id=649993d933d44c50b268df7fdf336fbf, scope.project.id=f7db7cf528494fbb8836c08d79b752ef) {{(pid=80377) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 02 10:01:50.968929 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b94daeb2-fc64-4e13-9946-0560de3bd6b1 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:01:50.971753 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b94daeb2-fc64-4e13-9946-0560de3bd6b1 None admin] RBAC: Authorization granted {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:01:50.981402 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: [pid: 80377|app: 0|req: 192/376] 10.222.0.82 () {56 vars in 1376 bytes} [Tue Aug 2 10:01:50 2022] GET /identity/v3/role_assignments?group.id=7a7f06a893c2484aafc43a46704b6e2a&scope.project.id=f7db7cf528494fbb8836c08d79b752ef&role.id=649993d933d44c50b268df7fdf336fbf => generated 260 bytes in 254 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 02 10:01:50.983191 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de30585d-1f8d-4c39-80fb-6248562738e5 None None] REQUEST_METHOD: `POST` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:01:50.983572 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de30585d-1f8d-4c39-80fb-6248562738e5 None None] SCRIPT_NAME: `/identity` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:01:50.983943 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de30585d-1f8d-4c39-80fb-6248562738e5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:01:51.004753 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0c5448f-522f-44ed-9775-641f4b0910fe None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:01:51.006660 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0c5448f-522f-44ed-9775-641f4b0910fe None admin] RBAC: Authorization granted {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:01:51.009967 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.auth.core [None req-de30585d-1f8d-4c39-80fb-6248562738e5 None None] MFA Rules not processed for user `891005fa5ff74041b75747d16cc66798`. Rule list: `[]` (Enabled: `True`). {{(pid=80377) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 02 10:01:51.011258 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.fernet_utils [None req-de30585d-1f8d-4c39-80fb-6248562738e5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80377) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 10:01:51.121674 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG dbcounter [-] [80378] Writing DB stats keystone:SELECT=100 {{(pid=80378) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 10:01:51.173832 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: [pid: 80377|app: 0|req: 193/377] 10.222.0.82 () {58 vars in 1023 bytes} [Tue Aug 2 10:01:50 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 187 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 02 10:01:51.173832 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de30585d-1f8d-4c39-80fb-6248562738e5 None None] Authenticating user token {{(pid=80377) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:01:51.247412 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 185/378] 10.222.0.82 () {58 vars in 1282 bytes} [Tue Aug 2 10:01:50 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 593 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 02 10:01:51.249382 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0c5448f-522f-44ed-9775-641f4b0910fe None admin] Authenticating user token {{(pid=80378) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:01:51.359784 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de30585d-1f8d-4c39-80fb-6248562738e5 None None] Validating token access rules against request {{(pid=80377) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:01:51.400858 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG dbcounter [-] [80377] Writing DB stats keystone:SELECT=100 {{(pid=80377) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 10:01:51.419038 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08bc74a4-da57-49a0-ac96-313791aeb8c2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '891005fa5ff74041b75747d16cc66798', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80377) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:01:51.420078 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08bc74a4-da57-49a0-ac96-313791aeb8c2 None admin] REQUEST_METHOD: `GET` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:01:51.420506 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08bc74a4-da57-49a0-ac96-313791aeb8c2 None admin] SCRIPT_NAME: `/identity` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:01:51.420950 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08bc74a4-da57-49a0-ac96-313791aeb8c2 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:01:51.422301 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08bc74a4-da57-49a0-ac96-313791aeb8c2 None admin] RBAC: Adding query filter params (group.id=7a7f06a893c2484aafc43a46704b6e2a, role.id=e1c04b19fc4543b8b54f58bbcffbc598, scope.project.id=f7db7cf528494fbb8836c08d79b752ef) {{(pid=80377) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 02 10:01:51.422618 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08bc74a4-da57-49a0-ac96-313791aeb8c2 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:01:51.425011 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08bc74a4-da57-49a0-ac96-313791aeb8c2 None admin] RBAC: Authorization granted {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:01:51.431739 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0c5448f-522f-44ed-9775-641f4b0910fe None admin] Validating token access rules against request {{(pid=80378) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:01:51.439557 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: [pid: 80377|app: 0|req: 194/379] 10.222.0.82 () {56 vars in 1376 bytes} [Tue Aug 2 10:01:51 2022] GET /identity/v3/role_assignments?group.id=7a7f06a893c2484aafc43a46704b6e2a&scope.project.id=f7db7cf528494fbb8836c08d79b752ef&role.id=e1c04b19fc4543b8b54f58bbcffbc598 => generated 260 bytes in 263 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 02 10:01:51.439557 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08bc74a4-da57-49a0-ac96-313791aeb8c2 None admin] Authenticating user token {{(pid=80377) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:01:51.547411 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6b8dc62-0fa0-41e3-967e-e0e3ecd93d5b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '891005fa5ff74041b75747d16cc66798', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80378) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:01:51.551915 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6b8dc62-0fa0-41e3-967e-e0e3ecd93d5b None admin] REQUEST_METHOD: `GET` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:01:51.552370 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6b8dc62-0fa0-41e3-967e-e0e3ecd93d5b None admin] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:01:51.552811 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6b8dc62-0fa0-41e3-967e-e0e3ecd93d5b None admin] PATH_INFO: `/v3/roles/member` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:01:51.595366 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6b8dc62-0fa0-41e3-967e-e0e3ecd93d5b None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 02 10:01:51.603056 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application [None req-c6b8dc62-0fa0-41e3-967e-e0e3ecd93d5b None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Aug 02 10:01:51.603056 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 02 10:01:51.603056 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 02 10:01:51.603056 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 02 10:01:51.603056 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 02 10:01:51.603056 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 02 10:01:51.603056 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 02 10:01:51.603056 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 02 10:01:51.603056 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 02 10:01:51.603056 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 02 10:01:51.603056 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 02 10:01:51.603056 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 02 10:01:51.603056 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 02 10:01:51.603056 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 02 10:01:51.603056 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 02 10:01:51.603056 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 02 10:01:51.604578 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 02 10:01:51.604578 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 02 10:01:51.604578 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 02 10:01:51.604578 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 02 10:01:51.604578 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 02 10:01:51.604578 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 02 10:01:51.604578 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 02 10:01:51.604578 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 02 10:01:51.604578 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 02 10:01:51.604578 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application return self.get_or_create( Aug 02 10:01:51.604578 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 02 10:01:51.604578 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application with Lock( Aug 02 10:01:51.604578 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 02 10:01:51.604578 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application return self._enter() Aug 02 10:01:51.604578 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 02 10:01:51.604578 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 02 10:01:51.605571 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 02 10:01:51.605571 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application return self.creator() Aug 02 10:01:51.605571 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 02 10:01:51.605571 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application created_value = creator( Aug 02 10:01:51.605571 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 02 10:01:51.605571 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 02 10:01:51.605571 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 02 10:01:51.605571 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 02 10:01:51.605571 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 02 10:01:51.605571 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 02 10:01:51.605571 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Aug 02 10:01:51.605571 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application Aug 02 10:01:51.605571 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 186/380] 10.222.0.82 () {56 vars in 1099 bytes} [Tue Aug 2 10:01:51 2022] GET /identity/v3/roles/member => generated 84 bytes in 357 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 02 10:01:51.635308 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6b8dc62-0fa0-41e3-967e-e0e3ecd93d5b None admin] Authenticating user token {{(pid=80378) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:01:51.700087 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08bc74a4-da57-49a0-ac96-313791aeb8c2 None admin] Validating token access rules against request {{(pid=80377) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:01:51.700478 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG dbcounter [-] [80378] Writing DB stats keystone:SELECT=100 {{(pid=80378) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 10:01:51.754399 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-751b11ac-717a-479b-aa55-4271a9614536 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '891005fa5ff74041b75747d16cc66798', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80377) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:01:51.755077 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-751b11ac-717a-479b-aa55-4271a9614536 None admin] REQUEST_METHOD: `GET` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:01:51.755352 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-751b11ac-717a-479b-aa55-4271a9614536 None admin] SCRIPT_NAME: `/identity` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:01:51.755622 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-751b11ac-717a-479b-aa55-4271a9614536 None admin] PATH_INFO: `/v3/projects/7047912c5acd4adeba5d87c3bf895e39` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:01:51.758498 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-751b11ac-717a-479b-aa55-4271a9614536 None admin] RBAC: Authorizing `identity:get_project(project_id=7047912c5acd4adeba5d87c3bf895e39)` {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:01:51.760043 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-751b11ac-717a-479b-aa55-4271a9614536 None admin] RBAC: Authorization granted {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:01:51.763469 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: [pid: 80377|app: 0|req: 195/381] 10.222.0.82 () {56 vars in 1186 bytes} [Tue Aug 2 10:01:51 2022] GET /identity/v3/projects/7047912c5acd4adeba5d87c3bf895e39 => generated 344 bytes in 330 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 02 10:01:51.772488 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-751b11ac-717a-479b-aa55-4271a9614536 None admin] Authenticating user token {{(pid=80377) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:01:51.884976 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG dbcounter [-] [80377] Writing DB stats keystone:SELECT=100 {{(pid=80377) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 10:01:51.887037 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6b8dc62-0fa0-41e3-967e-e0e3ecd93d5b None admin] Validating token access rules against request {{(pid=80378) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:01:51.967710 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-751b11ac-717a-479b-aa55-4271a9614536 None admin] Validating token access rules against request {{(pid=80377) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:01:52.017092 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3c436dd-9ad6-4a53-9e74-c97f9fb22c09 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '891005fa5ff74041b75747d16cc66798', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80378) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:01:52.018045 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3c436dd-9ad6-4a53-9e74-c97f9fb22c09 None admin] REQUEST_METHOD: `GET` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:01:52.018330 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3c436dd-9ad6-4a53-9e74-c97f9fb22c09 None admin] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:01:52.018682 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3c436dd-9ad6-4a53-9e74-c97f9fb22c09 None admin] PATH_INFO: `/v3/roles` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:01:52.019973 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3c436dd-9ad6-4a53-9e74-c97f9fb22c09 None admin] RBAC: Adding query filter params (name=member) {{(pid=80378) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 02 10:01:52.020853 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3c436dd-9ad6-4a53-9e74-c97f9fb22c09 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:01:52.023242 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3c436dd-9ad6-4a53-9e74-c97f9fb22c09 None admin] RBAC: Authorization granted {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:01:52.051644 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 187/382] 10.222.0.82 () {56 vars in 1101 bytes} [Tue Aug 2 10:01:51 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 417 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 02 10:01:52.052641 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96c895d8-5a67-414d-8714-8d1def8518b0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '891005fa5ff74041b75747d16cc66798', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80377) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:01:52.053387 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96c895d8-5a67-414d-8714-8d1def8518b0 None admin] REQUEST_METHOD: `GET` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:01:52.053707 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96c895d8-5a67-414d-8714-8d1def8518b0 None admin] SCRIPT_NAME: `/identity` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:01:52.054118 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96c895d8-5a67-414d-8714-8d1def8518b0 None admin] PATH_INFO: `/v3/groups/948c83fe907c4d3b8071d26809f7d765` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:01:52.057905 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3c436dd-9ad6-4a53-9e74-c97f9fb22c09 None admin] Authenticating user token {{(pid=80378) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:01:52.067239 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96c895d8-5a67-414d-8714-8d1def8518b0 None admin] RBAC: Authorizing `identity:get_group(group_id=948c83fe907c4d3b8071d26809f7d765)` {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:01:52.068977 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96c895d8-5a67-414d-8714-8d1def8518b0 None admin] RBAC: Authorization granted {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:01:52.071099 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: [pid: 80377|app: 0|req: 196/383] 10.222.0.82 () {56 vars in 1180 bytes} [Tue Aug 2 10:01:51 2022] GET /identity/v3/groups/948c83fe907c4d3b8071d26809f7d765 => generated 229 bytes in 300 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 02 10:01:52.081872 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96c895d8-5a67-414d-8714-8d1def8518b0 None admin] Authenticating user token {{(pid=80377) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:01:52.246822 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96c895d8-5a67-414d-8714-8d1def8518b0 None admin] Validating token access rules against request {{(pid=80377) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:01:52.271968 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG dbcounter [-] [80378] Writing DB stats keystone:SELECT=100 {{(pid=80378) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 10:01:52.296468 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG dbcounter [-] [80377] Writing DB stats keystone:SELECT=100 {{(pid=80377) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 10:01:52.321117 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6f7dcc9-18ad-48ef-a6ae-925694bd24a2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '891005fa5ff74041b75747d16cc66798', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80377) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:01:52.321756 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6f7dcc9-18ad-48ef-a6ae-925694bd24a2 None admin] REQUEST_METHOD: `GET` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:01:52.322366 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6f7dcc9-18ad-48ef-a6ae-925694bd24a2 None admin] SCRIPT_NAME: `/identity` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:01:52.322645 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6f7dcc9-18ad-48ef-a6ae-925694bd24a2 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:01:52.323461 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6f7dcc9-18ad-48ef-a6ae-925694bd24a2 None admin] RBAC: Adding query filter params (group.id=948c83fe907c4d3b8071d26809f7d765, role.id=237966442f05411191105bf336bc14f3, scope.project.id=7047912c5acd4adeba5d87c3bf895e39) {{(pid=80377) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 02 10:01:52.323771 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6f7dcc9-18ad-48ef-a6ae-925694bd24a2 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:01:52.325279 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6f7dcc9-18ad-48ef-a6ae-925694bd24a2 None admin] RBAC: Authorization granted {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:01:52.330056 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3c436dd-9ad6-4a53-9e74-c97f9fb22c09 None admin] Validating token access rules against request {{(pid=80378) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:01:52.344462 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: [pid: 80377|app: 0|req: 197/384] 10.222.0.82 () {56 vars in 1376 bytes} [Tue Aug 2 10:01:52 2022] GET /identity/v3/role_assignments?group.id=948c83fe907c4d3b8071d26809f7d765&scope.project.id=7047912c5acd4adeba5d87c3bf895e39&role.id=237966442f05411191105bf336bc14f3 => generated 260 bytes in 263 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 02 10:01:52.414270 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c38da538-e7a7-4783-be0c-ddd90e7e7eea None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '891005fa5ff74041b75747d16cc66798', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80378) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:01:52.414945 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c38da538-e7a7-4783-be0c-ddd90e7e7eea None admin] REQUEST_METHOD: `GET` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:01:52.415229 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c38da538-e7a7-4783-be0c-ddd90e7e7eea None admin] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:01:52.415504 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c38da538-e7a7-4783-be0c-ddd90e7e7eea None admin] PATH_INFO: `/v3/groups/7a7f06a893c2484aafc43a46704b6e2a` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:01:52.417065 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c38da538-e7a7-4783-be0c-ddd90e7e7eea None admin] RBAC: Authorizing `identity:get_group(group_id=7a7f06a893c2484aafc43a46704b6e2a)` {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:01:52.418740 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c38da538-e7a7-4783-be0c-ddd90e7e7eea None admin] RBAC: Authorization granted {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:01:52.421064 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 188/385] 10.222.0.82 () {56 vars in 1180 bytes} [Tue Aug 2 10:01:52 2022] GET /identity/v3/groups/7a7f06a893c2484aafc43a46704b6e2a => generated 226 bytes in 363 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 02 10:01:52.429358 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6f7dcc9-18ad-48ef-a6ae-925694bd24a2 None admin] Authenticating user token {{(pid=80377) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:01:52.641589 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6f7dcc9-18ad-48ef-a6ae-925694bd24a2 None admin] Validating token access rules against request {{(pid=80377) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:01:52.699151 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24c6a2a8-0e19-487f-b383-508edde66695 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '891005fa5ff74041b75747d16cc66798', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80377) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:01:52.700179 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24c6a2a8-0e19-487f-b383-508edde66695 None admin] REQUEST_METHOD: `GET` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:01:52.700631 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24c6a2a8-0e19-487f-b383-508edde66695 None admin] SCRIPT_NAME: `/identity` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:01:52.701072 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24c6a2a8-0e19-487f-b383-508edde66695 None admin] PATH_INFO: `/v3/projects/c1809c9e380b45ec803130fe38eeaf24` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:01:52.703083 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24c6a2a8-0e19-487f-b383-508edde66695 None admin] RBAC: Authorizing `identity:get_project(project_id=c1809c9e380b45ec803130fe38eeaf24)` {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:01:52.705610 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24c6a2a8-0e19-487f-b383-508edde66695 None admin] RBAC: Authorization granted {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:01:52.708910 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: [pid: 80377|app: 0|req: 198/386] 10.222.0.82 () {56 vars in 1186 bytes} [Tue Aug 2 10:01:52 2022] GET /identity/v3/projects/c1809c9e380b45ec803130fe38eeaf24 => generated 298 bytes in 281 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 02 10:01:52.716436 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c38da538-e7a7-4783-be0c-ddd90e7e7eea None admin] Authenticating user token {{(pid=80378) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:01:52.922554 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c38da538-e7a7-4783-be0c-ddd90e7e7eea None admin] Validating token access rules against request {{(pid=80378) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:01:53.010018 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98441678-b822-4ec4-979d-57067cd3f2ce None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '891005fa5ff74041b75747d16cc66798', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80378) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:01:53.011132 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98441678-b822-4ec4-979d-57067cd3f2ce None admin] REQUEST_METHOD: `PUT` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:01:53.011595 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98441678-b822-4ec4-979d-57067cd3f2ce None admin] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:01:53.012055 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98441678-b822-4ec4-979d-57067cd3f2ce None admin] PATH_INFO: `/v3/projects/c1809c9e380b45ec803130fe38eeaf24/groups/7a7f06a893c2484aafc43a46704b6e2a/roles/e1c04b19fc4543b8b54f58bbcffbc598` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:01:53.015574 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98441678-b822-4ec4-979d-57067cd3f2ce None admin] RBAC: Authorizing `identity:create_grant(project_id=c1809c9e380b45ec803130fe38eeaf24, group_id=7a7f06a893c2484aafc43a46704b6e2a, role_id=e1c04b19fc4543b8b54f58bbcffbc598)` {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:01:53.018259 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98441678-b822-4ec4-979d-57067cd3f2ce None admin] RBAC: Authorization granted {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:01:53.042958 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 189/387] 10.222.0.82 () {58 vars in 1442 bytes} [Tue Aug 2 10:01:52 2022] PUT /identity/v3/projects/c1809c9e380b45ec803130fe38eeaf24/groups/7a7f06a893c2484aafc43a46704b6e2a/roles/e1c04b19fc4543b8b54f58bbcffbc598 => generated 0 bytes in 327 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 02 10:01:53.894784 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-328bb3f1-a91e-4a6f-baf3-a1c848751c7c None None] REQUEST_METHOD: `GET` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:01:53.895586 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-328bb3f1-a91e-4a6f-baf3-a1c848751c7c None None] SCRIPT_NAME: `/identity` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:01:53.895586 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-328bb3f1-a91e-4a6f-baf3-a1c848751c7c None None] PATH_INFO: `/` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:01:53.898419 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: [pid: 80377|app: 0|req: 199/388] 10.222.0.82 () {54 vars in 926 bytes} [Tue Aug 2 10:01:53 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 02 10:01:53.908058 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1221c03-c8f1-46c1-a8ab-ae1277084290 None None] REQUEST_METHOD: `POST` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:01:53.908827 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1221c03-c8f1-46c1-a8ab-ae1277084290 None None] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:01:53.909162 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1221c03-c8f1-46c1-a8ab-ae1277084290 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:01:53.921567 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG dbcounter [-] [80378] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80378) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 10:01:53.941248 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.auth.core [None req-a1221c03-c8f1-46c1-a8ab-ae1277084290 None None] MFA Rules not processed for user `891005fa5ff74041b75747d16cc66798`. Rule list: `[]` (Enabled: `True`). {{(pid=80378) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 02 10:01:53.943463 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.fernet_utils [None req-a1221c03-c8f1-46c1-a8ab-ae1277084290 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80378) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 10:01:54.077481 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 190/389] 10.222.0.82 () {58 vars in 1023 bytes} [Tue Aug 2 10:01:53 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 172 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Aug 02 10:01:54.114105 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-328bb3f1-a91e-4a6f-baf3-a1c848751c7c None None] Authenticating user token {{(pid=80377) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:01:54.194893 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG dbcounter [-] [80377] Writing DB stats keystone:SELECT=100 {{(pid=80377) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 10:01:54.252575 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-328bb3f1-a91e-4a6f-baf3-a1c848751c7c None None] Validating token access rules against request {{(pid=80377) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:01:54.298298 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c82c40d7-a046-427d-b921-43027373710c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '891005fa5ff74041b75747d16cc66798', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80377) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:01:54.299495 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c82c40d7-a046-427d-b921-43027373710c None admin] REQUEST_METHOD: `GET` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:01:54.299992 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c82c40d7-a046-427d-b921-43027373710c None admin] SCRIPT_NAME: `/identity` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:01:54.300313 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c82c40d7-a046-427d-b921-43027373710c None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:01:54.314896 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c82c40d7-a046-427d-b921-43027373710c None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 02 10:01:54.320628 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application [None req-c82c40d7-a046-427d-b921-43027373710c None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Aug 02 10:01:54.320628 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 02 10:01:54.320628 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 02 10:01:54.320628 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 02 10:01:54.320628 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 02 10:01:54.320628 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 02 10:01:54.320628 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 02 10:01:54.320628 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 02 10:01:54.320628 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 02 10:01:54.320628 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 02 10:01:54.320628 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 02 10:01:54.320628 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 02 10:01:54.320628 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 02 10:01:54.320628 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 02 10:01:54.320628 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 02 10:01:54.320628 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 02 10:01:54.322758 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 02 10:01:54.322758 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 02 10:01:54.322758 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 02 10:01:54.322758 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 02 10:01:54.322758 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 02 10:01:54.322758 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 02 10:01:54.322758 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 02 10:01:54.322758 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 02 10:01:54.322758 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 02 10:01:54.322758 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application return self.get_or_create( Aug 02 10:01:54.322758 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 02 10:01:54.322758 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application with Lock( Aug 02 10:01:54.322758 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 02 10:01:54.322758 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application return self._enter() Aug 02 10:01:54.322758 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 02 10:01:54.322758 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 02 10:01:54.324792 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 02 10:01:54.324792 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application return self.creator() Aug 02 10:01:54.324792 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 02 10:01:54.324792 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application created_value = creator( Aug 02 10:01:54.324792 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 02 10:01:54.324792 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 02 10:01:54.324792 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 02 10:01:54.324792 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 02 10:01:54.324792 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 02 10:01:54.324792 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 02 10:01:54.324792 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Aug 02 10:01:54.324792 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application Aug 02 10:01:54.324792 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: [pid: 80377|app: 0|req: 200/390] 10.222.0.82 () {56 vars in 1114 bytes} [Tue Aug 2 10:01:54 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 209 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 02 10:01:54.331251 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1221c03-c8f1-46c1-a8ab-ae1277084290 None None] Authenticating user token {{(pid=80378) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:01:54.532646 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1221c03-c8f1-46c1-a8ab-ae1277084290 None None] Validating token access rules against request {{(pid=80378) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:01:54.580074 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f795d6df-5725-4e18-8a7b-92285d72a2dc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '891005fa5ff74041b75747d16cc66798', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80378) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:01:54.580999 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f795d6df-5725-4e18-8a7b-92285d72a2dc None admin] REQUEST_METHOD: `GET` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:01:54.581283 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f795d6df-5725-4e18-8a7b-92285d72a2dc None admin] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:01:54.581683 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f795d6df-5725-4e18-8a7b-92285d72a2dc None admin] PATH_INFO: `/v3/roles` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:01:54.582613 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f795d6df-5725-4e18-8a7b-92285d72a2dc None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=80378) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 02 10:01:54.583121 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f795d6df-5725-4e18-8a7b-92285d72a2dc None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:01:54.585109 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f795d6df-5725-4e18-8a7b-92285d72a2dc None admin] RBAC: Authorization granted {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:01:54.587749 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG dbcounter [-] [80378] Writing DB stats keystone:SELECT=100 {{(pid=80378) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 10:01:54.596950 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 191/391] 10.222.0.82 () {56 vars in 1111 bytes} [Tue Aug 2 10:01:54 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 266 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 02 10:01:54.606914 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c82c40d7-a046-427d-b921-43027373710c None admin] Authenticating user token {{(pid=80377) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:01:54.729517 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0c54969-3561-46fb-ba00-732953de8f36 None None] REQUEST_METHOD: `GET` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:01:54.730008 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0c54969-3561-46fb-ba00-732953de8f36 None None] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:01:54.730494 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0c54969-3561-46fb-ba00-732953de8f36 None None] PATH_INFO: `/` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:01:54.732963 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 192/392] 10.222.0.82 () {54 vars in 926 bytes} [Tue Aug 2 10:01:54 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 02 10:01:54.741950 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a871bcb-9e84-4e57-b879-6d2b95977f60 None None] REQUEST_METHOD: `POST` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:01:54.742420 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a871bcb-9e84-4e57-b879-6d2b95977f60 None None] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:01:54.742901 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a871bcb-9e84-4e57-b879-6d2b95977f60 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:01:54.781116 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.auth.core [None req-0a871bcb-9e84-4e57-b879-6d2b95977f60 None None] MFA Rules not processed for user `891005fa5ff74041b75747d16cc66798`. Rule list: `[]` (Enabled: `True`). {{(pid=80378) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 02 10:01:54.784156 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.fernet_utils [None req-0a871bcb-9e84-4e57-b879-6d2b95977f60 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80378) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 10:01:54.793366 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c82c40d7-a046-427d-b921-43027373710c None admin] Validating token access rules against request {{(pid=80377) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:01:54.836306 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG dbcounter [-] [80377] Writing DB stats keystone:SELECT=100 {{(pid=80377) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 10:01:54.885580 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 193/393] 10.222.0.82 () {58 vars in 1023 bytes} [Tue Aug 2 10:01:54 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 145 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 02 10:01:54.909236 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2460369-f37d-43c6-b80e-4fbf43659d36 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '891005fa5ff74041b75747d16cc66798', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80377) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:01:54.910445 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2460369-f37d-43c6-b80e-4fbf43659d36 None admin] REQUEST_METHOD: `GET` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:01:54.910946 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2460369-f37d-43c6-b80e-4fbf43659d36 None admin] SCRIPT_NAME: `/identity` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:01:54.911447 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2460369-f37d-43c6-b80e-4fbf43659d36 None admin] PATH_INFO: `/v3/groups/7a7f06a893c2484aafc43a46704b6e2a` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:01:54.913564 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2460369-f37d-43c6-b80e-4fbf43659d36 None admin] RBAC: Authorizing `identity:get_group(group_id=7a7f06a893c2484aafc43a46704b6e2a)` {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:01:54.916479 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2460369-f37d-43c6-b80e-4fbf43659d36 None admin] RBAC: Authorization granted {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:01:54.920037 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: [pid: 80377|app: 0|req: 201/394] 10.222.0.82 () {56 vars in 1180 bytes} [Tue Aug 2 10:01:54 2022] GET /identity/v3/groups/7a7f06a893c2484aafc43a46704b6e2a => generated 226 bytes in 313 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 02 10:01:54.929000 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2460369-f37d-43c6-b80e-4fbf43659d36 None admin] Authenticating user token {{(pid=80377) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:01:54.929296 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a871bcb-9e84-4e57-b879-6d2b95977f60 None None] Authenticating user token {{(pid=80378) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:01:55.075177 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a871bcb-9e84-4e57-b879-6d2b95977f60 None None] Validating token access rules against request {{(pid=80378) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:01:55.133886 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG dbcounter [-] [80378] Writing DB stats keystone:SELECT=100 {{(pid=80378) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 10:01:55.157400 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c2df291-1fa9-4787-a74e-f75d030fe70f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '891005fa5ff74041b75747d16cc66798', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80378) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:01:55.159114 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c2df291-1fa9-4787-a74e-f75d030fe70f None admin] REQUEST_METHOD: `GET` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:01:55.159791 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c2df291-1fa9-4787-a74e-f75d030fe70f None admin] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:01:55.160237 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c2df291-1fa9-4787-a74e-f75d030fe70f None admin] PATH_INFO: `/v3/projects/c1809c9e380b45ec803130fe38eeaf24` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:01:55.162276 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c2df291-1fa9-4787-a74e-f75d030fe70f None admin] RBAC: Authorizing `identity:get_project(project_id=c1809c9e380b45ec803130fe38eeaf24)` {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:01:55.165736 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c2df291-1fa9-4787-a74e-f75d030fe70f None admin] RBAC: Authorization granted {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:01:55.168697 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 194/395] 10.222.0.82 () {56 vars in 1186 bytes} [Tue Aug 2 10:01:54 2022] GET /identity/v3/projects/c1809c9e380b45ec803130fe38eeaf24 => generated 298 bytes in 241 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 02 10:01:55.171219 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3efa800-a3e0-4e86-92e4-49761245c4cc None None] REQUEST_METHOD: `GET` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:01:55.171808 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3efa800-a3e0-4e86-92e4-49761245c4cc None None] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:01:55.172283 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3efa800-a3e0-4e86-92e4-49761245c4cc None None] PATH_INFO: `/` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:01:55.174599 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 195/396] 10.222.0.82 () {54 vars in 926 bytes} [Tue Aug 2 10:01:55 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 02 10:01:55.185382 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd18b06d-0a67-41ea-9694-85972d27b387 None None] REQUEST_METHOD: `POST` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:01:55.185382 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd18b06d-0a67-41ea-9694-85972d27b387 None None] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:01:55.185382 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd18b06d-0a67-41ea-9694-85972d27b387 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:01:55.187145 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2460369-f37d-43c6-b80e-4fbf43659d36 None admin] Validating token access rules against request {{(pid=80377) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:01:55.206956 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.auth.core [None req-dd18b06d-0a67-41ea-9694-85972d27b387 None None] MFA Rules not processed for user `891005fa5ff74041b75747d16cc66798`. Rule list: `[]` (Enabled: `True`). {{(pid=80378) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 02 10:01:55.208259 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.fernet_utils [None req-dd18b06d-0a67-41ea-9694-85972d27b387 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80378) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 10:01:55.241141 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed5dc0d0-24f5-4474-95aa-12f0620fe874 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '891005fa5ff74041b75747d16cc66798', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80377) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:01:55.241992 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed5dc0d0-24f5-4474-95aa-12f0620fe874 None admin] REQUEST_METHOD: `GET` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:01:55.242401 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed5dc0d0-24f5-4474-95aa-12f0620fe874 None admin] SCRIPT_NAME: `/identity` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:01:55.242688 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed5dc0d0-24f5-4474-95aa-12f0620fe874 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:01:55.258108 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed5dc0d0-24f5-4474-95aa-12f0620fe874 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 02 10:01:55.267779 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application [None req-ed5dc0d0-24f5-4474-95aa-12f0620fe874 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Aug 02 10:01:55.267779 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 02 10:01:55.267779 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 02 10:01:55.267779 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 02 10:01:55.267779 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 02 10:01:55.267779 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 02 10:01:55.267779 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 02 10:01:55.267779 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 02 10:01:55.267779 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 02 10:01:55.267779 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 02 10:01:55.267779 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 02 10:01:55.267779 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 02 10:01:55.267779 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 02 10:01:55.267779 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 02 10:01:55.267779 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 02 10:01:55.267779 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 02 10:01:55.269143 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 02 10:01:55.269143 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 02 10:01:55.269143 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 02 10:01:55.269143 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 02 10:01:55.269143 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 02 10:01:55.269143 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 02 10:01:55.269143 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 02 10:01:55.269143 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 02 10:01:55.269143 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 02 10:01:55.269143 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application return self.get_or_create( Aug 02 10:01:55.269143 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 02 10:01:55.269143 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application with Lock( Aug 02 10:01:55.269143 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 02 10:01:55.269143 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application return self._enter() Aug 02 10:01:55.269143 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 02 10:01:55.269143 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 02 10:01:55.270478 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 02 10:01:55.270478 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application return self.creator() Aug 02 10:01:55.270478 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 02 10:01:55.270478 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application created_value = creator( Aug 02 10:01:55.270478 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 02 10:01:55.270478 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 02 10:01:55.270478 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 02 10:01:55.270478 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 02 10:01:55.270478 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 02 10:01:55.270478 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 02 10:01:55.270478 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Aug 02 10:01:55.270478 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application Aug 02 10:01:55.270478 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: [pid: 80377|app: 0|req: 202/397] 10.222.0.82 () {56 vars in 1114 bytes} [Tue Aug 2 10:01:54 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 341 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 02 10:01:55.272195 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed5dc0d0-24f5-4474-95aa-12f0620fe874 None admin] Authenticating user token {{(pid=80377) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:01:55.301962 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 196/398] 10.222.0.82 () {58 vars in 1023 bytes} [Tue Aug 2 10:01:55 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 113 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Aug 02 10:01:55.303507 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd18b06d-0a67-41ea-9694-85972d27b387 None None] Authenticating user token {{(pid=80378) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:01:55.332425 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG dbcounter [-] [80377] Writing DB stats keystone:SELECT=100 {{(pid=80377) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 10:01:55.445670 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed5dc0d0-24f5-4474-95aa-12f0620fe874 None admin] Validating token access rules against request {{(pid=80377) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:01:55.465632 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd18b06d-0a67-41ea-9694-85972d27b387 None None] Validating token access rules against request {{(pid=80378) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:01:55.493845 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG dbcounter [-] [80378] Writing DB stats keystone:SELECT=100 {{(pid=80378) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 10:01:55.520165 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4447c75-6dac-42c3-82db-2480e652b76f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '891005fa5ff74041b75747d16cc66798', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80378) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:01:55.520900 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4447c75-6dac-42c3-82db-2480e652b76f None admin] REQUEST_METHOD: `GET` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:01:55.521131 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4447c75-6dac-42c3-82db-2480e652b76f None admin] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:01:55.521653 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4447c75-6dac-42c3-82db-2480e652b76f None admin] PATH_INFO: `/v3/roles` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:01:55.522267 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4447c75-6dac-42c3-82db-2480e652b76f None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=80378) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 02 10:01:55.522456 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4447c75-6dac-42c3-82db-2480e652b76f None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:01:55.523863 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4447c75-6dac-42c3-82db-2480e652b76f None admin] RBAC: Authorization granted {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:01:55.526984 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e18a00d-73c7-4c0c-8107-ef39a9f9808b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '891005fa5ff74041b75747d16cc66798', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80377) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:01:55.527678 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e18a00d-73c7-4c0c-8107-ef39a9f9808b None admin] REQUEST_METHOD: `PUT` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:01:55.527957 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e18a00d-73c7-4c0c-8107-ef39a9f9808b None admin] SCRIPT_NAME: `/identity` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:01:55.528239 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e18a00d-73c7-4c0c-8107-ef39a9f9808b None admin] PATH_INFO: `/v3/projects/c1809c9e380b45ec803130fe38eeaf24/groups/7a7f06a893c2484aafc43a46704b6e2a/roles/649993d933d44c50b268df7fdf336fbf` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:01:55.534476 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e18a00d-73c7-4c0c-8107-ef39a9f9808b None admin] RBAC: Authorizing `identity:create_grant(project_id=c1809c9e380b45ec803130fe38eeaf24, group_id=7a7f06a893c2484aafc43a46704b6e2a, role_id=649993d933d44c50b268df7fdf336fbf)` {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:01:55.539881 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 197/399] 10.222.0.82 () {56 vars in 1111 bytes} [Tue Aug 2 10:01:55 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 237 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 02 10:01:55.540385 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e18a00d-73c7-4c0c-8107-ef39a9f9808b None admin] RBAC: Authorization granted {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:01:55.540532 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4447c75-6dac-42c3-82db-2480e652b76f None admin] Authenticating user token {{(pid=80378) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:01:55.567705 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: [pid: 80377|app: 0|req: 203/400] 10.222.0.82 () {58 vars in 1442 bytes} [Tue Aug 2 10:01:55 2022] PUT /identity/v3/projects/c1809c9e380b45ec803130fe38eeaf24/groups/7a7f06a893c2484aafc43a46704b6e2a/roles/649993d933d44c50b268df7fdf336fbf => generated 0 bytes in 297 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 02 10:01:55.569460 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e18a00d-73c7-4c0c-8107-ef39a9f9808b None admin] Authenticating user token {{(pid=80377) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:01:55.769246 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4447c75-6dac-42c3-82db-2480e652b76f None admin] Validating token access rules against request {{(pid=80378) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:01:55.809372 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e18a00d-73c7-4c0c-8107-ef39a9f9808b None admin] Validating token access rules against request {{(pid=80377) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:01:55.812254 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG dbcounter [-] [80377] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80377) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 10:01:55.832386 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6127dbf-9ab7-4e54-b032-3243b9a2124e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '891005fa5ff74041b75747d16cc66798', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80378) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:01:55.833444 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6127dbf-9ab7-4e54-b032-3243b9a2124e None admin] REQUEST_METHOD: `GET` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:01:55.833988 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6127dbf-9ab7-4e54-b032-3243b9a2124e None admin] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:01:55.834462 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6127dbf-9ab7-4e54-b032-3243b9a2124e None admin] PATH_INFO: `/v3/roles/member` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:01:55.849738 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6127dbf-9ab7-4e54-b032-3243b9a2124e None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 02 10:01:55.858113 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application [None req-f6127dbf-9ab7-4e54-b032-3243b9a2124e None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Aug 02 10:01:55.858113 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 02 10:01:55.858113 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 02 10:01:55.858113 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 02 10:01:55.858113 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 02 10:01:55.858113 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 02 10:01:55.858113 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 02 10:01:55.858113 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 02 10:01:55.858113 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 02 10:01:55.858113 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 02 10:01:55.858113 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 02 10:01:55.858113 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 02 10:01:55.858113 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 02 10:01:55.858113 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 02 10:01:55.858113 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 02 10:01:55.858113 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 02 10:01:55.859075 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 02 10:01:55.859075 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 02 10:01:55.859075 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 02 10:01:55.859075 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 02 10:01:55.859075 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 02 10:01:55.859075 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 02 10:01:55.859075 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 02 10:01:55.859075 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 02 10:01:55.859075 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 02 10:01:55.859075 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application return self.get_or_create( Aug 02 10:01:55.859075 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 02 10:01:55.859075 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application with Lock( Aug 02 10:01:55.859075 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 02 10:01:55.859075 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application return self._enter() Aug 02 10:01:55.859075 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 02 10:01:55.859075 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 02 10:01:55.859894 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 02 10:01:55.859894 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application return self.creator() Aug 02 10:01:55.859894 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 02 10:01:55.859894 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application created_value = creator( Aug 02 10:01:55.859894 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 02 10:01:55.859894 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 02 10:01:55.859894 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 02 10:01:55.859894 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 02 10:01:55.859894 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 02 10:01:55.859894 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 02 10:01:55.859894 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Aug 02 10:01:55.859894 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application Aug 02 10:01:55.862885 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 198/401] 10.222.0.82 () {56 vars in 1099 bytes} [Tue Aug 2 10:01:55 2022] GET /identity/v3/roles/member => generated 84 bytes in 323 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 02 10:01:55.872375 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6127dbf-9ab7-4e54-b032-3243b9a2124e None admin] Authenticating user token {{(pid=80378) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:01:55.882978 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afd0e60c-9511-44c4-98f9-ea0c9301178f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '891005fa5ff74041b75747d16cc66798', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80377) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:01:55.883681 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afd0e60c-9511-44c4-98f9-ea0c9301178f None admin] REQUEST_METHOD: `GET` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:01:55.883966 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afd0e60c-9511-44c4-98f9-ea0c9301178f None admin] SCRIPT_NAME: `/identity` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:01:55.884244 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afd0e60c-9511-44c4-98f9-ea0c9301178f None admin] PATH_INFO: `/v3/groups/7a7f06a893c2484aafc43a46704b6e2a` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:01:55.885463 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-afd0e60c-9511-44c4-98f9-ea0c9301178f None admin] RBAC: Authorizing `identity:get_group(group_id=7a7f06a893c2484aafc43a46704b6e2a)` {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:01:55.886975 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-afd0e60c-9511-44c4-98f9-ea0c9301178f None admin] RBAC: Authorization granted {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:01:55.889001 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: [pid: 80377|app: 0|req: 204/402] 10.222.0.82 () {56 vars in 1180 bytes} [Tue Aug 2 10:01:55 2022] GET /identity/v3/groups/7a7f06a893c2484aafc43a46704b6e2a => generated 226 bytes in 320 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 02 10:01:55.895092 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afd0e60c-9511-44c4-98f9-ea0c9301178f None admin] Authenticating user token {{(pid=80377) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:01:55.976970 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG dbcounter [-] [80378] Writing DB stats keystone:SELECT=100 {{(pid=80378) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 10:01:56.058472 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6127dbf-9ab7-4e54-b032-3243b9a2124e None admin] Validating token access rules against request {{(pid=80378) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:01:56.060561 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afd0e60c-9511-44c4-98f9-ea0c9301178f None admin] Validating token access rules against request {{(pid=80377) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:01:56.112161 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09fdeabb-fea2-4f0f-955b-6979813d814e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '891005fa5ff74041b75747d16cc66798', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80378) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:01:56.112848 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09fdeabb-fea2-4f0f-955b-6979813d814e None admin] REQUEST_METHOD: `GET` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:01:56.113668 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09fdeabb-fea2-4f0f-955b-6979813d814e None admin] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:01:56.113983 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09fdeabb-fea2-4f0f-955b-6979813d814e None admin] PATH_INFO: `/v3/roles` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:01:56.115052 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09fdeabb-fea2-4f0f-955b-6979813d814e None admin] RBAC: Adding query filter params (name=member) {{(pid=80378) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 02 10:01:56.115368 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09fdeabb-fea2-4f0f-955b-6979813d814e None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:01:56.117074 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09fdeabb-fea2-4f0f-955b-6979813d814e None admin] RBAC: Authorization granted {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:01:56.126629 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a7a1f7a-c0f6-4432-b43a-c50902b224da None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '891005fa5ff74041b75747d16cc66798', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80377) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:01:56.127688 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a7a1f7a-c0f6-4432-b43a-c50902b224da None admin] REQUEST_METHOD: `GET` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:01:56.128196 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 199/403] 10.222.0.82 () {56 vars in 1101 bytes} [Tue Aug 2 10:01:55 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 257 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 02 10:01:56.128322 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a7a1f7a-c0f6-4432-b43a-c50902b224da None admin] SCRIPT_NAME: `/identity` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:01:56.128707 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a7a1f7a-c0f6-4432-b43a-c50902b224da None admin] PATH_INFO: `/v3/projects/f7db7cf528494fbb8836c08d79b752ef` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:01:56.129183 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a76218e8-d61f-47ee-9636-f3f8c52eead1 None None] REQUEST_METHOD: `GET` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:01:56.129476 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a76218e8-d61f-47ee-9636-f3f8c52eead1 None None] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:01:56.129837 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a76218e8-d61f-47ee-9636-f3f8c52eead1 None None] PATH_INFO: `/` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:01:56.131118 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a7a1f7a-c0f6-4432-b43a-c50902b224da None admin] RBAC: Authorizing `identity:get_project(project_id=f7db7cf528494fbb8836c08d79b752ef)` {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:01:56.133842 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a7a1f7a-c0f6-4432-b43a-c50902b224da None admin] RBAC: Authorization granted {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:01:56.136296 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 200/404] 10.222.0.82 () {54 vars in 926 bytes} [Tue Aug 2 10:01:56 2022] GET /identity => generated 270 bytes in 8 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 02 10:01:56.137012 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a76218e8-d61f-47ee-9636-f3f8c52eead1 None None] Authenticating user token {{(pid=80378) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:01:56.141225 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: [pid: 80377|app: 0|req: 205/405] 10.222.0.82 () {56 vars in 1186 bytes} [Tue Aug 2 10:01:55 2022] GET /identity/v3/projects/f7db7cf528494fbb8836c08d79b752ef => generated 302 bytes in 243 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 02 10:01:56.144303 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d42381f4-8f4f-4a5d-9459-0b2749370533 None None] REQUEST_METHOD: `POST` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:01:56.144794 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d42381f4-8f4f-4a5d-9459-0b2749370533 None None] SCRIPT_NAME: `/identity` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:01:56.145278 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d42381f4-8f4f-4a5d-9459-0b2749370533 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:01:56.182302 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.auth.core [None req-d42381f4-8f4f-4a5d-9459-0b2749370533 None None] MFA Rules not processed for user `891005fa5ff74041b75747d16cc66798`. Rule list: `[]` (Enabled: `True`). {{(pid=80377) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 02 10:01:56.184458 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.fernet_utils [None req-d42381f4-8f4f-4a5d-9459-0b2749370533 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80377) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 10:01:56.240466 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG dbcounter [-] [80377] Writing DB stats keystone:SELECT=100 {{(pid=80377) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 10:01:56.291702 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a76218e8-d61f-47ee-9636-f3f8c52eead1 None None] Validating token access rules against request {{(pid=80378) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:01:56.306700 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG dbcounter [-] [80378] Writing DB stats keystone:SELECT=100 {{(pid=80378) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 10:01:56.320544 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: [pid: 80377|app: 0|req: 206/406] 10.222.0.82 () {58 vars in 1023 bytes} [Tue Aug 2 10:01:56 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 178 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 02 10:01:56.321324 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d42381f4-8f4f-4a5d-9459-0b2749370533 None None] Authenticating user token {{(pid=80377) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:01:56.368498 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19f5eee6-b492-4894-b969-93a85cfe91ff None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '891005fa5ff74041b75747d16cc66798', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80378) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:01:56.369351 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19f5eee6-b492-4894-b969-93a85cfe91ff None admin] REQUEST_METHOD: `GET` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:01:56.369731 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19f5eee6-b492-4894-b969-93a85cfe91ff None admin] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:01:56.370231 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19f5eee6-b492-4894-b969-93a85cfe91ff None admin] PATH_INFO: `/v3/groups/7a7f06a893c2484aafc43a46704b6e2a` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:01:56.371522 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19f5eee6-b492-4894-b969-93a85cfe91ff None admin] RBAC: Authorizing `identity:get_group(group_id=7a7f06a893c2484aafc43a46704b6e2a)` {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:01:56.373874 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19f5eee6-b492-4894-b969-93a85cfe91ff None admin] RBAC: Authorization granted {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:01:56.376876 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 201/407] 10.222.0.82 () {56 vars in 1180 bytes} [Tue Aug 2 10:01:56 2022] GET /identity/v3/groups/7a7f06a893c2484aafc43a46704b6e2a => generated 226 bytes in 240 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 02 10:01:56.379033 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19f5eee6-b492-4894-b969-93a85cfe91ff None admin] Authenticating user token {{(pid=80378) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:01:56.513181 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d42381f4-8f4f-4a5d-9459-0b2749370533 None None] Validating token access rules against request {{(pid=80377) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:01:56.565528 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19f5eee6-b492-4894-b969-93a85cfe91ff None admin] Validating token access rules against request {{(pid=80378) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:01:56.583035 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dccf4f5f-544f-4773-b49c-7f6907c7f582 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '891005fa5ff74041b75747d16cc66798', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80377) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:01:56.584056 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dccf4f5f-544f-4773-b49c-7f6907c7f582 None admin] REQUEST_METHOD: `PUT` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:01:56.584521 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dccf4f5f-544f-4773-b49c-7f6907c7f582 None admin] SCRIPT_NAME: `/identity` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:01:56.584974 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dccf4f5f-544f-4773-b49c-7f6907c7f582 None admin] PATH_INFO: `/v3/projects/f7db7cf528494fbb8836c08d79b752ef/groups/7a7f06a893c2484aafc43a46704b6e2a/roles/649993d933d44c50b268df7fdf336fbf` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:01:56.588598 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dccf4f5f-544f-4773-b49c-7f6907c7f582 None admin] RBAC: Authorizing `identity:create_grant(project_id=f7db7cf528494fbb8836c08d79b752ef, group_id=7a7f06a893c2484aafc43a46704b6e2a, role_id=649993d933d44c50b268df7fdf336fbf)` {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:01:56.591158 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dccf4f5f-544f-4773-b49c-7f6907c7f582 None admin] RBAC: Authorization granted {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:01:56.613683 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7441a06c-86a3-47ca-ac1f-4ff196348717 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '891005fa5ff74041b75747d16cc66798', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80378) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:01:56.614405 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7441a06c-86a3-47ca-ac1f-4ff196348717 None admin] REQUEST_METHOD: `GET` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:01:56.614686 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7441a06c-86a3-47ca-ac1f-4ff196348717 None admin] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:01:56.614962 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7441a06c-86a3-47ca-ac1f-4ff196348717 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:01:56.622029 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: [pid: 80377|app: 0|req: 207/408] 10.222.0.82 () {58 vars in 1442 bytes} [Tue Aug 2 10:01:56 2022] PUT /identity/v3/projects/f7db7cf528494fbb8836c08d79b752ef/groups/7a7f06a893c2484aafc43a46704b6e2a/roles/649993d933d44c50b268df7fdf336fbf => generated 0 bytes in 295 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 02 10:01:56.622029 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dccf4f5f-544f-4773-b49c-7f6907c7f582 None admin] Authenticating user token {{(pid=80377) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:01:56.638990 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7441a06c-86a3-47ca-ac1f-4ff196348717 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 02 10:01:56.649451 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application [None req-7441a06c-86a3-47ca-ac1f-4ff196348717 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 02 10:01:56.649451 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 02 10:01:56.649451 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 02 10:01:56.649451 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 02 10:01:56.649451 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 02 10:01:56.649451 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 02 10:01:56.649451 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 02 10:01:56.649451 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 02 10:01:56.649451 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 02 10:01:56.649451 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 02 10:01:56.649451 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 02 10:01:56.649451 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 02 10:01:56.649451 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 02 10:01:56.649451 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 02 10:01:56.649451 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 02 10:01:56.649451 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 02 10:01:56.650505 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 02 10:01:56.650505 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 02 10:01:56.650505 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 02 10:01:56.650505 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 02 10:01:56.650505 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 02 10:01:56.650505 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 02 10:01:56.650505 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 02 10:01:56.650505 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 02 10:01:56.650505 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 02 10:01:56.650505 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application return self.get_or_create( Aug 02 10:01:56.650505 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 02 10:01:56.650505 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application with Lock( Aug 02 10:01:56.650505 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 02 10:01:56.650505 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application return self._enter() Aug 02 10:01:56.650505 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 02 10:01:56.650505 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 02 10:01:56.651431 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 02 10:01:56.651431 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application return self.creator() Aug 02 10:01:56.651431 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 02 10:01:56.651431 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application created_value = creator( Aug 02 10:01:56.651431 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 02 10:01:56.651431 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 02 10:01:56.651431 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 02 10:01:56.651431 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 02 10:01:56.651431 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 02 10:01:56.651431 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 02 10:01:56.651431 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 02 10:01:56.651431 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application Aug 02 10:01:56.651431 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 202/409] 10.222.0.82 () {56 vars in 1096 bytes} [Tue Aug 2 10:01:56 2022] GET /identity/v3/roles/admin => generated 83 bytes in 274 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 02 10:01:56.653815 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f6242a6-7044-49da-85d2-cdc51971c716 None None] REQUEST_METHOD: `GET` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:01:56.653815 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f6242a6-7044-49da-85d2-cdc51971c716 None None] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:01:56.654353 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f6242a6-7044-49da-85d2-cdc51971c716 None None] PATH_INFO: `/` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:01:56.656599 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 203/410] 10.222.0.82 () {54 vars in 926 bytes} [Tue Aug 2 10:01:56 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 02 10:01:56.661182 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f6242a6-7044-49da-85d2-cdc51971c716 None None] Authenticating user token {{(pid=80378) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:01:56.691089 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG dbcounter [-] [80377] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80377) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 10:01:56.737847 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG dbcounter [-] [80378] Writing DB stats keystone:SELECT=100 {{(pid=80378) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 10:01:56.841829 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dccf4f5f-544f-4773-b49c-7f6907c7f582 None admin] Validating token access rules against request {{(pid=80377) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:01:56.884325 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f6242a6-7044-49da-85d2-cdc51971c716 None None] Validating token access rules against request {{(pid=80378) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:01:56.920297 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-187b8c4b-a18a-4d08-82f2-c93d41f8668c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '891005fa5ff74041b75747d16cc66798', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80377) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:01:56.921365 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-187b8c4b-a18a-4d08-82f2-c93d41f8668c None admin] REQUEST_METHOD: `GET` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:01:56.922042 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-187b8c4b-a18a-4d08-82f2-c93d41f8668c None admin] SCRIPT_NAME: `/identity` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:01:56.922521 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-187b8c4b-a18a-4d08-82f2-c93d41f8668c None admin] PATH_INFO: `/v3/projects/f7db7cf528494fbb8836c08d79b752ef` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:01:56.925121 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-187b8c4b-a18a-4d08-82f2-c93d41f8668c None admin] RBAC: Authorizing `identity:get_project(project_id=f7db7cf528494fbb8836c08d79b752ef)` {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:01:56.927784 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-187b8c4b-a18a-4d08-82f2-c93d41f8668c None admin] RBAC: Authorization granted {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:01:56.931577 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: [pid: 80377|app: 0|req: 208/411] 10.222.0.82 () {56 vars in 1186 bytes} [Tue Aug 2 10:01:56 2022] GET /identity/v3/projects/f7db7cf528494fbb8836c08d79b752ef => generated 302 bytes in 315 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 02 10:01:56.934251 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d8c4fad-8210-4594-8ee7-927ef6518a1b None None] REQUEST_METHOD: `POST` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:01:56.934720 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d8c4fad-8210-4594-8ee7-927ef6518a1b None None] SCRIPT_NAME: `/identity` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:01:56.935197 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d8c4fad-8210-4594-8ee7-927ef6518a1b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:01:56.960884 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c02c18f7-e96c-4896-bdc8-d163a044d4c7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '891005fa5ff74041b75747d16cc66798', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80378) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:01:56.961954 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c02c18f7-e96c-4896-bdc8-d163a044d4c7 None admin] REQUEST_METHOD: `GET` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:01:56.962448 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c02c18f7-e96c-4896-bdc8-d163a044d4c7 None admin] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:01:56.962969 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c02c18f7-e96c-4896-bdc8-d163a044d4c7 None admin] PATH_INFO: `/v3/roles` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:01:56.964470 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c02c18f7-e96c-4896-bdc8-d163a044d4c7 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80378) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 02 10:01:56.964995 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c02c18f7-e96c-4896-bdc8-d163a044d4c7 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:01:56.967726 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c02c18f7-e96c-4896-bdc8-d163a044d4c7 None admin] RBAC: Authorization granted {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:01:56.974583 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.auth.core [None req-2d8c4fad-8210-4594-8ee7-927ef6518a1b None None] MFA Rules not processed for user `891005fa5ff74041b75747d16cc66798`. Rule list: `[]` (Enabled: `True`). {{(pid=80377) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 02 10:01:56.977039 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.fernet_utils [None req-2d8c4fad-8210-4594-8ee7-927ef6518a1b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80377) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 10:01:56.983579 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 204/412] 10.222.0.82 () {56 vars in 1099 bytes} [Tue Aug 2 10:01:56 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 322 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 02 10:01:56.984725 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c02c18f7-e96c-4896-bdc8-d163a044d4c7 None admin] Authenticating user token {{(pid=80378) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:01:57.110283 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: [pid: 80377|app: 0|req: 209/413] 10.222.0.82 () {58 vars in 1023 bytes} [Tue Aug 2 10:01:56 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 174 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 02 10:01:57.112804 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d8c4fad-8210-4594-8ee7-927ef6518a1b None None] Authenticating user token {{(pid=80377) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:01:57.187093 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG dbcounter [-] [80378] Writing DB stats keystone:SELECT=100 {{(pid=80378) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 10:01:57.193451 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG dbcounter [-] [80377] Writing DB stats keystone:SELECT=100 {{(pid=80377) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 10:01:57.195997 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c02c18f7-e96c-4896-bdc8-d163a044d4c7 None admin] Validating token access rules against request {{(pid=80378) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:01:57.247609 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81c50b08-dc05-4ff0-9ac7-47afc82c7bf5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '891005fa5ff74041b75747d16cc66798', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80378) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:01:57.248587 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81c50b08-dc05-4ff0-9ac7-47afc82c7bf5 None admin] REQUEST_METHOD: `PUT` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:01:57.248936 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81c50b08-dc05-4ff0-9ac7-47afc82c7bf5 None admin] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:01:57.249209 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81c50b08-dc05-4ff0-9ac7-47afc82c7bf5 None admin] PATH_INFO: `/v3/projects/f7db7cf528494fbb8836c08d79b752ef/groups/7a7f06a893c2484aafc43a46704b6e2a/roles/e1c04b19fc4543b8b54f58bbcffbc598` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:01:57.251840 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81c50b08-dc05-4ff0-9ac7-47afc82c7bf5 None admin] RBAC: Authorizing `identity:create_grant(project_id=f7db7cf528494fbb8836c08d79b752ef, group_id=7a7f06a893c2484aafc43a46704b6e2a, role_id=e1c04b19fc4543b8b54f58bbcffbc598)` {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:01:57.253955 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81c50b08-dc05-4ff0-9ac7-47afc82c7bf5 None admin] RBAC: Authorization granted {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:01:57.276902 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 205/414] 10.222.0.82 () {58 vars in 1442 bytes} [Tue Aug 2 10:01:56 2022] PUT /identity/v3/projects/f7db7cf528494fbb8836c08d79b752ef/groups/7a7f06a893c2484aafc43a46704b6e2a/roles/e1c04b19fc4543b8b54f58bbcffbc598 => generated 0 bytes in 292 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 02 10:01:57.276902 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81c50b08-dc05-4ff0-9ac7-47afc82c7bf5 None admin] Authenticating user token {{(pid=80378) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:01:57.330079 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d8c4fad-8210-4594-8ee7-927ef6518a1b None None] Validating token access rules against request {{(pid=80377) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:01:57.390091 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b094de98-4faf-4f3c-8e9b-b41f13e5be20 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '891005fa5ff74041b75747d16cc66798', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80377) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:01:57.391837 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b094de98-4faf-4f3c-8e9b-b41f13e5be20 None admin] REQUEST_METHOD: `GET` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:01:57.392684 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b094de98-4faf-4f3c-8e9b-b41f13e5be20 None admin] SCRIPT_NAME: `/identity` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:01:57.393268 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b094de98-4faf-4f3c-8e9b-b41f13e5be20 None admin] PATH_INFO: `/v3/groups/948c83fe907c4d3b8071d26809f7d765` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:01:57.394956 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b094de98-4faf-4f3c-8e9b-b41f13e5be20 None admin] RBAC: Authorizing `identity:get_group(group_id=948c83fe907c4d3b8071d26809f7d765)` {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:01:57.396984 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b094de98-4faf-4f3c-8e9b-b41f13e5be20 None admin] RBAC: Authorization granted {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:01:57.400187 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: [pid: 80377|app: 0|req: 210/415] 10.222.0.82 () {56 vars in 1180 bytes} [Tue Aug 2 10:01:57 2022] GET /identity/v3/groups/948c83fe907c4d3b8071d26809f7d765 => generated 229 bytes in 288 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 02 10:01:57.405867 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81c50b08-dc05-4ff0-9ac7-47afc82c7bf5 None admin] Validating token access rules against request {{(pid=80378) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:01:57.406055 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b094de98-4faf-4f3c-8e9b-b41f13e5be20 None admin] Authenticating user token {{(pid=80377) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:01:57.533402 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58f0d125-3b8a-453d-b934-c487d133b3a2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '891005fa5ff74041b75747d16cc66798', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80378) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:01:57.534207 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58f0d125-3b8a-453d-b934-c487d133b3a2 None admin] REQUEST_METHOD: `GET` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:01:57.535503 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58f0d125-3b8a-453d-b934-c487d133b3a2 None admin] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:01:57.535503 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58f0d125-3b8a-453d-b934-c487d133b3a2 None admin] PATH_INFO: `/v3/roles/member` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:01:57.556233 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58f0d125-3b8a-453d-b934-c487d133b3a2 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 02 10:01:57.566766 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application [None req-58f0d125-3b8a-453d-b934-c487d133b3a2 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Aug 02 10:01:57.566766 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 02 10:01:57.566766 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 02 10:01:57.566766 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 02 10:01:57.566766 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 02 10:01:57.566766 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 02 10:01:57.566766 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 02 10:01:57.566766 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 02 10:01:57.566766 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 02 10:01:57.566766 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 02 10:01:57.566766 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 02 10:01:57.566766 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 02 10:01:57.566766 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 02 10:01:57.566766 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 02 10:01:57.566766 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 02 10:01:57.566766 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 02 10:01:57.567729 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 02 10:01:57.567729 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 02 10:01:57.567729 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 02 10:01:57.567729 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 02 10:01:57.567729 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 02 10:01:57.567729 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 02 10:01:57.567729 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 02 10:01:57.567729 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 02 10:01:57.567729 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 02 10:01:57.567729 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application return self.get_or_create( Aug 02 10:01:57.567729 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 02 10:01:57.567729 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application with Lock( Aug 02 10:01:57.567729 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 02 10:01:57.567729 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application return self._enter() Aug 02 10:01:57.567729 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 02 10:01:57.567729 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 02 10:01:57.568560 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 02 10:01:57.568560 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application return self.creator() Aug 02 10:01:57.568560 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 02 10:01:57.568560 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application created_value = creator( Aug 02 10:01:57.568560 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 02 10:01:57.568560 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 02 10:01:57.568560 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 02 10:01:57.568560 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 02 10:01:57.568560 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 02 10:01:57.568560 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 02 10:01:57.568560 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Aug 02 10:01:57.568560 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application Aug 02 10:01:57.571511 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 206/416] 10.222.0.82 () {56 vars in 1099 bytes} [Tue Aug 2 10:01:57 2022] GET /identity/v3/roles/member => generated 84 bytes in 295 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 02 10:01:57.581699 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG dbcounter [-] [80377] Writing DB stats keystone:SELECT=100 {{(pid=80377) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 10:01:57.587060 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58f0d125-3b8a-453d-b934-c487d133b3a2 None admin] Authenticating user token {{(pid=80378) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:01:57.596504 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b094de98-4faf-4f3c-8e9b-b41f13e5be20 None admin] Validating token access rules against request {{(pid=80377) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:01:57.638207 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG dbcounter [-] [80378] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80378) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 10:01:57.676549 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea647205-875b-4578-b00c-4551a73c0ff7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '891005fa5ff74041b75747d16cc66798', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80377) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:01:57.677459 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea647205-875b-4578-b00c-4551a73c0ff7 None admin] REQUEST_METHOD: `GET` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:01:57.677842 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea647205-875b-4578-b00c-4551a73c0ff7 None admin] SCRIPT_NAME: `/identity` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:01:57.678207 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea647205-875b-4578-b00c-4551a73c0ff7 None admin] PATH_INFO: `/v3/projects/7047912c5acd4adeba5d87c3bf895e39` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:01:57.681290 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea647205-875b-4578-b00c-4551a73c0ff7 None admin] RBAC: Authorizing `identity:get_project(project_id=7047912c5acd4adeba5d87c3bf895e39)` {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:01:57.685386 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea647205-875b-4578-b00c-4551a73c0ff7 None admin] RBAC: Authorization granted {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:01:57.688598 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: [pid: 80377|app: 0|req: 211/417] 10.222.0.82 () {56 vars in 1186 bytes} [Tue Aug 2 10:01:57 2022] GET /identity/v3/projects/7047912c5acd4adeba5d87c3bf895e39 => generated 344 bytes in 283 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 02 10:01:57.697833 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea647205-875b-4578-b00c-4551a73c0ff7 None admin] Authenticating user token {{(pid=80377) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:01:57.746869 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58f0d125-3b8a-453d-b934-c487d133b3a2 None admin] Validating token access rules against request {{(pid=80378) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:01:57.819059 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc26a7a4-c2ef-4777-88fd-7381be3b19a6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '891005fa5ff74041b75747d16cc66798', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80378) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:01:57.819706 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc26a7a4-c2ef-4777-88fd-7381be3b19a6 None admin] REQUEST_METHOD: `GET` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:01:57.819990 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc26a7a4-c2ef-4777-88fd-7381be3b19a6 None admin] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:01:57.820265 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc26a7a4-c2ef-4777-88fd-7381be3b19a6 None admin] PATH_INFO: `/v3/roles` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:01:57.821003 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc26a7a4-c2ef-4777-88fd-7381be3b19a6 None admin] RBAC: Adding query filter params (name=member) {{(pid=80378) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 02 10:01:57.821317 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc26a7a4-c2ef-4777-88fd-7381be3b19a6 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:01:57.822777 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc26a7a4-c2ef-4777-88fd-7381be3b19a6 None admin] RBAC: Authorization granted {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:01:57.843319 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 207/418] 10.222.0.82 () {56 vars in 1101 bytes} [Tue Aug 2 10:01:57 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 260 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 02 10:01:57.847574 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc26a7a4-c2ef-4777-88fd-7381be3b19a6 None admin] Authenticating user token {{(pid=80378) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:01:57.953967 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea647205-875b-4578-b00c-4551a73c0ff7 None admin] Validating token access rules against request {{(pid=80377) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:01:58.024027 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-992b6486-2572-45b2-9ce9-9f3f9a84c824 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '891005fa5ff74041b75747d16cc66798', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80377) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:01:58.025436 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-992b6486-2572-45b2-9ce9-9f3f9a84c824 None admin] REQUEST_METHOD: `PUT` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:01:58.025977 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-992b6486-2572-45b2-9ce9-9f3f9a84c824 None admin] SCRIPT_NAME: `/identity` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:01:58.026437 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-992b6486-2572-45b2-9ce9-9f3f9a84c824 None admin] PATH_INFO: `/v3/projects/7047912c5acd4adeba5d87c3bf895e39/groups/948c83fe907c4d3b8071d26809f7d765/roles/237966442f05411191105bf336bc14f3` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:01:58.031627 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-992b6486-2572-45b2-9ce9-9f3f9a84c824 None admin] RBAC: Authorizing `identity:create_grant(project_id=7047912c5acd4adeba5d87c3bf895e39, group_id=948c83fe907c4d3b8071d26809f7d765, role_id=237966442f05411191105bf336bc14f3)` {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:01:58.034738 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-992b6486-2572-45b2-9ce9-9f3f9a84c824 None admin] RBAC: Authorization granted {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:01:58.064548 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: [pid: 80377|app: 0|req: 212/419] 10.222.0.82 () {58 vars in 1442 bytes} [Tue Aug 2 10:01:57 2022] PUT /identity/v3/projects/7047912c5acd4adeba5d87c3bf895e39/groups/948c83fe907c4d3b8071d26809f7d765/roles/237966442f05411191105bf336bc14f3 => generated 0 bytes in 367 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 02 10:01:58.068571 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG dbcounter [-] [80378] Writing DB stats keystone:SELECT=100 {{(pid=80378) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 10:01:58.097326 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc26a7a4-c2ef-4777-88fd-7381be3b19a6 None admin] Validating token access rules against request {{(pid=80378) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:01:58.176221 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6cb4fe9d-a5a6-4f11-a7e0-557b51ebec72 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '891005fa5ff74041b75747d16cc66798', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80378) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:01:58.176887 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cb4fe9d-a5a6-4f11-a7e0-557b51ebec72 None admin] REQUEST_METHOD: `GET` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:01:58.177178 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cb4fe9d-a5a6-4f11-a7e0-557b51ebec72 None admin] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:01:58.177459 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cb4fe9d-a5a6-4f11-a7e0-557b51ebec72 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:01:58.327603 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6cb4fe9d-a5a6-4f11-a7e0-557b51ebec72 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:01:58.330789 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6cb4fe9d-a5a6-4f11-a7e0-557b51ebec72 None admin] RBAC: Authorization granted {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:01:58.517499 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG dbcounter [-] [80378] Writing DB stats keystone:SELECT=100 {{(pid=80378) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 10:01:58.558388 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 208/420] 10.222.0.82 () {58 vars in 1282 bytes} [Tue Aug 2 10:01:57 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 711 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 02 10:01:58.565618 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-992b6486-2572-45b2-9ce9-9f3f9a84c824 None admin] Authenticating user token {{(pid=80377) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:01:58.643417 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG dbcounter [-] [80377] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80377) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 10:01:58.827298 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-992b6486-2572-45b2-9ce9-9f3f9a84c824 None admin] Validating token access rules against request {{(pid=80377) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:01:58.923349 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5662bc67-c4e6-4699-b393-109403695b5e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '891005fa5ff74041b75747d16cc66798', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80377) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:01:58.925817 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40bd62d1-f942-4753-8fb7-dc4a7d8e31c0 None None] REQUEST_METHOD: `GET` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:01:58.926398 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40bd62d1-f942-4753-8fb7-dc4a7d8e31c0 None None] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:01:58.926879 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40bd62d1-f942-4753-8fb7-dc4a7d8e31c0 None None] PATH_INFO: `/` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:01:58.929828 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 209/421] 10.222.0.82 () {54 vars in 926 bytes} [Tue Aug 2 10:01:58 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 02 10:01:58.931165 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5662bc67-c4e6-4699-b393-109403695b5e None admin] REQUEST_METHOD: `GET` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:01:58.931635 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5662bc67-c4e6-4699-b393-109403695b5e None admin] SCRIPT_NAME: `/identity` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:01:58.932210 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5662bc67-c4e6-4699-b393-109403695b5e None admin] PATH_INFO: `/v3/projects/c1809c9e380b45ec803130fe38eeaf24` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:01:58.934464 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5662bc67-c4e6-4699-b393-109403695b5e None admin] RBAC: Authorizing `identity:get_project(project_id=c1809c9e380b45ec803130fe38eeaf24)` {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:01:58.937508 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5662bc67-c4e6-4699-b393-109403695b5e None admin] RBAC: Authorization granted {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:01:58.940368 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c62c505-882d-419e-afab-33e750f6fd53 None None] REQUEST_METHOD: `POST` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:01:58.941025 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c62c505-882d-419e-afab-33e750f6fd53 None None] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:01:58.941516 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c62c505-882d-419e-afab-33e750f6fd53 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:01:58.945597 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: [pid: 80377|app: 0|req: 213/422] 10.222.0.82 () {56 vars in 1186 bytes} [Tue Aug 2 10:01:58 2022] GET /identity/v3/projects/c1809c9e380b45ec803130fe38eeaf24 => generated 298 bytes in 381 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 02 10:01:58.954314 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5662bc67-c4e6-4699-b393-109403695b5e None admin] Authenticating user token {{(pid=80377) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:01:58.990844 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.auth.core [None req-6c62c505-882d-419e-afab-33e750f6fd53 None None] MFA Rules not processed for user `891005fa5ff74041b75747d16cc66798`. Rule list: `[]` (Enabled: `True`). {{(pid=80378) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 02 10:01:58.993117 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.fernet_utils [None req-6c62c505-882d-419e-afab-33e750f6fd53 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80378) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 10:01:59.198616 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 210/423] 10.222.0.82 () {58 vars in 1023 bytes} [Tue Aug 2 10:01:58 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 260 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Aug 02 10:01:59.223216 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG dbcounter [-] [80377] Writing DB stats keystone:SELECT=100 {{(pid=80377) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 10:01:59.248081 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5662bc67-c4e6-4699-b393-109403695b5e None admin] Validating token access rules against request {{(pid=80377) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:01:59.272997 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c62c505-882d-419e-afab-33e750f6fd53 None None] Authenticating user token {{(pid=80378) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:01:59.311935 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c94c4112-e7ba-4f50-bc1e-4e3817538c05 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '891005fa5ff74041b75747d16cc66798', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80377) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:01:59.313150 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c94c4112-e7ba-4f50-bc1e-4e3817538c05 None admin] REQUEST_METHOD: `GET` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:01:59.314286 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c94c4112-e7ba-4f50-bc1e-4e3817538c05 None admin] SCRIPT_NAME: `/identity` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:01:59.314770 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c94c4112-e7ba-4f50-bc1e-4e3817538c05 None admin] PATH_INFO: `/v3/groups/7a7f06a893c2484aafc43a46704b6e2a` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:01:59.318384 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c94c4112-e7ba-4f50-bc1e-4e3817538c05 None admin] RBAC: Authorizing `identity:get_group(group_id=7a7f06a893c2484aafc43a46704b6e2a)` {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:01:59.321117 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c94c4112-e7ba-4f50-bc1e-4e3817538c05 None admin] RBAC: Authorization granted {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:01:59.324614 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: [pid: 80377|app: 0|req: 214/424] 10.222.0.82 () {56 vars in 1180 bytes} [Tue Aug 2 10:01:58 2022] GET /identity/v3/groups/7a7f06a893c2484aafc43a46704b6e2a => generated 226 bytes in 371 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 02 10:01:59.334825 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c94c4112-e7ba-4f50-bc1e-4e3817538c05 None admin] Authenticating user token {{(pid=80377) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:01:59.542662 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c94c4112-e7ba-4f50-bc1e-4e3817538c05 None admin] Validating token access rules against request {{(pid=80377) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:01:59.612470 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4087bea7-d267-4728-b2ca-a68ce884d65e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '891005fa5ff74041b75747d16cc66798', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80377) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:01:59.613667 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4087bea7-d267-4728-b2ca-a68ce884d65e None admin] REQUEST_METHOD: `GET` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:01:59.617819 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4087bea7-d267-4728-b2ca-a68ce884d65e None admin] SCRIPT_NAME: `/identity` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:01:59.617819 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4087bea7-d267-4728-b2ca-a68ce884d65e None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:01:59.619249 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4087bea7-d267-4728-b2ca-a68ce884d65e None admin] RBAC: Adding query filter params (group.id=7a7f06a893c2484aafc43a46704b6e2a, role.id=e1c04b19fc4543b8b54f58bbcffbc598, scope.project.id=c1809c9e380b45ec803130fe38eeaf24) {{(pid=80377) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 02 10:01:59.619815 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4087bea7-d267-4728-b2ca-a68ce884d65e None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:01:59.622637 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4087bea7-d267-4728-b2ca-a68ce884d65e None admin] RBAC: Authorization granted {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:01:59.650127 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: [pid: 80377|app: 0|req: 215/425] 10.222.0.82 () {56 vars in 1376 bytes} [Tue Aug 2 10:01:59 2022] GET /identity/v3/role_assignments?group.id=7a7f06a893c2484aafc43a46704b6e2a&scope.project.id=c1809c9e380b45ec803130fe38eeaf24&role.id=e1c04b19fc4543b8b54f58bbcffbc598 => generated 611 bytes in 316 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 02 10:01:59.653225 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c62c505-882d-419e-afab-33e750f6fd53 None None] Validating token access rules against request {{(pid=80378) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:01:59.693152 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG dbcounter [-] [80378] Writing DB stats keystone:SELECT=100 {{(pid=80378) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 10:01:59.754953 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5fc00b9-ef66-4ad5-bf6b-42eb757670d5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '891005fa5ff74041b75747d16cc66798', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80378) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:01:59.756065 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5fc00b9-ef66-4ad5-bf6b-42eb757670d5 None admin] REQUEST_METHOD: `GET` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:01:59.756651 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5fc00b9-ef66-4ad5-bf6b-42eb757670d5 None admin] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:01:59.757144 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5fc00b9-ef66-4ad5-bf6b-42eb757670d5 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:01:59.790197 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5fc00b9-ef66-4ad5-bf6b-42eb757670d5 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 02 10:01:59.801245 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application [None req-c5fc00b9-ef66-4ad5-bf6b-42eb757670d5 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Aug 02 10:01:59.801245 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 02 10:01:59.801245 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 02 10:01:59.801245 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 02 10:01:59.801245 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 02 10:01:59.801245 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 02 10:01:59.801245 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 02 10:01:59.801245 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 02 10:01:59.801245 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 02 10:01:59.801245 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 02 10:01:59.801245 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 02 10:01:59.801245 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 02 10:01:59.801245 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 02 10:01:59.801245 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 02 10:01:59.801245 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 02 10:01:59.801245 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 02 10:01:59.802987 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 02 10:01:59.802987 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 02 10:01:59.802987 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 02 10:01:59.802987 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 02 10:01:59.802987 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 02 10:01:59.802987 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 02 10:01:59.802987 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 02 10:01:59.802987 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 02 10:01:59.802987 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 02 10:01:59.802987 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application return self.get_or_create( Aug 02 10:01:59.802987 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 02 10:01:59.802987 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application with Lock( Aug 02 10:01:59.802987 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 02 10:01:59.802987 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application return self._enter() Aug 02 10:01:59.802987 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 02 10:01:59.802987 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 02 10:01:59.804663 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 02 10:01:59.804663 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application return self.creator() Aug 02 10:01:59.804663 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 02 10:01:59.804663 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application created_value = creator( Aug 02 10:01:59.804663 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 02 10:01:59.804663 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 02 10:01:59.804663 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 02 10:01:59.804663 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 02 10:01:59.804663 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 02 10:01:59.804663 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 02 10:01:59.804663 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Aug 02 10:01:59.804663 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application Aug 02 10:01:59.813931 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 211/426] 10.222.0.82 () {56 vars in 1114 bytes} [Tue Aug 2 10:01:59 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 531 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 02 10:01:59.818143 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4087bea7-d267-4728-b2ca-a68ce884d65e None admin] Authenticating user token {{(pid=80377) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:01:59.913915 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG dbcounter [-] [80377] Writing DB stats keystone:SELECT=100 {{(pid=80377) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 10:02:00.074593 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec222ee9-7498-4b3d-9fef-7b1e784d2be9 None None] REQUEST_METHOD: `GET` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:02:00.075008 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec222ee9-7498-4b3d-9fef-7b1e784d2be9 None None] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:02:00.075392 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec222ee9-7498-4b3d-9fef-7b1e784d2be9 None None] PATH_INFO: `/` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:02:00.080093 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 212/427] 10.222.0.82 () {54 vars in 926 bytes} [Tue Aug 2 10:02:00 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 02 10:02:00.087793 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6975276-ba9e-4311-91b6-1ac108bc4def None None] REQUEST_METHOD: `POST` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:02:00.088249 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6975276-ba9e-4311-91b6-1ac108bc4def None None] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:02:00.088739 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6975276-ba9e-4311-91b6-1ac108bc4def None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:02:00.098844 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4087bea7-d267-4728-b2ca-a68ce884d65e None admin] Validating token access rules against request {{(pid=80377) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:02:00.137175 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.auth.core [None req-a6975276-ba9e-4311-91b6-1ac108bc4def None None] MFA Rules not processed for user `891005fa5ff74041b75747d16cc66798`. Rule list: `[]` (Enabled: `True`). {{(pid=80378) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 02 10:02:00.139298 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.fernet_utils [None req-a6975276-ba9e-4311-91b6-1ac108bc4def None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80378) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 10:02:00.189909 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14f24973-2baa-4f7a-b6b9-46953d68dccb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '891005fa5ff74041b75747d16cc66798', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80377) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:02:00.191172 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14f24973-2baa-4f7a-b6b9-46953d68dccb None admin] REQUEST_METHOD: `GET` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:02:00.191768 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14f24973-2baa-4f7a-b6b9-46953d68dccb None admin] SCRIPT_NAME: `/identity` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:02:00.192273 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14f24973-2baa-4f7a-b6b9-46953d68dccb None admin] PATH_INFO: `/v3/roles` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:02:00.193747 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14f24973-2baa-4f7a-b6b9-46953d68dccb None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=80377) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 02 10:02:00.194345 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14f24973-2baa-4f7a-b6b9-46953d68dccb None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:02:00.197368 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14f24973-2baa-4f7a-b6b9-46953d68dccb None admin] RBAC: Authorization granted {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:02:00.217058 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: [pid: 80377|app: 0|req: 216/428] 10.222.0.82 () {56 vars in 1111 bytes} [Tue Aug 2 10:01:59 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 400 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 02 10:02:00.226556 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14f24973-2baa-4f7a-b6b9-46953d68dccb None admin] Authenticating user token {{(pid=80377) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:02:00.293681 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 213/429] 10.222.0.82 () {58 vars in 1023 bytes} [Tue Aug 2 10:02:00 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 208 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 02 10:02:00.337557 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6975276-ba9e-4311-91b6-1ac108bc4def None None] Authenticating user token {{(pid=80378) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:02:00.420998 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG dbcounter [-] [80377] Writing DB stats keystone:SELECT=100 {{(pid=80377) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 10:02:00.456012 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14f24973-2baa-4f7a-b6b9-46953d68dccb None admin] Validating token access rules against request {{(pid=80377) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:02:00.552609 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35fb1824-489c-4132-956b-9e49e019b6a9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '891005fa5ff74041b75747d16cc66798', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80377) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:02:00.553639 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35fb1824-489c-4132-956b-9e49e019b6a9 None admin] REQUEST_METHOD: `GET` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:02:00.554179 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35fb1824-489c-4132-956b-9e49e019b6a9 None admin] SCRIPT_NAME: `/identity` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:02:00.554664 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35fb1824-489c-4132-956b-9e49e019b6a9 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:02:00.573288 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG dbcounter [-] [80378] Writing DB stats keystone:SELECT=100 {{(pid=80378) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 10:02:00.581678 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6975276-ba9e-4311-91b6-1ac108bc4def None None] Validating token access rules against request {{(pid=80378) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:02:00.645987 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05d86bbf-986b-4d1c-b80f-86a028e0e3f8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '891005fa5ff74041b75747d16cc66798', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80378) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:02:00.646562 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05d86bbf-986b-4d1c-b80f-86a028e0e3f8 None admin] REQUEST_METHOD: `GET` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:02:00.646954 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05d86bbf-986b-4d1c-b80f-86a028e0e3f8 None admin] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:02:00.647249 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05d86bbf-986b-4d1c-b80f-86a028e0e3f8 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:02:00.667428 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05d86bbf-986b-4d1c-b80f-86a028e0e3f8 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 02 10:02:00.678065 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application [None req-05d86bbf-986b-4d1c-b80f-86a028e0e3f8 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Aug 02 10:02:00.678065 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 02 10:02:00.678065 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 02 10:02:00.678065 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 02 10:02:00.678065 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 02 10:02:00.678065 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 02 10:02:00.678065 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 02 10:02:00.678065 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 02 10:02:00.678065 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 02 10:02:00.678065 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 02 10:02:00.678065 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 02 10:02:00.678065 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 02 10:02:00.678065 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 02 10:02:00.678065 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 02 10:02:00.678065 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 02 10:02:00.678065 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 02 10:02:00.679803 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 02 10:02:00.679803 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 02 10:02:00.679803 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 02 10:02:00.679803 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 02 10:02:00.679803 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 02 10:02:00.679803 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 02 10:02:00.679803 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 02 10:02:00.679803 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 02 10:02:00.679803 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 02 10:02:00.679803 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application return self.get_or_create( Aug 02 10:02:00.679803 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 02 10:02:00.679803 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application with Lock( Aug 02 10:02:00.679803 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 02 10:02:00.679803 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application return self._enter() Aug 02 10:02:00.679803 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 02 10:02:00.679803 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 02 10:02:00.682317 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 02 10:02:00.682317 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application return self.creator() Aug 02 10:02:00.682317 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 02 10:02:00.682317 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application created_value = creator( Aug 02 10:02:00.682317 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 02 10:02:00.682317 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 02 10:02:00.682317 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 02 10:02:00.682317 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 02 10:02:00.682317 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 02 10:02:00.682317 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 02 10:02:00.682317 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Aug 02 10:02:00.682317 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application Aug 02 10:02:00.682317 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 214/430] 10.222.0.82 () {56 vars in 1114 bytes} [Tue Aug 2 10:02:00 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 344 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 02 10:02:00.690235 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05d86bbf-986b-4d1c-b80f-86a028e0e3f8 None admin] Authenticating user token {{(pid=80378) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:02:00.704447 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35fb1824-489c-4132-956b-9e49e019b6a9 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:02:00.708195 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35fb1824-489c-4132-956b-9e49e019b6a9 None admin] RBAC: Authorization granted {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:02:00.845420 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05d86bbf-986b-4d1c-b80f-86a028e0e3f8 None admin] Validating token access rules against request {{(pid=80378) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:02:00.898554 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG dbcounter [-] [80377] Writing DB stats keystone:SELECT=100 {{(pid=80377) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 10:02:00.908629 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: [pid: 80377|app: 0|req: 217/431] 10.222.0.82 () {58 vars in 1282 bytes} [Tue Aug 2 10:02:00 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 683 msecs (HTTP/1.1 200) 6 headers in 358 bytes (2 switches on core 0) Aug 02 10:02:00.918870 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35fb1824-489c-4132-956b-9e49e019b6a9 None admin] Authenticating user token {{(pid=80377) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:02:00.931957 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77833e94-27b8-4725-b34f-259dfacacbda None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '891005fa5ff74041b75747d16cc66798', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80378) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:02:00.932989 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77833e94-27b8-4725-b34f-259dfacacbda None admin] REQUEST_METHOD: `GET` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:02:00.933490 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77833e94-27b8-4725-b34f-259dfacacbda None admin] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:02:00.934012 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77833e94-27b8-4725-b34f-259dfacacbda None admin] PATH_INFO: `/v3/roles` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:02:00.935478 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77833e94-27b8-4725-b34f-259dfacacbda None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=80378) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 02 10:02:00.935975 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77833e94-27b8-4725-b34f-259dfacacbda None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:02:00.938499 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77833e94-27b8-4725-b34f-259dfacacbda None admin] RBAC: Authorization granted {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:02:00.959641 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 215/432] 10.222.0.82 () {56 vars in 1111 bytes} [Tue Aug 2 10:02:00 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 271 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 02 10:02:00.968012 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77833e94-27b8-4725-b34f-259dfacacbda None admin] Authenticating user token {{(pid=80378) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:02:01.028795 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG dbcounter [-] [80378] Writing DB stats keystone:SELECT=100 {{(pid=80378) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 10:02:01.094469 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35fb1824-489c-4132-956b-9e49e019b6a9 None admin] Validating token access rules against request {{(pid=80377) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:02:01.164410 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec5777de-e0ae-4eaa-ab43-5c04b1b5bc8c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '891005fa5ff74041b75747d16cc66798', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80377) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:02:01.165754 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec5777de-e0ae-4eaa-ab43-5c04b1b5bc8c None admin] REQUEST_METHOD: `GET` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:02:01.166474 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec5777de-e0ae-4eaa-ab43-5c04b1b5bc8c None admin] SCRIPT_NAME: `/identity` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:02:01.167152 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec5777de-e0ae-4eaa-ab43-5c04b1b5bc8c None admin] PATH_INFO: `/v3/projects/c1809c9e380b45ec803130fe38eeaf24` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:02:01.169558 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec5777de-e0ae-4eaa-ab43-5c04b1b5bc8c None admin] RBAC: Authorizing `identity:get_project(project_id=c1809c9e380b45ec803130fe38eeaf24)` {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:02:01.172941 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec5777de-e0ae-4eaa-ab43-5c04b1b5bc8c None admin] RBAC: Authorization granted {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:02:01.176961 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: [pid: 80377|app: 0|req: 218/433] 10.222.0.82 () {56 vars in 1186 bytes} [Tue Aug 2 10:02:00 2022] GET /identity/v3/projects/c1809c9e380b45ec803130fe38eeaf24 => generated 298 bytes in 259 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 02 10:02:01.185929 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec5777de-e0ae-4eaa-ab43-5c04b1b5bc8c None admin] Authenticating user token {{(pid=80377) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:02:01.187514 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77833e94-27b8-4725-b34f-259dfacacbda None admin] Validating token access rules against request {{(pid=80378) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:02:01.276406 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fae2dc0-906c-480f-bec6-51fe8e9d3340 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '891005fa5ff74041b75747d16cc66798', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80378) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:02:01.277455 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fae2dc0-906c-480f-bec6-51fe8e9d3340 None admin] REQUEST_METHOD: `GET` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:02:01.277871 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fae2dc0-906c-480f-bec6-51fe8e9d3340 None admin] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:02:01.278333 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fae2dc0-906c-480f-bec6-51fe8e9d3340 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:02:01.299482 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG dbcounter [-] [80377] Writing DB stats keystone:SELECT=100 {{(pid=80377) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 10:02:01.387839 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec5777de-e0ae-4eaa-ab43-5c04b1b5bc8c None admin] Validating token access rules against request {{(pid=80377) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:02:01.408477 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8fae2dc0-906c-480f-bec6-51fe8e9d3340 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:02:01.411194 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8fae2dc0-906c-480f-bec6-51fe8e9d3340 None admin] RBAC: Authorization granted {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:02:01.435492 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f06945b3-fd42-4457-a87d-bc46dea92ef2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '891005fa5ff74041b75747d16cc66798', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80377) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:02:01.436067 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f06945b3-fd42-4457-a87d-bc46dea92ef2 None admin] REQUEST_METHOD: `GET` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:02:01.436375 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f06945b3-fd42-4457-a87d-bc46dea92ef2 None admin] SCRIPT_NAME: `/identity` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:02:01.436633 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f06945b3-fd42-4457-a87d-bc46dea92ef2 None admin] PATH_INFO: `/v3/groups/7a7f06a893c2484aafc43a46704b6e2a` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:02:01.437905 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f06945b3-fd42-4457-a87d-bc46dea92ef2 None admin] RBAC: Authorizing `identity:get_group(group_id=7a7f06a893c2484aafc43a46704b6e2a)` {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:02:01.439414 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f06945b3-fd42-4457-a87d-bc46dea92ef2 None admin] RBAC: Authorization granted {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:02:01.441064 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: [pid: 80377|app: 0|req: 219/434] 10.222.0.82 () {56 vars in 1180 bytes} [Tue Aug 2 10:02:01 2022] GET /identity/v3/groups/7a7f06a893c2484aafc43a46704b6e2a => generated 226 bytes in 256 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 02 10:02:01.447705 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5238e432-2e18-4828-ad00-d5dd427b2823 None None] REQUEST_METHOD: `GET` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:02:01.448216 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5238e432-2e18-4828-ad00-d5dd427b2823 None None] SCRIPT_NAME: `/identity` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:02:01.448714 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5238e432-2e18-4828-ad00-d5dd427b2823 None None] PATH_INFO: `/` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:02:01.449219 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG dbcounter [-] [80378] Writing DB stats keystone:SELECT=100 {{(pid=80378) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 10:02:01.451484 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: [pid: 80377|app: 0|req: 220/435] 10.222.0.82 () {54 vars in 926 bytes} [Tue Aug 2 10:02:01 2022] GET /identity => generated 270 bytes in 10 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 02 10:02:01.459389 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5238e432-2e18-4828-ad00-d5dd427b2823 None None] Authenticating user token {{(pid=80377) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:02:01.573357 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 216/436] 10.222.0.82 () {58 vars in 1282 bytes} [Tue Aug 2 10:02:00 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 606 msecs (HTTP/1.1 200) 6 headers in 358 bytes (2 switches on core 0) Aug 02 10:02:01.575681 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6db54bf9-d19d-461b-9af0-2f90e6676cb2 None None] REQUEST_METHOD: `POST` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:02:01.575972 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6db54bf9-d19d-461b-9af0-2f90e6676cb2 None None] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:02:01.576272 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6db54bf9-d19d-461b-9af0-2f90e6676cb2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:02:01.601535 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.auth.core [None req-6db54bf9-d19d-461b-9af0-2f90e6676cb2 None None] MFA Rules not processed for user `891005fa5ff74041b75747d16cc66798`. Rule list: `[]` (Enabled: `True`). {{(pid=80378) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 02 10:02:01.603033 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.fernet_utils [None req-6db54bf9-d19d-461b-9af0-2f90e6676cb2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80378) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 10:02:01.632939 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5238e432-2e18-4828-ad00-d5dd427b2823 None None] Validating token access rules against request {{(pid=80377) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:02:01.675804 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG dbcounter [-] [80377] Writing DB stats keystone:SELECT=100 {{(pid=80377) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 10:02:01.695293 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fec96ae-a2fc-4b15-88df-11cf1885fd6e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '891005fa5ff74041b75747d16cc66798', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80377) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:02:01.696211 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fec96ae-a2fc-4b15-88df-11cf1885fd6e None admin] REQUEST_METHOD: `GET` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:02:01.696689 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fec96ae-a2fc-4b15-88df-11cf1885fd6e None admin] SCRIPT_NAME: `/identity` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:02:01.697116 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fec96ae-a2fc-4b15-88df-11cf1885fd6e None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:02:01.698188 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3fec96ae-a2fc-4b15-88df-11cf1885fd6e None admin] RBAC: Adding query filter params (group.id=7a7f06a893c2484aafc43a46704b6e2a, role.id=649993d933d44c50b268df7fdf336fbf, scope.project.id=c1809c9e380b45ec803130fe38eeaf24) {{(pid=80377) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 02 10:02:01.698716 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3fec96ae-a2fc-4b15-88df-11cf1885fd6e None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:02:01.700661 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3fec96ae-a2fc-4b15-88df-11cf1885fd6e None admin] RBAC: Authorization granted {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:02:01.708693 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: [pid: 80377|app: 0|req: 221/437] 10.222.0.82 () {56 vars in 1376 bytes} [Tue Aug 2 10:02:01 2022] GET /identity/v3/role_assignments?group.id=7a7f06a893c2484aafc43a46704b6e2a&scope.project.id=c1809c9e380b45ec803130fe38eeaf24&role.id=649993d933d44c50b268df7fdf336fbf => generated 611 bytes in 250 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 02 10:02:01.709317 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fec96ae-a2fc-4b15-88df-11cf1885fd6e None admin] Authenticating user token {{(pid=80377) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:02:01.758470 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 217/438] 10.222.0.82 () {58 vars in 1023 bytes} [Tue Aug 2 10:02:01 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 184 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 02 10:02:01.796570 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6db54bf9-d19d-461b-9af0-2f90e6676cb2 None None] Authenticating user token {{(pid=80378) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:02:01.898637 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG dbcounter [-] [80378] Writing DB stats keystone:SELECT=100 {{(pid=80378) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 10:02:01.907891 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fec96ae-a2fc-4b15-88df-11cf1885fd6e None admin] Validating token access rules against request {{(pid=80377) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:02:01.975883 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dcc590dd-4f49-4fa0-8bda-5346b5e65a51 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '891005fa5ff74041b75747d16cc66798', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80377) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:02:01.976615 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcc590dd-4f49-4fa0-8bda-5346b5e65a51 None admin] REQUEST_METHOD: `GET` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:02:01.976898 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcc590dd-4f49-4fa0-8bda-5346b5e65a51 None admin] SCRIPT_NAME: `/identity` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:02:01.977177 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcc590dd-4f49-4fa0-8bda-5346b5e65a51 None admin] PATH_INFO: `/v3/projects/f7db7cf528494fbb8836c08d79b752ef` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:02:01.978447 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6db54bf9-d19d-461b-9af0-2f90e6676cb2 None None] Validating token access rules against request {{(pid=80378) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:02:01.978629 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dcc590dd-4f49-4fa0-8bda-5346b5e65a51 None admin] RBAC: Authorizing `identity:get_project(project_id=f7db7cf528494fbb8836c08d79b752ef)` {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:02:01.980002 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dcc590dd-4f49-4fa0-8bda-5346b5e65a51 None admin] RBAC: Authorization granted {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:02:01.982211 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: [pid: 80377|app: 0|req: 222/439] 10.222.0.82 () {56 vars in 1186 bytes} [Tue Aug 2 10:02:01 2022] GET /identity/v3/projects/f7db7cf528494fbb8836c08d79b752ef => generated 302 bytes in 273 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 02 10:02:01.984602 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab911161-1caf-47bb-9d7b-d7d790913c62 None None] REQUEST_METHOD: `GET` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:02:01.984909 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab911161-1caf-47bb-9d7b-d7d790913c62 None None] SCRIPT_NAME: `/identity` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:02:01.985326 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab911161-1caf-47bb-9d7b-d7d790913c62 None None] PATH_INFO: `/` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:02:01.987344 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: [pid: 80377|app: 0|req: 223/440] 10.222.0.82 () {54 vars in 926 bytes} [Tue Aug 2 10:02:01 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 02 10:02:01.991832 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab911161-1caf-47bb-9d7b-d7d790913c62 None None] Authenticating user token {{(pid=80377) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:02:02.036222 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8dfcac1a-e965-4c64-ad7d-ee51dcf66c69 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '891005fa5ff74041b75747d16cc66798', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80378) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:02:02.038038 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8dfcac1a-e965-4c64-ad7d-ee51dcf66c69 None admin] REQUEST_METHOD: `GET` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:02:02.038491 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8dfcac1a-e965-4c64-ad7d-ee51dcf66c69 None admin] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:02:02.040645 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8dfcac1a-e965-4c64-ad7d-ee51dcf66c69 None admin] PATH_INFO: `/v3/roles/member` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:02:02.062489 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8dfcac1a-e965-4c64-ad7d-ee51dcf66c69 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 02 10:02:02.067927 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application [None req-8dfcac1a-e965-4c64-ad7d-ee51dcf66c69 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Aug 02 10:02:02.067927 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 02 10:02:02.067927 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 02 10:02:02.067927 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 02 10:02:02.067927 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 02 10:02:02.067927 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 02 10:02:02.067927 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 02 10:02:02.067927 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 02 10:02:02.067927 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 02 10:02:02.067927 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 02 10:02:02.067927 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 02 10:02:02.067927 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 02 10:02:02.067927 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 02 10:02:02.067927 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 02 10:02:02.067927 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 02 10:02:02.067927 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 02 10:02:02.069111 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 02 10:02:02.069111 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 02 10:02:02.069111 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 02 10:02:02.069111 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 02 10:02:02.069111 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 02 10:02:02.069111 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 02 10:02:02.069111 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 02 10:02:02.069111 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 02 10:02:02.069111 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 02 10:02:02.069111 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application return self.get_or_create( Aug 02 10:02:02.069111 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 02 10:02:02.069111 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application with Lock( Aug 02 10:02:02.069111 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 02 10:02:02.069111 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application return self._enter() Aug 02 10:02:02.069111 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 02 10:02:02.069111 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 02 10:02:02.069972 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 02 10:02:02.069972 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application return self.creator() Aug 02 10:02:02.069972 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 02 10:02:02.069972 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application created_value = creator( Aug 02 10:02:02.069972 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 02 10:02:02.069972 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 02 10:02:02.069972 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 02 10:02:02.069972 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 02 10:02:02.069972 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 02 10:02:02.069972 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 02 10:02:02.069972 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Aug 02 10:02:02.069972 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application Aug 02 10:02:02.069972 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 218/441] 10.222.0.82 () {56 vars in 1099 bytes} [Tue Aug 2 10:02:01 2022] GET /identity/v3/roles/member => generated 84 bytes in 274 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 02 10:02:02.070836 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-828ab559-9b53-4dd1-9ab8-b436cdbb5e02 None None] REQUEST_METHOD: `POST` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:02:02.070836 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-828ab559-9b53-4dd1-9ab8-b436cdbb5e02 None None] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:02:02.071058 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-828ab559-9b53-4dd1-9ab8-b436cdbb5e02 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:02:02.079748 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG dbcounter [-] [80377] Writing DB stats keystone:SELECT=100 {{(pid=80377) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 10:02:02.090890 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.auth.core [None req-828ab559-9b53-4dd1-9ab8-b436cdbb5e02 None None] MFA Rules not processed for user `891005fa5ff74041b75747d16cc66798`. Rule list: `[]` (Enabled: `True`). {{(pid=80378) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 02 10:02:02.092862 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.fernet_utils [None req-828ab559-9b53-4dd1-9ab8-b436cdbb5e02 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80378) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 10:02:02.173622 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab911161-1caf-47bb-9d7b-d7d790913c62 None None] Validating token access rules against request {{(pid=80377) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:02:02.215655 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 219/442] 10.222.0.82 () {58 vars in 1023 bytes} [Tue Aug 2 10:02:02 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 146 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 02 10:02:02.216765 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-828ab559-9b53-4dd1-9ab8-b436cdbb5e02 None None] Authenticating user token {{(pid=80378) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:02:02.239084 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0889477d-6db5-4e51-bba9-8d5ccf12792a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '891005fa5ff74041b75747d16cc66798', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80377) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:02:02.240226 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0889477d-6db5-4e51-bba9-8d5ccf12792a None admin] REQUEST_METHOD: `GET` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:02:02.240722 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0889477d-6db5-4e51-bba9-8d5ccf12792a None admin] SCRIPT_NAME: `/identity` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:02:02.241206 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0889477d-6db5-4e51-bba9-8d5ccf12792a None admin] PATH_INFO: `/v3/groups/7a7f06a893c2484aafc43a46704b6e2a` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:02:02.243651 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0889477d-6db5-4e51-bba9-8d5ccf12792a None admin] RBAC: Authorizing `identity:get_group(group_id=7a7f06a893c2484aafc43a46704b6e2a)` {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:02:02.245882 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0889477d-6db5-4e51-bba9-8d5ccf12792a None admin] RBAC: Authorization granted {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:02:02.250707 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: [pid: 80377|app: 0|req: 224/443] 10.222.0.82 () {56 vars in 1180 bytes} [Tue Aug 2 10:02:01 2022] GET /identity/v3/groups/7a7f06a893c2484aafc43a46704b6e2a => generated 226 bytes in 258 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 02 10:02:02.250707 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0889477d-6db5-4e51-bba9-8d5ccf12792a None admin] Authenticating user token {{(pid=80377) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:02:02.282071 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG dbcounter [-] [80378] Writing DB stats keystone:SELECT=100 {{(pid=80378) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 10:02:02.381205 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-828ab559-9b53-4dd1-9ab8-b436cdbb5e02 None None] Validating token access rules against request {{(pid=80378) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:02:02.383896 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0889477d-6db5-4e51-bba9-8d5ccf12792a None admin] Validating token access rules against request {{(pid=80377) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:02:02.410561 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG dbcounter [-] [80377] Writing DB stats keystone:SELECT=100 {{(pid=80377) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 10:02:02.426233 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb504f6a-fe2b-48cd-98cb-da4389d8802e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '891005fa5ff74041b75747d16cc66798', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80378) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:02:02.427063 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb504f6a-fe2b-48cd-98cb-da4389d8802e None admin] REQUEST_METHOD: `GET` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:02:02.427272 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb504f6a-fe2b-48cd-98cb-da4389d8802e None admin] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:02:02.427668 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb504f6a-fe2b-48cd-98cb-da4389d8802e None admin] PATH_INFO: `/v3/roles` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:02:02.428843 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb504f6a-fe2b-48cd-98cb-da4389d8802e None admin] RBAC: Adding query filter params (name=member) {{(pid=80378) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 02 10:02:02.429630 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb504f6a-fe2b-48cd-98cb-da4389d8802e None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:02:02.432854 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb504f6a-fe2b-48cd-98cb-da4389d8802e None admin] RBAC: Authorization granted {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:02:02.437155 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d47e67b-8933-4203-b17a-29f8acc51a67 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '891005fa5ff74041b75747d16cc66798', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80377) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:02:02.437903 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d47e67b-8933-4203-b17a-29f8acc51a67 None admin] REQUEST_METHOD: `GET` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:02:02.438268 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d47e67b-8933-4203-b17a-29f8acc51a67 None admin] SCRIPT_NAME: `/identity` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:02:02.438727 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d47e67b-8933-4203-b17a-29f8acc51a67 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:02:02.447504 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 220/444] 10.222.0.82 () {56 vars in 1101 bytes} [Tue Aug 2 10:02:02 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 229 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 02 10:02:02.447504 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb504f6a-fe2b-48cd-98cb-da4389d8802e None admin] Authenticating user token {{(pid=80378) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:02:02.458894 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d47e67b-8933-4203-b17a-29f8acc51a67 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 02 10:02:02.468498 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application [None req-7d47e67b-8933-4203-b17a-29f8acc51a67 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 02 10:02:02.468498 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 02 10:02:02.468498 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 02 10:02:02.468498 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 02 10:02:02.468498 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 02 10:02:02.468498 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 02 10:02:02.468498 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 02 10:02:02.468498 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 02 10:02:02.468498 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 02 10:02:02.468498 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 02 10:02:02.468498 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 02 10:02:02.468498 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 02 10:02:02.468498 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 02 10:02:02.468498 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 02 10:02:02.468498 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 02 10:02:02.468498 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 02 10:02:02.469626 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 02 10:02:02.469626 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 02 10:02:02.469626 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 02 10:02:02.469626 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 02 10:02:02.469626 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 02 10:02:02.469626 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 02 10:02:02.469626 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 02 10:02:02.469626 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 02 10:02:02.469626 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 02 10:02:02.469626 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application return self.get_or_create( Aug 02 10:02:02.469626 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 02 10:02:02.469626 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application with Lock( Aug 02 10:02:02.469626 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 02 10:02:02.469626 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application return self._enter() Aug 02 10:02:02.469626 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 02 10:02:02.469626 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 02 10:02:02.470779 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 02 10:02:02.470779 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application return self.creator() Aug 02 10:02:02.470779 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 02 10:02:02.470779 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application created_value = creator( Aug 02 10:02:02.470779 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 02 10:02:02.470779 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 02 10:02:02.470779 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 02 10:02:02.470779 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 02 10:02:02.470779 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 02 10:02:02.470779 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 02 10:02:02.470779 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 02 10:02:02.470779 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application Aug 02 10:02:02.470779 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: [pid: 80377|app: 0|req: 225/445] 10.222.0.82 () {56 vars in 1096 bytes} [Tue Aug 2 10:02:02 2022] GET /identity/v3/roles/admin => generated 83 bytes in 221 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 02 10:02:02.471502 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d47e67b-8933-4203-b17a-29f8acc51a67 None admin] Authenticating user token {{(pid=80377) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:02:02.607478 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb504f6a-fe2b-48cd-98cb-da4389d8802e None admin] Validating token access rules against request {{(pid=80378) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:02:02.611571 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG dbcounter [-] [80378] Writing DB stats keystone:SELECT=100 {{(pid=80378) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 10:02:02.651257 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d47e67b-8933-4203-b17a-29f8acc51a67 None admin] Validating token access rules against request {{(pid=80377) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:02:02.668945 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f830fe9-3045-4948-8d54-e92fd88f7522 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '891005fa5ff74041b75747d16cc66798', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80378) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:02:02.669949 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f830fe9-3045-4948-8d54-e92fd88f7522 None admin] REQUEST_METHOD: `GET` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:02:02.670245 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f830fe9-3045-4948-8d54-e92fd88f7522 None admin] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:02:02.670534 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f830fe9-3045-4948-8d54-e92fd88f7522 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:02:02.671326 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f830fe9-3045-4948-8d54-e92fd88f7522 None admin] RBAC: Adding query filter params (group.id=7a7f06a893c2484aafc43a46704b6e2a, role.id=649993d933d44c50b268df7fdf336fbf, scope.project.id=f7db7cf528494fbb8836c08d79b752ef) {{(pid=80378) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 02 10:02:02.671636 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f830fe9-3045-4948-8d54-e92fd88f7522 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:02:02.673155 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f830fe9-3045-4948-8d54-e92fd88f7522 None admin] RBAC: Authorization granted {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:02:02.679690 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 221/446] 10.222.0.82 () {56 vars in 1376 bytes} [Tue Aug 2 10:02:02 2022] GET /identity/v3/role_assignments?group.id=7a7f06a893c2484aafc43a46704b6e2a&scope.project.id=f7db7cf528494fbb8836c08d79b752ef&role.id=649993d933d44c50b268df7fdf336fbf => generated 611 bytes in 233 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 02 10:02:02.680563 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f830fe9-3045-4948-8d54-e92fd88f7522 None admin] Authenticating user token {{(pid=80378) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:02:02.715784 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11e5c105-6d82-4bd0-9ac9-0953cfda4282 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '891005fa5ff74041b75747d16cc66798', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80377) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:02:02.717193 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11e5c105-6d82-4bd0-9ac9-0953cfda4282 None admin] REQUEST_METHOD: `GET` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:02:02.717694 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11e5c105-6d82-4bd0-9ac9-0953cfda4282 None admin] SCRIPT_NAME: `/identity` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:02:02.718283 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11e5c105-6d82-4bd0-9ac9-0953cfda4282 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:02:02.816628 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG dbcounter [-] [80377] Writing DB stats keystone:SELECT=100 {{(pid=80377) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 10:02:02.865593 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11e5c105-6d82-4bd0-9ac9-0953cfda4282 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:02:02.867739 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11e5c105-6d82-4bd0-9ac9-0953cfda4282 None admin] RBAC: Authorization granted {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:02:02.886235 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f830fe9-3045-4948-8d54-e92fd88f7522 None admin] Validating token access rules against request {{(pid=80378) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:02:02.955012 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e66774c-6c20-4ec7-8150-aec6c4dcf355 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '891005fa5ff74041b75747d16cc66798', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80378) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:02:02.955836 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e66774c-6c20-4ec7-8150-aec6c4dcf355 None admin] REQUEST_METHOD: `GET` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:02:02.956245 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e66774c-6c20-4ec7-8150-aec6c4dcf355 None admin] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:02:02.956650 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e66774c-6c20-4ec7-8150-aec6c4dcf355 None admin] PATH_INFO: `/v3/roles` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:02:02.957619 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e66774c-6c20-4ec7-8150-aec6c4dcf355 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80378) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 02 10:02:02.958154 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e66774c-6c20-4ec7-8150-aec6c4dcf355 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:02:02.960388 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e66774c-6c20-4ec7-8150-aec6c4dcf355 None admin] RBAC: Authorization granted {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:02:02.972433 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 222/447] 10.222.0.82 () {56 vars in 1099 bytes} [Tue Aug 2 10:02:02 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 292 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 02 10:02:02.978462 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e66774c-6c20-4ec7-8150-aec6c4dcf355 None admin] Authenticating user token {{(pid=80378) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:02:03.030880 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG dbcounter [-] [80378] Writing DB stats keystone:SELECT=100 {{(pid=80378) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 10:02:03.059649 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: [pid: 80377|app: 0|req: 226/448] 10.222.0.82 () {58 vars in 1282 bytes} [Tue Aug 2 10:02:02 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 589 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 02 10:02:03.068630 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11e5c105-6d82-4bd0-9ac9-0953cfda4282 None admin] Authenticating user token {{(pid=80377) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:02:03.132221 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e66774c-6c20-4ec7-8150-aec6c4dcf355 None admin] Validating token access rules against request {{(pid=80378) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:02:03.182309 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d18d042d-c03a-459b-b58a-20a900b2d755 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '891005fa5ff74041b75747d16cc66798', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80378) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:02:03.182938 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d18d042d-c03a-459b-b58a-20a900b2d755 None admin] REQUEST_METHOD: `GET` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:02:03.183224 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d18d042d-c03a-459b-b58a-20a900b2d755 None admin] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:02:03.183502 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d18d042d-c03a-459b-b58a-20a900b2d755 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:02:03.185913 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG dbcounter [-] [80377] Writing DB stats keystone:SELECT=100 {{(pid=80377) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 10:02:03.228514 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11e5c105-6d82-4bd0-9ac9-0953cfda4282 None admin] Validating token access rules against request {{(pid=80377) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:02:03.290682 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b6eb4af-fc2e-4647-8ee9-77831a6335d6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '891005fa5ff74041b75747d16cc66798', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80377) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:02:03.291527 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b6eb4af-fc2e-4647-8ee9-77831a6335d6 None admin] REQUEST_METHOD: `GET` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:02:03.291883 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b6eb4af-fc2e-4647-8ee9-77831a6335d6 None admin] SCRIPT_NAME: `/identity` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:02:03.292232 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b6eb4af-fc2e-4647-8ee9-77831a6335d6 None admin] PATH_INFO: `/v3/projects/f7db7cf528494fbb8836c08d79b752ef` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:02:03.293552 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b6eb4af-fc2e-4647-8ee9-77831a6335d6 None admin] RBAC: Authorizing `identity:get_project(project_id=f7db7cf528494fbb8836c08d79b752ef)` {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:02:03.295373 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b6eb4af-fc2e-4647-8ee9-77831a6335d6 None admin] RBAC: Authorization granted {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:02:03.297262 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: [pid: 80377|app: 0|req: 227/449] 10.222.0.82 () {56 vars in 1186 bytes} [Tue Aug 2 10:02:03 2022] GET /identity/v3/projects/f7db7cf528494fbb8836c08d79b752ef => generated 302 bytes in 230 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 02 10:02:03.303550 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b6eb4af-fc2e-4647-8ee9-77831a6335d6 None admin] Authenticating user token {{(pid=80377) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:02:03.305023 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d18d042d-c03a-459b-b58a-20a900b2d755 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:02:03.310667 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d18d042d-c03a-459b-b58a-20a900b2d755 None admin] RBAC: Authorization granted {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:02:03.361100 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG dbcounter [-] [80378] Writing DB stats keystone:SELECT=100 {{(pid=80378) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 10:02:03.454967 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b6eb4af-fc2e-4647-8ee9-77831a6335d6 None admin] Validating token access rules against request {{(pid=80377) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:02:03.468224 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 223/450] 10.222.0.82 () {58 vars in 1282 bytes} [Tue Aug 2 10:02:02 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 483 msecs (HTTP/1.1 200) 6 headers in 358 bytes (2 switches on core 0) Aug 02 10:02:03.470617 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d18d042d-c03a-459b-b58a-20a900b2d755 None admin] Authenticating user token {{(pid=80378) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:02:03.497580 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-132021b7-eb3f-4656-a9b7-ebd115d42a87 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '891005fa5ff74041b75747d16cc66798', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80377) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:02:03.500313 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-132021b7-eb3f-4656-a9b7-ebd115d42a87 None admin] REQUEST_METHOD: `GET` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:02:03.500313 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-132021b7-eb3f-4656-a9b7-ebd115d42a87 None admin] SCRIPT_NAME: `/identity` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:02:03.500313 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-132021b7-eb3f-4656-a9b7-ebd115d42a87 None admin] PATH_INFO: `/v3/groups/7a7f06a893c2484aafc43a46704b6e2a` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:02:03.500313 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-132021b7-eb3f-4656-a9b7-ebd115d42a87 None admin] RBAC: Authorizing `identity:get_group(group_id=7a7f06a893c2484aafc43a46704b6e2a)` {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:02:03.502228 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-132021b7-eb3f-4656-a9b7-ebd115d42a87 None admin] RBAC: Authorization granted {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:02:03.504448 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: [pid: 80377|app: 0|req: 228/451] 10.222.0.82 () {56 vars in 1180 bytes} [Tue Aug 2 10:02:03 2022] GET /identity/v3/groups/7a7f06a893c2484aafc43a46704b6e2a => generated 226 bytes in 201 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 02 10:02:03.510533 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-132021b7-eb3f-4656-a9b7-ebd115d42a87 None admin] Authenticating user token {{(pid=80377) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:02:03.543215 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG dbcounter [-] [80377] Writing DB stats keystone:SELECT=100 {{(pid=80377) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 10:02:03.634362 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d18d042d-c03a-459b-b58a-20a900b2d755 None admin] Validating token access rules against request {{(pid=80378) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:02:03.671780 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG dbcounter [-] [80378] Writing DB stats keystone:SELECT=100 {{(pid=80378) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 10:02:03.679938 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4794c8df-0f42-4921-b8da-c219e821ecae None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '891005fa5ff74041b75747d16cc66798', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80378) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:02:03.680496 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-132021b7-eb3f-4656-a9b7-ebd115d42a87 None admin] Validating token access rules against request {{(pid=80377) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:02:03.680666 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4794c8df-0f42-4921-b8da-c219e821ecae None admin] REQUEST_METHOD: `GET` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:02:03.680871 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4794c8df-0f42-4921-b8da-c219e821ecae None admin] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:02:03.681165 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4794c8df-0f42-4921-b8da-c219e821ecae None admin] PATH_INFO: `/v3/projects/7047912c5acd4adeba5d87c3bf895e39` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:02:03.682612 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4794c8df-0f42-4921-b8da-c219e821ecae None admin] RBAC: Authorizing `identity:get_project(project_id=7047912c5acd4adeba5d87c3bf895e39)` {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:02:03.684324 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4794c8df-0f42-4921-b8da-c219e821ecae None admin] RBAC: Authorization granted {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:02:03.686249 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 224/452] 10.222.0.82 () {56 vars in 1186 bytes} [Tue Aug 2 10:02:03 2022] GET /identity/v3/projects/7047912c5acd4adeba5d87c3bf895e39 => generated 344 bytes in 217 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 02 10:02:03.695567 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4794c8df-0f42-4921-b8da-c219e821ecae None admin] Authenticating user token {{(pid=80378) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:02:03.745011 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e33299e0-cb60-447f-8938-b136ac739ae2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '891005fa5ff74041b75747d16cc66798', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80377) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:02:03.745972 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e33299e0-cb60-447f-8938-b136ac739ae2 None admin] REQUEST_METHOD: `GET` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:02:03.746103 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e33299e0-cb60-447f-8938-b136ac739ae2 None admin] SCRIPT_NAME: `/identity` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:02:03.746501 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e33299e0-cb60-447f-8938-b136ac739ae2 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:02:03.747947 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e33299e0-cb60-447f-8938-b136ac739ae2 None admin] RBAC: Adding query filter params (group.id=7a7f06a893c2484aafc43a46704b6e2a, role.id=e1c04b19fc4543b8b54f58bbcffbc598, scope.project.id=f7db7cf528494fbb8836c08d79b752ef) {{(pid=80377) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 02 10:02:03.748461 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e33299e0-cb60-447f-8938-b136ac739ae2 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:02:03.750836 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e33299e0-cb60-447f-8938-b136ac739ae2 None admin] RBAC: Authorization granted {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:02:03.758853 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: [pid: 80377|app: 0|req: 229/453] 10.222.0.82 () {56 vars in 1376 bytes} [Tue Aug 2 10:02:03 2022] GET /identity/v3/role_assignments?group.id=7a7f06a893c2484aafc43a46704b6e2a&scope.project.id=f7db7cf528494fbb8836c08d79b752ef&role.id=e1c04b19fc4543b8b54f58bbcffbc598 => generated 611 bytes in 249 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 02 10:02:03.868809 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4794c8df-0f42-4921-b8da-c219e821ecae None admin] Validating token access rules against request {{(pid=80378) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:02:03.924126 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25808c62-6721-4cc6-aae8-34292c3277b5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '891005fa5ff74041b75747d16cc66798', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80378) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:02:03.925216 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25808c62-6721-4cc6-aae8-34292c3277b5 None admin] REQUEST_METHOD: `GET` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:02:03.925697 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25808c62-6721-4cc6-aae8-34292c3277b5 None admin] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:02:03.926271 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25808c62-6721-4cc6-aae8-34292c3277b5 None admin] PATH_INFO: `/v3/groups/948c83fe907c4d3b8071d26809f7d765` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:02:03.928420 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25808c62-6721-4cc6-aae8-34292c3277b5 None admin] RBAC: Authorizing `identity:get_group(group_id=948c83fe907c4d3b8071d26809f7d765)` {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:02:03.931270 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25808c62-6721-4cc6-aae8-34292c3277b5 None admin] RBAC: Authorization granted {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:02:03.934217 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 225/454] 10.222.0.82 () {56 vars in 1180 bytes} [Tue Aug 2 10:02:03 2022] GET /identity/v3/groups/948c83fe907c4d3b8071d26809f7d765 => generated 229 bytes in 240 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 02 10:02:03.944152 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e33299e0-cb60-447f-8938-b136ac739ae2 None admin] Authenticating user token {{(pid=80377) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:02:04.056812 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG dbcounter [-] [80377] Writing DB stats keystone:SELECT=100 {{(pid=80377) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 10:02:04.083699 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e33299e0-cb60-447f-8938-b136ac739ae2 None admin] Validating token access rules against request {{(pid=80377) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:02:04.144512 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed8547a9-16cb-45db-863d-670f82694c56 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '891005fa5ff74041b75747d16cc66798', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80377) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:02:04.145141 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed8547a9-16cb-45db-863d-670f82694c56 None admin] REQUEST_METHOD: `GET` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:02:04.145473 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed8547a9-16cb-45db-863d-670f82694c56 None admin] SCRIPT_NAME: `/identity` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:02:04.145702 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed8547a9-16cb-45db-863d-670f82694c56 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:02:04.146544 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed8547a9-16cb-45db-863d-670f82694c56 None admin] RBAC: Adding query filter params (group.id=948c83fe907c4d3b8071d26809f7d765, role.id=237966442f05411191105bf336bc14f3, scope.project.id=7047912c5acd4adeba5d87c3bf895e39) {{(pid=80377) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 02 10:02:04.146855 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed8547a9-16cb-45db-863d-670f82694c56 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:02:04.148315 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed8547a9-16cb-45db-863d-670f82694c56 None admin] RBAC: Authorization granted {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:02:04.160948 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: [pid: 80377|app: 0|req: 230/455] 10.222.0.82 () {56 vars in 1376 bytes} [Tue Aug 2 10:02:03 2022] GET /identity/v3/role_assignments?group.id=948c83fe907c4d3b8071d26809f7d765&scope.project.id=7047912c5acd4adeba5d87c3bf895e39&role.id=237966442f05411191105bf336bc14f3 => generated 611 bytes in 218 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 02 10:02:07.999450 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3435e9b0-115d-4c8c-b82e-9255bb1165bb None None] REQUEST_METHOD: `GET` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:02:08.000206 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3435e9b0-115d-4c8c-b82e-9255bb1165bb None None] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:02:08.000206 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3435e9b0-115d-4c8c-b82e-9255bb1165bb None None] PATH_INFO: `/` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:02:08.001463 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 226/456] 10.222.0.82 () {54 vars in 926 bytes} [Tue Aug 2 10:02:07 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 02 10:02:08.008669 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbf2011a-38ca-4a8e-a5c2-d21d1dcecd23 None None] REQUEST_METHOD: `POST` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:02:08.009384 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbf2011a-38ca-4a8e-a5c2-d21d1dcecd23 None None] SCRIPT_NAME: `/identity` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:02:08.009541 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbf2011a-38ca-4a8e-a5c2-d21d1dcecd23 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:02:08.058881 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.auth.core [None req-fbf2011a-38ca-4a8e-a5c2-d21d1dcecd23 None None] MFA Rules not processed for user `891005fa5ff74041b75747d16cc66798`. Rule list: `[]` (Enabled: `True`). {{(pid=80377) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 02 10:02:08.060532 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.fernet_utils [None req-fbf2011a-38ca-4a8e-a5c2-d21d1dcecd23 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80377) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 10:02:08.185078 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: [pid: 80377|app: 0|req: 231/457] 10.222.0.82 () {58 vars in 1023 bytes} [Tue Aug 2 10:02:08 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 178 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Aug 02 10:02:08.275903 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3435e9b0-115d-4c8c-b82e-9255bb1165bb None None] Authenticating user token {{(pid=80378) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:02:08.354256 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fc808f0-95a4-42b2-b86c-cbba5a9549dc None None] REQUEST_METHOD: `GET` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:02:08.354822 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fc808f0-95a4-42b2-b86c-cbba5a9549dc None None] SCRIPT_NAME: `/identity` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:02:08.355403 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fc808f0-95a4-42b2-b86c-cbba5a9549dc None None] PATH_INFO: `/` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:02:08.358375 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: [pid: 80377|app: 0|req: 232/458] 10.222.0.82 () {54 vars in 926 bytes} [Tue Aug 2 10:02:08 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 02 10:02:08.369592 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00f4b910-b674-46a8-bc24-bc911a13ee68 None None] REQUEST_METHOD: `POST` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:02:08.370522 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00f4b910-b674-46a8-bc24-bc911a13ee68 None None] SCRIPT_NAME: `/identity` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:02:08.371197 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00f4b910-b674-46a8-bc24-bc911a13ee68 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:02:08.410256 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG dbcounter [-] [80378] Writing DB stats keystone:SELECT=100 {{(pid=80378) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 10:02:08.421145 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.auth.core [None req-00f4b910-b674-46a8-bc24-bc911a13ee68 None None] MFA Rules not processed for user `891005fa5ff74041b75747d16cc66798`. Rule list: `[]` (Enabled: `True`). {{(pid=80377) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 02 10:02:08.423232 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.fernet_utils [None req-00f4b910-b674-46a8-bc24-bc911a13ee68 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80377) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 10:02:08.473299 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3435e9b0-115d-4c8c-b82e-9255bb1165bb None None] Validating token access rules against request {{(pid=80378) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:02:08.545431 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8fa00b1-07be-496a-a471-d725a37c00e7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '891005fa5ff74041b75747d16cc66798', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80378) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:02:08.546823 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8fa00b1-07be-496a-a471-d725a37c00e7 None admin] REQUEST_METHOD: `GET` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:02:08.547383 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8fa00b1-07be-496a-a471-d725a37c00e7 None admin] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:02:08.548226 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8fa00b1-07be-496a-a471-d725a37c00e7 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:02:08.561960 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: [pid: 80377|app: 0|req: 233/459] 10.222.0.82 () {58 vars in 1023 bytes} [Tue Aug 2 10:02:08 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 194 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 02 10:02:08.564078 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-836b0d85-54e2-4afe-9fcc-b98850f39323 None None] REQUEST_METHOD: `GET` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:02:08.564516 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-836b0d85-54e2-4afe-9fcc-b98850f39323 None None] SCRIPT_NAME: `/identity` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:02:08.564967 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-836b0d85-54e2-4afe-9fcc-b98850f39323 None None] PATH_INFO: `/` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:02:08.566686 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: [pid: 80377|app: 0|req: 234/460] 10.222.0.82 () {54 vars in 926 bytes} [Tue Aug 2 10:02:08 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 02 10:02:08.573815 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8fa00b1-07be-496a-a471-d725a37c00e7 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:02:08.577871 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8fa00b1-07be-496a-a471-d725a37c00e7 None admin] RBAC: Authorization granted {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:02:08.594718 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e781d36c-21ae-497f-832b-4a36eef95224 None None] REQUEST_METHOD: `POST` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:02:08.595416 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e781d36c-21ae-497f-832b-4a36eef95224 None None] SCRIPT_NAME: `/identity` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:02:08.595958 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e781d36c-21ae-497f-832b-4a36eef95224 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:02:08.599513 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application [None req-e8fa00b1-07be-496a-a471-d725a37c00e7 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 02 10:02:08.599513 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 02 10:02:08.599513 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 02 10:02:08.599513 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 02 10:02:08.599513 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 02 10:02:08.599513 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 02 10:02:08.599513 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 02 10:02:08.599513 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 02 10:02:08.599513 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 02 10:02:08.599513 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 02 10:02:08.599513 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 02 10:02:08.599513 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 02 10:02:08.599513 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 02 10:02:08.599513 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 02 10:02:08.599513 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 02 10:02:08.600487 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 02 10:02:08.600487 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 02 10:02:08.600487 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 02 10:02:08.600487 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 02 10:02:08.600487 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 02 10:02:08.600487 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 02 10:02:08.600487 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application return self.get_or_create( Aug 02 10:02:08.600487 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 02 10:02:08.600487 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application with Lock( Aug 02 10:02:08.600487 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 02 10:02:08.600487 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application return self._enter() Aug 02 10:02:08.600487 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 02 10:02:08.600487 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 02 10:02:08.600487 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 02 10:02:08.600487 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application return self.creator() Aug 02 10:02:08.600487 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 02 10:02:08.600487 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application created_value = creator( Aug 02 10:02:08.601627 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 02 10:02:08.601627 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 02 10:02:08.601627 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 02 10:02:08.601627 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application Aug 02 10:02:08.609424 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 227/461] 10.222.0.82 () {56 vars in 1108 bytes} [Tue Aug 2 10:02:08 2022] GET /identity/v3/domains/Default => generated 87 bytes in 329 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 02 10:02:08.611639 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8fa00b1-07be-496a-a471-d725a37c00e7 None admin] Authenticating user token {{(pid=80378) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:02:08.617497 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG dbcounter [-] [80377] Writing DB stats keystone:SELECT=100 {{(pid=80377) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 10:02:08.643513 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.auth.core [None req-e781d36c-21ae-497f-832b-4a36eef95224 None None] MFA Rules not processed for user `891005fa5ff74041b75747d16cc66798`. Rule list: `[]` (Enabled: `True`). {{(pid=80377) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 02 10:02:08.646271 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.fernet_utils [None req-e781d36c-21ae-497f-832b-4a36eef95224 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80377) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 10:02:08.752748 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: [pid: 80377|app: 0|req: 235/462] 10.222.0.82 () {58 vars in 1023 bytes} [Tue Aug 2 10:02:08 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 159 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 02 10:02:08.759499 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e781d36c-21ae-497f-832b-4a36eef95224 None None] Authenticating user token {{(pid=80377) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:02:08.794720 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8fa00b1-07be-496a-a471-d725a37c00e7 None admin] Validating token access rules against request {{(pid=80378) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:02:08.821374 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG dbcounter [-] [80378] Writing DB stats keystone:SELECT=100 {{(pid=80378) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 10:02:08.853999 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b16d745-f4c1-4673-b00a-537dc1ef57cc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '891005fa5ff74041b75747d16cc66798', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80378) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:02:08.854698 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b16d745-f4c1-4673-b00a-537dc1ef57cc None admin] REQUEST_METHOD: `GET` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:02:08.855171 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b16d745-f4c1-4673-b00a-537dc1ef57cc None admin] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:02:08.855639 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b16d745-f4c1-4673-b00a-537dc1ef57cc None admin] PATH_INFO: `/v3/domains` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:02:08.856819 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b16d745-f4c1-4673-b00a-537dc1ef57cc None admin] RBAC: Adding query filter params (name=Default) {{(pid=80378) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 02 10:02:08.857200 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b16d745-f4c1-4673-b00a-537dc1ef57cc None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:02:08.858739 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b16d745-f4c1-4673-b00a-537dc1ef57cc None admin] RBAC: Authorization granted {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:02:08.878240 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 228/463] 10.222.0.82 () {56 vars in 1109 bytes} [Tue Aug 2 10:02:08 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 267 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 02 10:02:08.879804 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06cf0aea-669c-4700-a888-a29a6a0d6b4a None None] REQUEST_METHOD: `GET` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:02:08.880260 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06cf0aea-669c-4700-a888-a29a6a0d6b4a None None] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:02:08.880641 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06cf0aea-669c-4700-a888-a29a6a0d6b4a None None] PATH_INFO: `/` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:02:08.882893 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 229/464] 10.222.0.82 () {54 vars in 926 bytes} [Tue Aug 2 10:02:08 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 02 10:02:08.883799 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06cf0aea-669c-4700-a888-a29a6a0d6b4a None None] Authenticating user token {{(pid=80378) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:02:08.956273 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e781d36c-21ae-497f-832b-4a36eef95224 None None] Validating token access rules against request {{(pid=80377) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:02:09.000462 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d575d77b-700b-4b32-b272-c382cc666b27 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '891005fa5ff74041b75747d16cc66798', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80377) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:02:09.002012 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d575d77b-700b-4b32-b272-c382cc666b27 None admin] REQUEST_METHOD: `GET` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:02:09.002462 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d575d77b-700b-4b32-b272-c382cc666b27 None admin] SCRIPT_NAME: `/identity` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:02:09.003021 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d575d77b-700b-4b32-b272-c382cc666b27 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:02:09.006106 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG dbcounter [-] [80377] Writing DB stats keystone:SELECT=100 {{(pid=80377) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 10:02:09.019273 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d575d77b-700b-4b32-b272-c382cc666b27 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:02:09.021816 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d575d77b-700b-4b32-b272-c382cc666b27 None admin] RBAC: Authorization granted {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:02:09.048397 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application [None req-d575d77b-700b-4b32-b272-c382cc666b27 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 02 10:02:09.048397 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 02 10:02:09.048397 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 02 10:02:09.048397 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 02 10:02:09.048397 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 02 10:02:09.048397 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 02 10:02:09.048397 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 02 10:02:09.048397 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 02 10:02:09.048397 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 02 10:02:09.048397 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 02 10:02:09.048397 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 02 10:02:09.048397 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 02 10:02:09.048397 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 02 10:02:09.048397 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 02 10:02:09.048397 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 02 10:02:09.050121 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 02 10:02:09.050121 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 02 10:02:09.050121 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 02 10:02:09.050121 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 02 10:02:09.050121 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 02 10:02:09.050121 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 02 10:02:09.050121 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application return self.get_or_create( Aug 02 10:02:09.050121 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 02 10:02:09.050121 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application with Lock( Aug 02 10:02:09.050121 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 02 10:02:09.050121 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application return self._enter() Aug 02 10:02:09.050121 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 02 10:02:09.050121 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 02 10:02:09.050121 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 02 10:02:09.050121 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application return self.creator() Aug 02 10:02:09.050121 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 02 10:02:09.050121 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application created_value = creator( Aug 02 10:02:09.052644 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 02 10:02:09.052644 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 02 10:02:09.052644 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 02 10:02:09.052644 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application Aug 02 10:02:09.052644 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: [pid: 80377|app: 0|req: 236/465] 10.222.0.82 () {56 vars in 1108 bytes} [Tue Aug 2 10:02:08 2022] GET /identity/v3/domains/Default => generated 87 bytes in 297 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 02 10:02:09.052644 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d575d77b-700b-4b32-b272-c382cc666b27 None admin] Authenticating user token {{(pid=80377) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:02:09.086182 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06cf0aea-669c-4700-a888-a29a6a0d6b4a None None] Validating token access rules against request {{(pid=80378) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:02:09.149372 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e1da175-09fe-4586-80b9-ee8e46510602 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '891005fa5ff74041b75747d16cc66798', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80378) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:02:09.151035 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e1da175-09fe-4586-80b9-ee8e46510602 None admin] REQUEST_METHOD: `GET` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:02:09.151320 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e1da175-09fe-4586-80b9-ee8e46510602 None admin] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:02:09.151595 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e1da175-09fe-4586-80b9-ee8e46510602 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:02:09.172532 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e1da175-09fe-4586-80b9-ee8e46510602 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:02:09.175715 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e1da175-09fe-4586-80b9-ee8e46510602 None admin] RBAC: Authorization granted {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:02:09.204969 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application [None req-3e1da175-09fe-4586-80b9-ee8e46510602 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 02 10:02:09.204969 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 02 10:02:09.204969 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 02 10:02:09.204969 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 02 10:02:09.204969 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 02 10:02:09.204969 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 02 10:02:09.204969 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 02 10:02:09.204969 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 02 10:02:09.204969 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 02 10:02:09.204969 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 02 10:02:09.204969 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 02 10:02:09.204969 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 02 10:02:09.204969 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 02 10:02:09.204969 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 02 10:02:09.204969 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 02 10:02:09.207112 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 02 10:02:09.207112 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 02 10:02:09.207112 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 02 10:02:09.207112 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 02 10:02:09.207112 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 02 10:02:09.207112 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 02 10:02:09.207112 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application return self.get_or_create( Aug 02 10:02:09.207112 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 02 10:02:09.207112 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application with Lock( Aug 02 10:02:09.207112 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 02 10:02:09.207112 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application return self._enter() Aug 02 10:02:09.207112 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 02 10:02:09.207112 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 02 10:02:09.207112 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 02 10:02:09.207112 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application return self.creator() Aug 02 10:02:09.207112 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 02 10:02:09.207112 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application created_value = creator( Aug 02 10:02:09.208871 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 02 10:02:09.208871 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 02 10:02:09.208871 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 02 10:02:09.208871 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application Aug 02 10:02:09.208871 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 230/466] 10.222.0.82 () {56 vars in 1108 bytes} [Tue Aug 2 10:02:08 2022] GET /identity/v3/domains/Default => generated 87 bytes in 324 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 02 10:02:09.209934 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7af3103f-9fde-4cf0-aa3a-cff042acbe0d None None] REQUEST_METHOD: `POST` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:02:09.210210 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7af3103f-9fde-4cf0-aa3a-cff042acbe0d None None] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:02:09.211105 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7af3103f-9fde-4cf0-aa3a-cff042acbe0d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:02:09.238152 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.auth.core [None req-7af3103f-9fde-4cf0-aa3a-cff042acbe0d None None] MFA Rules not processed for user `891005fa5ff74041b75747d16cc66798`. Rule list: `[]` (Enabled: `True`). {{(pid=80378) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 02 10:02:09.239391 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.fernet_utils [None req-7af3103f-9fde-4cf0-aa3a-cff042acbe0d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80378) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 10:02:09.241422 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d575d77b-700b-4b32-b272-c382cc666b27 None admin] Validating token access rules against request {{(pid=80377) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:02:09.276996 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG dbcounter [-] [80378] Writing DB stats keystone:SELECT=100 {{(pid=80378) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 10:02:09.286696 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b67ffe7-c0b6-481a-9269-c7e6bae42abe None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '891005fa5ff74041b75747d16cc66798', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80377) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:02:09.287453 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b67ffe7-c0b6-481a-9269-c7e6bae42abe None admin] REQUEST_METHOD: `POST` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:02:09.287778 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b67ffe7-c0b6-481a-9269-c7e6bae42abe None admin] SCRIPT_NAME: `/identity` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:02:09.288053 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b67ffe7-c0b6-481a-9269-c7e6bae42abe None admin] PATH_INFO: `/v3/users` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:02:09.288750 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b67ffe7-c0b6-481a-9269-c7e6bae42abe None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:02:09.290273 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b67ffe7-c0b6-481a-9269-c7e6bae42abe None admin] RBAC: Authorization granted {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:02:09.319176 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: [pid: 80377|app: 0|req: 237/467] 10.222.0.82 () {60 vars in 1132 bytes} [Tue Aug 2 10:02:09 2022] POST /identity/v3/users => generated 246 bytes in 268 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 02 10:02:09.320772 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b67ffe7-c0b6-481a-9269-c7e6bae42abe None admin] Authenticating user token {{(pid=80377) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:02:09.353631 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 231/468] 10.222.0.82 () {58 vars in 1023 bytes} [Tue Aug 2 10:02:09 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 146 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 02 10:02:09.354715 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7af3103f-9fde-4cf0-aa3a-cff042acbe0d None None] Authenticating user token {{(pid=80378) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:02:09.418889 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG dbcounter [-] [80377] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80377) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 10:02:09.513977 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b67ffe7-c0b6-481a-9269-c7e6bae42abe None admin] Validating token access rules against request {{(pid=80377) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:02:09.542724 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7af3103f-9fde-4cf0-aa3a-cff042acbe0d None None] Validating token access rules against request {{(pid=80378) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:02:09.578754 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72642db9-6b3b-460e-a4e4-bc7764668265 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '891005fa5ff74041b75747d16cc66798', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80377) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:02:09.579623 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72642db9-6b3b-460e-a4e4-bc7764668265 None admin] REQUEST_METHOD: `GET` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:02:09.580309 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72642db9-6b3b-460e-a4e4-bc7764668265 None admin] SCRIPT_NAME: `/identity` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:02:09.580309 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72642db9-6b3b-460e-a4e4-bc7764668265 None admin] PATH_INFO: `/v3/domains` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:02:09.581217 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72642db9-6b3b-460e-a4e4-bc7764668265 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80377) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 02 10:02:09.581629 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72642db9-6b3b-460e-a4e4-bc7764668265 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:02:09.583767 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72642db9-6b3b-460e-a4e4-bc7764668265 None admin] RBAC: Authorization granted {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:02:09.605495 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: [pid: 80377|app: 0|req: 238/469] 10.222.0.82 () {56 vars in 1109 bytes} [Tue Aug 2 10:02:09 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 286 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 02 10:02:09.607711 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffaec515-1633-4c9b-a8cb-d7657f50f034 None None] REQUEST_METHOD: `GET` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:02:09.608098 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffaec515-1633-4c9b-a8cb-d7657f50f034 None None] SCRIPT_NAME: `/identity` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:02:09.608507 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffaec515-1633-4c9b-a8cb-d7657f50f034 None None] PATH_INFO: `/` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:02:09.611149 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: [pid: 80377|app: 0|req: 239/470] 10.222.0.82 () {54 vars in 926 bytes} [Tue Aug 2 10:02:09 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 02 10:02:09.612704 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffaec515-1633-4c9b-a8cb-d7657f50f034 None None] Authenticating user token {{(pid=80377) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:02:09.621523 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a57c762d-7cc1-4ae6-999b-17df8ead2ac4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '891005fa5ff74041b75747d16cc66798', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80378) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:02:09.622613 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a57c762d-7cc1-4ae6-999b-17df8ead2ac4 None admin] REQUEST_METHOD: `GET` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:02:09.623089 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a57c762d-7cc1-4ae6-999b-17df8ead2ac4 None admin] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:02:09.623560 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a57c762d-7cc1-4ae6-999b-17df8ead2ac4 None admin] PATH_INFO: `/v3/domains` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:02:09.624846 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a57c762d-7cc1-4ae6-999b-17df8ead2ac4 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80378) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 02 10:02:09.625387 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a57c762d-7cc1-4ae6-999b-17df8ead2ac4 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:02:09.628483 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a57c762d-7cc1-4ae6-999b-17df8ead2ac4 None admin] RBAC: Authorization granted {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:02:09.648813 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 232/471] 10.222.0.82 () {56 vars in 1109 bytes} [Tue Aug 2 10:02:09 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 294 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 02 10:02:09.650124 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a57c762d-7cc1-4ae6-999b-17df8ead2ac4 None admin] Authenticating user token {{(pid=80378) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:02:09.690684 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG dbcounter [-] [80378] Writing DB stats keystone:SELECT=100 {{(pid=80378) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 10:02:09.806488 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffaec515-1633-4c9b-a8cb-d7657f50f034 None None] Validating token access rules against request {{(pid=80377) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:02:09.808988 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG dbcounter [-] [80377] Writing DB stats keystone:SELECT=100 {{(pid=80377) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 10:02:09.833419 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a57c762d-7cc1-4ae6-999b-17df8ead2ac4 None admin] Validating token access rules against request {{(pid=80378) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:02:09.867556 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-253e52c0-038c-49ce-a4ff-eb69924422b6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '891005fa5ff74041b75747d16cc66798', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80377) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:02:09.868648 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-253e52c0-038c-49ce-a4ff-eb69924422b6 None admin] REQUEST_METHOD: `GET` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:02:09.869141 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-253e52c0-038c-49ce-a4ff-eb69924422b6 None admin] SCRIPT_NAME: `/identity` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:02:09.869640 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-253e52c0-038c-49ce-a4ff-eb69924422b6 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:02:09.892740 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-253e52c0-038c-49ce-a4ff-eb69924422b6 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:02:09.892936 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-821614c4-af04-42df-8eca-ba3722687d2a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '891005fa5ff74041b75747d16cc66798', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80378) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:02:09.893404 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-821614c4-af04-42df-8eca-ba3722687d2a None admin] REQUEST_METHOD: `POST` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:02:09.893718 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-821614c4-af04-42df-8eca-ba3722687d2a None admin] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:02:09.894075 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-821614c4-af04-42df-8eca-ba3722687d2a None admin] PATH_INFO: `/v3/users` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:02:09.894812 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-821614c4-af04-42df-8eca-ba3722687d2a None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:02:09.896188 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-253e52c0-038c-49ce-a4ff-eb69924422b6 None admin] RBAC: Authorization granted {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:02:09.898303 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-821614c4-af04-42df-8eca-ba3722687d2a None admin] RBAC: Authorization granted {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:02:09.929830 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 233/472] 10.222.0.82 () {60 vars in 1132 bytes} [Tue Aug 2 10:02:09 2022] POST /identity/v3/users => generated 248 bytes in 274 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 02 10:02:09.930135 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application [None req-253e52c0-038c-49ce-a4ff-eb69924422b6 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 02 10:02:09.930135 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 02 10:02:09.930135 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 02 10:02:09.930135 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 02 10:02:09.930135 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 02 10:02:09.930135 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 02 10:02:09.930135 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 02 10:02:09.930135 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 02 10:02:09.930135 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 02 10:02:09.930135 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 02 10:02:09.930135 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 02 10:02:09.930135 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 02 10:02:09.930135 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 02 10:02:09.930135 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 02 10:02:09.930135 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 02 10:02:09.931196 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 02 10:02:09.931196 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 02 10:02:09.931196 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 02 10:02:09.931196 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 02 10:02:09.931196 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 02 10:02:09.931196 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 02 10:02:09.931196 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application return self.get_or_create( Aug 02 10:02:09.931196 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 02 10:02:09.931196 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application with Lock( Aug 02 10:02:09.931196 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 02 10:02:09.931196 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application return self._enter() Aug 02 10:02:09.931196 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 02 10:02:09.931196 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 02 10:02:09.931196 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 02 10:02:09.931196 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application return self.creator() Aug 02 10:02:09.931196 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 02 10:02:09.931196 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application created_value = creator( Aug 02 10:02:09.932212 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 02 10:02:09.932212 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 02 10:02:09.932212 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 02 10:02:09.932212 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application Aug 02 10:02:09.932436 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e04dedac-0cd2-44bc-9d56-5b1ef4fbbeba None None] REQUEST_METHOD: `POST` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:02:09.932436 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e04dedac-0cd2-44bc-9d56-5b1ef4fbbeba None None] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:02:09.932436 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e04dedac-0cd2-44bc-9d56-5b1ef4fbbeba None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:02:09.937679 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: [pid: 80377|app: 0|req: 240/473] 10.222.0.82 () {56 vars in 1108 bytes} [Tue Aug 2 10:02:09 2022] GET /identity/v3/domains/Default => generated 87 bytes in 322 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 02 10:02:09.937679 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-253e52c0-038c-49ce-a4ff-eb69924422b6 None admin] Authenticating user token {{(pid=80377) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:02:09.977433 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.auth.core [None req-e04dedac-0cd2-44bc-9d56-5b1ef4fbbeba None None] MFA Rules not processed for user `891005fa5ff74041b75747d16cc66798`. Rule list: `[]` (Enabled: `True`). {{(pid=80378) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 02 10:02:09.979545 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.fernet_utils [None req-e04dedac-0cd2-44bc-9d56-5b1ef4fbbeba None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80378) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 10:02:10.117049 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 234/474] 10.222.0.82 () {58 vars in 1023 bytes} [Tue Aug 2 10:02:09 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 193 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 02 10:02:10.118638 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e04dedac-0cd2-44bc-9d56-5b1ef4fbbeba None None] Authenticating user token {{(pid=80378) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:02:10.134986 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG dbcounter [-] [80378] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80378) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 10:02:10.151972 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-253e52c0-038c-49ce-a4ff-eb69924422b6 None admin] Validating token access rules against request {{(pid=80377) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:02:10.201322 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-953f82e8-1d07-47e3-89ab-312c60a11118 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '891005fa5ff74041b75747d16cc66798', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80377) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:02:10.201992 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-953f82e8-1d07-47e3-89ab-312c60a11118 None admin] REQUEST_METHOD: `POST` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:02:10.202321 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-953f82e8-1d07-47e3-89ab-312c60a11118 None admin] SCRIPT_NAME: `/identity` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:02:10.202634 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-953f82e8-1d07-47e3-89ab-312c60a11118 None admin] PATH_INFO: `/v3/users` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:02:10.203566 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-953f82e8-1d07-47e3-89ab-312c60a11118 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:02:10.206309 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-953f82e8-1d07-47e3-89ab-312c60a11118 None admin] RBAC: Authorization granted {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:02:10.244054 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: [pid: 80377|app: 0|req: 241/475] 10.222.0.82 () {60 vars in 1132 bytes} [Tue Aug 2 10:02:09 2022] POST /identity/v3/users => generated 248 bytes in 310 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 02 10:02:10.245178 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-953f82e8-1d07-47e3-89ab-312c60a11118 None admin] Authenticating user token {{(pid=80377) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:02:10.286346 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG dbcounter [-] [80377] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80377) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 10:02:10.349464 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e04dedac-0cd2-44bc-9d56-5b1ef4fbbeba None None] Validating token access rules against request {{(pid=80378) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:02:10.414508 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd5ae396-8862-41d1-8113-0bdfda1f0ebd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '891005fa5ff74041b75747d16cc66798', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80378) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:02:10.415893 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd5ae396-8862-41d1-8113-0bdfda1f0ebd None admin] REQUEST_METHOD: `GET` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:02:10.416562 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd5ae396-8862-41d1-8113-0bdfda1f0ebd None admin] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:02:10.417221 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd5ae396-8862-41d1-8113-0bdfda1f0ebd None admin] PATH_INFO: `/v3/domains` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:02:10.418733 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd5ae396-8862-41d1-8113-0bdfda1f0ebd None admin] RBAC: Adding query filter params (name=Default) {{(pid=80378) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 02 10:02:10.419488 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd5ae396-8862-41d1-8113-0bdfda1f0ebd None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:02:10.422124 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd5ae396-8862-41d1-8113-0bdfda1f0ebd None admin] RBAC: Authorization granted {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:02:10.444238 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 235/476] 10.222.0.82 () {56 vars in 1109 bytes} [Tue Aug 2 10:02:10 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 327 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 02 10:02:10.456201 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd5ae396-8862-41d1-8113-0bdfda1f0ebd None admin] Authenticating user token {{(pid=80378) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:02:10.508537 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-953f82e8-1d07-47e3-89ab-312c60a11118 None admin] Validating token access rules against request {{(pid=80377) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:02:10.570959 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c166cc72-fe40-45bb-9524-95f7b13f2f50 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '891005fa5ff74041b75747d16cc66798', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80377) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:02:10.571960 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c166cc72-fe40-45bb-9524-95f7b13f2f50 None admin] REQUEST_METHOD: `POST` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:02:10.572434 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c166cc72-fe40-45bb-9524-95f7b13f2f50 None admin] SCRIPT_NAME: `/identity` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:02:10.572904 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c166cc72-fe40-45bb-9524-95f7b13f2f50 None admin] PATH_INFO: `/v3/roles` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:02:10.574160 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c166cc72-fe40-45bb-9524-95f7b13f2f50 None admin] RBAC: Authorizing `identity:create_role()` {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:02:10.576530 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c166cc72-fe40-45bb-9524-95f7b13f2f50 None admin] RBAC: Authorization granted {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:02:10.589831 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.sql.core [None req-c166cc72-fe40-45bb-9524-95f7b13f2f50 None admin] Conflict role: (pymysql.err.IntegrityError) (1062, "Duplicate entry 'anotherrole-<>' for key 'role.ixu_role_name_domain_id'") Aug 02 10:02:10.589831 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: [SQL: INSERT INTO `role` (id, name, description, extra) VALUES (%(id)s, %(name)s, %(description)s, %(extra)s)] Aug 02 10:02:10.589831 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: [parameters: {'id': '532b9fbef3ca41e19a9cc5c052e93cd9', 'name': 'anotherrole', 'description': None, 'extra': '{}'}] Aug 02 10:02:10.589831 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: (Background on this error at: https://sqlalche.me/e/14/gkpj) {{(pid=80377) wrapper /opt/stack/keystone/keystone/common/sql/core.py:533}} Aug 02 10:02:10.641095 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application [None req-c166cc72-fe40-45bb-9524-95f7b13f2f50 None admin] Conflict occurred attempting to store role - Duplicate entry found with name anotherrole.: keystone.exception.Conflict: Conflict occurred attempting to store role - Duplicate entry found with name anotherrole. Aug 02 10:02:10.641095 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 02 10:02:10.641095 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1819, in _execute_context Aug 02 10:02:10.641095 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application self.dialect.do_execute( Aug 02 10:02:10.641095 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/default.py", line 732, in do_execute Aug 02 10:02:10.641095 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application cursor.execute(statement, parameters) Aug 02 10:02:10.641095 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 148, in execute Aug 02 10:02:10.641095 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application result = self._query(query) Aug 02 10:02:10.641095 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 310, in _query Aug 02 10:02:10.641095 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application conn.query(q) Aug 02 10:02:10.641095 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 548, in query Aug 02 10:02:10.641095 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application self._affected_rows = self._read_query_result(unbuffered=unbuffered) Aug 02 10:02:10.641095 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 775, in _read_query_result Aug 02 10:02:10.641095 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application result.read() Aug 02 10:02:10.641095 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 1156, in read Aug 02 10:02:10.643159 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application first_packet = self.connection._read_packet() Aug 02 10:02:10.643159 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 725, in _read_packet Aug 02 10:02:10.643159 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application packet.raise_for_error() Aug 02 10:02:10.643159 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/protocol.py", line 221, in raise_for_error Aug 02 10:02:10.643159 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application err.raise_mysql_exception(self._data) Aug 02 10:02:10.643159 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/err.py", line 143, in raise_mysql_exception Aug 02 10:02:10.643159 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application raise errorclass(errno, errval) Aug 02 10:02:10.643159 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application pymysql.err.IntegrityError: (1062, "Duplicate entry 'anotherrole-<>' for key 'role.ixu_role_name_domain_id'") Aug 02 10:02:10.643159 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application Aug 02 10:02:10.643159 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application The above exception was the direct cause of the following exception: Aug 02 10:02:10.643159 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application Aug 02 10:02:10.643159 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 02 10:02:10.643159 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/sql/core.py", line 528, in wrapper Aug 02 10:02:10.643159 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application return method(*args, **kwargs) Aug 02 10:02:10.643159 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 33, in create_role Aug 02 10:02:10.643159 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application return ref.to_dict() Aug 02 10:02:10.643159 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ Aug 02 10:02:10.643159 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application next(self.gen) Aug 02 10:02:10.644929 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 1060, in _transaction_scope Aug 02 10:02:10.644929 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application yield resource Aug 02 10:02:10.644929 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ Aug 02 10:02:10.644929 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application next(self.gen) Aug 02 10:02:10.644929 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 662, in _session Aug 02 10:02:10.644929 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application self.session.rollback() Aug 02 10:02:10.644929 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 227, in __exit__ Aug 02 10:02:10.644929 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application self.force_reraise() Aug 02 10:02:10.644929 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 200, in force_reraise Aug 02 10:02:10.644929 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application raise self.value Aug 02 10:02:10.644929 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 659, in _session Aug 02 10:02:10.644929 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application self._end_session_transaction(self.session) Aug 02 10:02:10.644929 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 687, in _end_session_transaction Aug 02 10:02:10.644929 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application session.commit() Aug 02 10:02:10.644929 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 1451, in commit Aug 02 10:02:10.644929 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application self._transaction.commit(_to_root=self.future) Aug 02 10:02:10.644929 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 829, in commit Aug 02 10:02:10.648022 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application self._prepare_impl() Aug 02 10:02:10.648022 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 808, in _prepare_impl Aug 02 10:02:10.648022 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application self.session.flush() Aug 02 10:02:10.648022 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3383, in flush Aug 02 10:02:10.648022 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application self._flush(objects) Aug 02 10:02:10.648022 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3523, in _flush Aug 02 10:02:10.648022 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application transaction.rollback(_capture_exception=True) Aug 02 10:02:10.648022 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/langhelpers.py", line 70, in __exit__ Aug 02 10:02:10.648022 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application compat.raise_( Aug 02 10:02:10.648022 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/compat.py", line 208, in raise_ Aug 02 10:02:10.648022 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application raise exception Aug 02 10:02:10.648022 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3483, in _flush Aug 02 10:02:10.648022 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application flush_context.execute() Aug 02 10:02:10.648022 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/unitofwork.py", line 456, in execute Aug 02 10:02:10.648022 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application rec.execute(self) Aug 02 10:02:10.648022 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/unitofwork.py", line 630, in execute Aug 02 10:02:10.648022 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application util.preloaded.orm_persistence.save_obj( Aug 02 10:02:10.651352 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/persistence.py", line 245, in save_obj Aug 02 10:02:10.651352 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application _emit_insert_statements( Aug 02 10:02:10.651352 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/persistence.py", line 1097, in _emit_insert_statements Aug 02 10:02:10.651352 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application c = connection._execute_20( Aug 02 10:02:10.651352 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1631, in _execute_20 Aug 02 10:02:10.651352 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application return meth(self, args_10style, kwargs_10style, execution_options) Aug 02 10:02:10.651352 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/sql/elements.py", line 332, in _execute_on_connection Aug 02 10:02:10.651352 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application return connection._execute_clauseelement( Aug 02 10:02:10.651352 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1498, in _execute_clauseelement Aug 02 10:02:10.651352 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application ret = self._execute_context( Aug 02 10:02:10.651352 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1862, in _execute_context Aug 02 10:02:10.651352 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application self._handle_dbapi_exception( Aug 02 10:02:10.651352 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 2041, in _handle_dbapi_exception Aug 02 10:02:10.651352 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application util.raise_(newraise, with_traceback=exc_info[2], from_=e) Aug 02 10:02:10.651352 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/compat.py", line 208, in raise_ Aug 02 10:02:10.651352 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application raise exception Aug 02 10:02:10.656659 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1819, in _execute_context Aug 02 10:02:10.656659 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application self.dialect.do_execute( Aug 02 10:02:10.656659 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/default.py", line 732, in do_execute Aug 02 10:02:10.656659 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application cursor.execute(statement, parameters) Aug 02 10:02:10.656659 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 148, in execute Aug 02 10:02:10.656659 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application result = self._query(query) Aug 02 10:02:10.656659 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 310, in _query Aug 02 10:02:10.656659 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application conn.query(q) Aug 02 10:02:10.656659 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 548, in query Aug 02 10:02:10.656659 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application self._affected_rows = self._read_query_result(unbuffered=unbuffered) Aug 02 10:02:10.656659 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 775, in _read_query_result Aug 02 10:02:10.656659 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application result.read() Aug 02 10:02:10.656659 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 1156, in read Aug 02 10:02:10.656659 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application first_packet = self.connection._read_packet() Aug 02 10:02:10.656659 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 725, in _read_packet Aug 02 10:02:10.656659 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application packet.raise_for_error() Aug 02 10:02:10.660614 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/protocol.py", line 221, in raise_for_error Aug 02 10:02:10.660614 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application err.raise_mysql_exception(self._data) Aug 02 10:02:10.660614 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/err.py", line 143, in raise_mysql_exception Aug 02 10:02:10.660614 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application raise errorclass(errno, errval) Aug 02 10:02:10.660614 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application oslo_db.exception.DBDuplicateEntry: (pymysql.err.IntegrityError) (1062, "Duplicate entry 'anotherrole-<>' for key 'role.ixu_role_name_domain_id'") Aug 02 10:02:10.660614 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application [SQL: INSERT INTO `role` (id, name, description, extra) VALUES (%(id)s, %(name)s, %(description)s, %(extra)s)] Aug 02 10:02:10.660614 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application [parameters: {'id': '532b9fbef3ca41e19a9cc5c052e93cd9', 'name': 'anotherrole', 'description': None, 'extra': '{}'}] Aug 02 10:02:10.660614 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application (Background on this error at: https://sqlalche.me/e/14/gkpj) Aug 02 10:02:10.660614 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application Aug 02 10:02:10.660614 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 02 10:02:10.660614 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application Aug 02 10:02:10.660614 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 02 10:02:10.660614 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 02 10:02:10.660614 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 02 10:02:10.660614 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 02 10:02:10.660614 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 02 10:02:10.665680 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 02 10:02:10.665680 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 02 10:02:10.665680 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 02 10:02:10.665680 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 02 10:02:10.665680 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 02 10:02:10.665680 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 02 10:02:10.665680 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 115, in post Aug 02 10:02:10.665680 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application ref = PROVIDERS.role_api.create_role( Aug 02 10:02:10.665680 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 02 10:02:10.665680 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 02 10:02:10.665680 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1298, in create_role Aug 02 10:02:10.665680 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application ret = self.driver.create_role(role_id, role) Aug 02 10:02:10.665680 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/sql/core.py", line 563, in wrapper Aug 02 10:02:10.665680 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application raise exception.Conflict(type=conflict_type, Aug 02 10:02:10.665680 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application keystone.exception.Conflict: Conflict occurred attempting to store role - Duplicate entry found with name anotherrole. Aug 02 10:02:10.665680 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application Aug 02 10:02:10.670547 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: [pid: 80377|app: 0|req: 242/477] 10.222.0.82 () {60 vars in 1131 bytes} [Tue Aug 2 10:02:10 2022] POST /identity/v3/roles => generated 144 bytes in 397 msecs (HTTP/1.1 409) 5 headers in 183 bytes (1 switches on core 0) Aug 02 10:02:10.670547 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c166cc72-fe40-45bb-9524-95f7b13f2f50 None admin] Authenticating user token {{(pid=80377) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:02:10.707161 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG dbcounter [-] [80378] Writing DB stats keystone:SELECT=100 {{(pid=80378) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 10:02:10.766241 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd5ae396-8862-41d1-8113-0bdfda1f0ebd None admin] Validating token access rules against request {{(pid=80378) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:02:10.821309 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG dbcounter [-] [80377] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80377) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 10:02:10.880792 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c166cc72-fe40-45bb-9524-95f7b13f2f50 None admin] Validating token access rules against request {{(pid=80377) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:02:10.890381 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7330ef5-e09f-446f-a6cc-7454501b584c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '891005fa5ff74041b75747d16cc66798', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80378) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:02:10.891966 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7330ef5-e09f-446f-a6cc-7454501b584c None admin] REQUEST_METHOD: `POST` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:02:10.892754 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7330ef5-e09f-446f-a6cc-7454501b584c None admin] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:02:10.893525 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7330ef5-e09f-446f-a6cc-7454501b584c None admin] PATH_INFO: `/v3/users` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:02:10.895145 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7330ef5-e09f-446f-a6cc-7454501b584c None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:02:10.898322 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7330ef5-e09f-446f-a6cc-7454501b584c None admin] RBAC: Authorization granted {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:02:10.939938 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 236/478] 10.222.0.82 () {60 vars in 1132 bytes} [Tue Aug 2 10:02:10 2022] POST /identity/v3/users => generated 249 bytes in 485 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 02 10:02:10.954311 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfa80165-853a-4550-900d-c01f743c25f3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '891005fa5ff74041b75747d16cc66798', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80377) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:02:10.955880 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfa80165-853a-4550-900d-c01f743c25f3 None admin] REQUEST_METHOD: `GET` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:02:10.956750 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfa80165-853a-4550-900d-c01f743c25f3 None admin] SCRIPT_NAME: `/identity` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:02:10.957549 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfa80165-853a-4550-900d-c01f743c25f3 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:02:10.979948 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cfa80165-853a-4550-900d-c01f743c25f3 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 02 10:02:10.992270 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application [None req-cfa80165-853a-4550-900d-c01f743c25f3 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Aug 02 10:02:10.992270 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 02 10:02:10.992270 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 02 10:02:10.992270 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 02 10:02:10.992270 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 02 10:02:10.992270 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 02 10:02:10.992270 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 02 10:02:10.992270 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 02 10:02:10.992270 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 02 10:02:10.992270 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 02 10:02:10.992270 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 02 10:02:10.992270 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 02 10:02:10.992270 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 02 10:02:10.992270 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 02 10:02:10.992270 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 02 10:02:10.992270 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 02 10:02:10.994133 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 02 10:02:10.994133 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 02 10:02:10.994133 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 02 10:02:10.994133 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 02 10:02:10.994133 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 02 10:02:10.994133 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 02 10:02:10.994133 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 02 10:02:10.994133 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 02 10:02:10.994133 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 02 10:02:10.994133 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application return self.get_or_create( Aug 02 10:02:10.994133 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 02 10:02:10.994133 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application with Lock( Aug 02 10:02:10.994133 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 02 10:02:10.994133 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application return self._enter() Aug 02 10:02:10.994133 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 02 10:02:10.994133 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 02 10:02:10.995745 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 02 10:02:10.995745 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application return self.creator() Aug 02 10:02:10.995745 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 02 10:02:10.995745 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application created_value = creator( Aug 02 10:02:10.995745 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 02 10:02:10.995745 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 02 10:02:10.995745 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 02 10:02:10.995745 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 02 10:02:10.995745 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 02 10:02:10.995745 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 02 10:02:10.995745 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Aug 02 10:02:10.995745 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application Aug 02 10:02:11.000787 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: [pid: 80377|app: 0|req: 243/479] 10.222.0.82 () {56 vars in 1114 bytes} [Tue Aug 2 10:02:10 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 351 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 02 10:02:11.011858 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7330ef5-e09f-446f-a6cc-7454501b584c None admin] Authenticating user token {{(pid=80378) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:02:11.264376 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7330ef5-e09f-446f-a6cc-7454501b584c None admin] Validating token access rules against request {{(pid=80378) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:02:11.312325 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG dbcounter [-] [80378] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80378) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 10:02:11.334120 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65ca4e46-a26b-4d0c-ad93-06c568f5a2c3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '891005fa5ff74041b75747d16cc66798', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80378) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:02:11.334755 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65ca4e46-a26b-4d0c-ad93-06c568f5a2c3 None admin] REQUEST_METHOD: `GET` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:02:11.335039 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65ca4e46-a26b-4d0c-ad93-06c568f5a2c3 None admin] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:02:11.335309 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65ca4e46-a26b-4d0c-ad93-06c568f5a2c3 None admin] PATH_INFO: `/v3/roles` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:02:11.336037 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65ca4e46-a26b-4d0c-ad93-06c568f5a2c3 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=80378) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 02 10:02:11.336342 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65ca4e46-a26b-4d0c-ad93-06c568f5a2c3 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:02:11.337822 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65ca4e46-a26b-4d0c-ad93-06c568f5a2c3 None admin] RBAC: Authorization granted {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:02:11.349935 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 237/480] 10.222.0.82 () {56 vars in 1111 bytes} [Tue Aug 2 10:02:11 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 339 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 02 10:02:12.778791 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d7ef41e-2102-4545-a7aa-4cbc81a74f7c None None] REQUEST_METHOD: `GET` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:02:12.778791 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d7ef41e-2102-4545-a7aa-4cbc81a74f7c None None] SCRIPT_NAME: `/identity` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:02:12.778791 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d7ef41e-2102-4545-a7aa-4cbc81a74f7c None None] PATH_INFO: `/` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:02:12.782765 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: [pid: 80377|app: 0|req: 244/481] 10.222.0.82 () {54 vars in 926 bytes} [Tue Aug 2 10:02:12 2022] GET /identity => generated 270 bytes in 7 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 02 10:02:12.789935 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad438c59-a6f2-4134-a267-a24540969cab None None] REQUEST_METHOD: `POST` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:02:12.790711 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad438c59-a6f2-4134-a267-a24540969cab None None] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:02:12.790711 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad438c59-a6f2-4134-a267-a24540969cab None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:02:12.814990 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.auth.core [None req-ad438c59-a6f2-4134-a267-a24540969cab None None] MFA Rules not processed for user `891005fa5ff74041b75747d16cc66798`. Rule list: `[]` (Enabled: `True`). {{(pid=80378) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 02 10:02:12.816257 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.fernet_utils [None req-ad438c59-a6f2-4134-a267-a24540969cab None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80378) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 10:02:12.969377 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 238/482] 10.222.0.82 () {58 vars in 1023 bytes} [Tue Aug 2 10:02:12 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 180 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Aug 02 10:02:13.007932 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d7ef41e-2102-4545-a7aa-4cbc81a74f7c None None] Authenticating user token {{(pid=80377) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:02:13.271211 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d7ef41e-2102-4545-a7aa-4cbc81a74f7c None None] Validating token access rules against request {{(pid=80377) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:02:13.360024 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92b7693b-82a2-4058-b99d-186db3c6673b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '891005fa5ff74041b75747d16cc66798', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80377) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:02:13.361260 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92b7693b-82a2-4058-b99d-186db3c6673b None admin] REQUEST_METHOD: `GET` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:02:13.361540 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92b7693b-82a2-4058-b99d-186db3c6673b None admin] SCRIPT_NAME: `/identity` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:02:13.361830 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92b7693b-82a2-4058-b99d-186db3c6673b None admin] PATH_INFO: `/v3/roles/service` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:02:13.364308 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG dbcounter [-] [80377] Writing DB stats keystone:SELECT=100 {{(pid=80377) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 10:02:13.380958 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92b7693b-82a2-4058-b99d-186db3c6673b None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 02 10:02:13.391052 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application [None req-92b7693b-82a2-4058-b99d-186db3c6673b None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Aug 02 10:02:13.391052 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 02 10:02:13.391052 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 02 10:02:13.391052 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 02 10:02:13.391052 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 02 10:02:13.391052 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 02 10:02:13.391052 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 02 10:02:13.391052 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 02 10:02:13.391052 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 02 10:02:13.391052 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 02 10:02:13.391052 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 02 10:02:13.391052 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 02 10:02:13.391052 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 02 10:02:13.391052 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 02 10:02:13.391052 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 02 10:02:13.391052 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 02 10:02:13.393313 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 02 10:02:13.393313 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 02 10:02:13.393313 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 02 10:02:13.393313 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 02 10:02:13.393313 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 02 10:02:13.393313 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 02 10:02:13.393313 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 02 10:02:13.393313 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 02 10:02:13.393313 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 02 10:02:13.393313 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application return self.get_or_create( Aug 02 10:02:13.393313 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 02 10:02:13.393313 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application with Lock( Aug 02 10:02:13.393313 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 02 10:02:13.393313 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application return self._enter() Aug 02 10:02:13.393313 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 02 10:02:13.393313 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 02 10:02:13.395680 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 02 10:02:13.395680 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application return self.creator() Aug 02 10:02:13.395680 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 02 10:02:13.395680 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application created_value = creator( Aug 02 10:02:13.395680 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 02 10:02:13.395680 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 02 10:02:13.395680 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 02 10:02:13.395680 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 02 10:02:13.395680 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 02 10:02:13.395680 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 02 10:02:13.395680 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Aug 02 10:02:13.395680 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application Aug 02 10:02:13.400842 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: [pid: 80377|app: 0|req: 245/483] 10.222.0.82 () {56 vars in 1102 bytes} [Tue Aug 2 10:02:13 2022] GET /identity/v3/roles/service => generated 85 bytes in 387 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 02 10:02:13.402624 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad438c59-a6f2-4134-a267-a24540969cab None None] Authenticating user token {{(pid=80378) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:02:13.640325 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad438c59-a6f2-4134-a267-a24540969cab None None] Validating token access rules against request {{(pid=80378) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:02:13.669836 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG dbcounter [-] [80378] Writing DB stats keystone:SELECT=100 {{(pid=80378) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 10:02:13.719083 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b31f659-cfbb-4deb-9286-bd7e0f50fdb4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '891005fa5ff74041b75747d16cc66798', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80378) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:02:13.720708 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b31f659-cfbb-4deb-9286-bd7e0f50fdb4 None admin] REQUEST_METHOD: `GET` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:02:13.721221 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b31f659-cfbb-4deb-9286-bd7e0f50fdb4 None admin] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:02:13.721719 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b31f659-cfbb-4deb-9286-bd7e0f50fdb4 None admin] PATH_INFO: `/v3/roles` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:02:13.723100 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b31f659-cfbb-4deb-9286-bd7e0f50fdb4 None admin] RBAC: Adding query filter params (name=service) {{(pid=80378) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 02 10:02:13.723624 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b31f659-cfbb-4deb-9286-bd7e0f50fdb4 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:02:13.726206 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b31f659-cfbb-4deb-9286-bd7e0f50fdb4 None admin] RBAC: Authorization granted {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:02:13.728761 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee605fcc-9498-4215-9e64-b8211f785d69 None None] REQUEST_METHOD: `GET` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:02:13.729349 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee605fcc-9498-4215-9e64-b8211f785d69 None None] SCRIPT_NAME: `/identity` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:02:13.729896 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee605fcc-9498-4215-9e64-b8211f785d69 None None] PATH_INFO: `/` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:02:13.731749 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: [pid: 80377|app: 0|req: 246/484] 10.222.0.82 () {54 vars in 926 bytes} [Tue Aug 2 10:02:13 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 02 10:02:13.738579 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28fd5a54-35df-4b85-ae99-a7feed46932c None None] REQUEST_METHOD: `POST` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:02:13.739261 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28fd5a54-35df-4b85-ae99-a7feed46932c None None] SCRIPT_NAME: `/identity` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:02:13.739757 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28fd5a54-35df-4b85-ae99-a7feed46932c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:02:13.743281 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 239/485] 10.222.0.82 () {56 vars in 1103 bytes} [Tue Aug 2 10:02:13 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 342 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 02 10:02:13.752902 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b31f659-cfbb-4deb-9286-bd7e0f50fdb4 None admin] Authenticating user token {{(pid=80378) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:02:13.784382 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.auth.core [None req-28fd5a54-35df-4b85-ae99-a7feed46932c None None] MFA Rules not processed for user `891005fa5ff74041b75747d16cc66798`. Rule list: `[]` (Enabled: `True`). {{(pid=80377) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 02 10:02:13.786430 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.fernet_utils [None req-28fd5a54-35df-4b85-ae99-a7feed46932c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80377) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 10:02:14.025498 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b31f659-cfbb-4deb-9286-bd7e0f50fdb4 None admin] Validating token access rules against request {{(pid=80378) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:02:14.046609 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: [pid: 80377|app: 0|req: 247/486] 10.222.0.82 () {58 vars in 1023 bytes} [Tue Aug 2 10:02:13 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 309 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Aug 02 10:02:14.100439 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84c88f14-1e1b-4c55-8fbd-f44f93ed731e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '891005fa5ff74041b75747d16cc66798', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80378) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:02:14.102187 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84c88f14-1e1b-4c55-8fbd-f44f93ed731e None admin] REQUEST_METHOD: `GET` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:02:14.103008 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84c88f14-1e1b-4c55-8fbd-f44f93ed731e None admin] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:02:14.103793 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84c88f14-1e1b-4c55-8fbd-f44f93ed731e None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:02:14.127258 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84c88f14-1e1b-4c55-8fbd-f44f93ed731e None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:02:14.130265 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84c88f14-1e1b-4c55-8fbd-f44f93ed731e None admin] RBAC: Authorization granted {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:02:14.134770 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28fd5a54-35df-4b85-ae99-a7feed46932c None None] Authenticating user token {{(pid=80377) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:02:14.169688 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application [None req-84c88f14-1e1b-4c55-8fbd-f44f93ed731e None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 02 10:02:14.169688 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 02 10:02:14.169688 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 02 10:02:14.169688 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 02 10:02:14.169688 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 02 10:02:14.169688 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 02 10:02:14.169688 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 02 10:02:14.169688 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 02 10:02:14.169688 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 02 10:02:14.169688 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 02 10:02:14.169688 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 02 10:02:14.169688 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 02 10:02:14.169688 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 02 10:02:14.169688 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 02 10:02:14.169688 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 02 10:02:14.170668 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 02 10:02:14.170668 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 02 10:02:14.170668 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 02 10:02:14.170668 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 02 10:02:14.170668 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 02 10:02:14.170668 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 02 10:02:14.170668 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application return self.get_or_create( Aug 02 10:02:14.170668 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 02 10:02:14.170668 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application with Lock( Aug 02 10:02:14.170668 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 02 10:02:14.170668 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application return self._enter() Aug 02 10:02:14.170668 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 02 10:02:14.170668 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 02 10:02:14.170668 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 02 10:02:14.170668 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application return self.creator() Aug 02 10:02:14.170668 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 02 10:02:14.170668 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application created_value = creator( Aug 02 10:02:14.171685 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 02 10:02:14.171685 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 02 10:02:14.171685 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 02 10:02:14.171685 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application Aug 02 10:02:14.175586 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 240/487] 10.222.0.82 () {56 vars in 1108 bytes} [Tue Aug 2 10:02:13 2022] GET /identity/v3/domains/Default => generated 87 bytes in 425 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 02 10:02:14.190841 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84c88f14-1e1b-4c55-8fbd-f44f93ed731e None admin] Authenticating user token {{(pid=80378) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:02:14.242603 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG dbcounter [-] [80378] Writing DB stats keystone:SELECT=100 {{(pid=80378) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 10:02:14.371932 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84c88f14-1e1b-4c55-8fbd-f44f93ed731e None admin] Validating token access rules against request {{(pid=80378) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:02:14.436939 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-977164f2-09b0-411f-9f3d-57c0e5be1bbe None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '891005fa5ff74041b75747d16cc66798', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80378) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:02:14.438658 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-977164f2-09b0-411f-9f3d-57c0e5be1bbe None admin] REQUEST_METHOD: `GET` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:02:14.441158 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-977164f2-09b0-411f-9f3d-57c0e5be1bbe None admin] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:02:14.442646 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-977164f2-09b0-411f-9f3d-57c0e5be1bbe None admin] PATH_INFO: `/v3/domains` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:02:14.443832 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-977164f2-09b0-411f-9f3d-57c0e5be1bbe None admin] RBAC: Adding query filter params (name=Default) {{(pid=80378) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 02 10:02:14.444550 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-977164f2-09b0-411f-9f3d-57c0e5be1bbe None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:02:14.447705 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-977164f2-09b0-411f-9f3d-57c0e5be1bbe None admin] RBAC: Authorization granted {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:02:14.455129 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28fd5a54-35df-4b85-ae99-a7feed46932c None None] Validating token access rules against request {{(pid=80377) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:02:14.479887 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 241/488] 10.222.0.82 () {56 vars in 1109 bytes} [Tue Aug 2 10:02:14 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 290 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 02 10:02:14.490698 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-977164f2-09b0-411f-9f3d-57c0e5be1bbe None admin] Authenticating user token {{(pid=80378) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:02:14.547521 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG dbcounter [-] [80377] Writing DB stats keystone:SELECT=100 {{(pid=80377) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 10:02:14.595524 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36eefc3a-a74a-431c-8932-6c529484d870 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '891005fa5ff74041b75747d16cc66798', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80377) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:02:14.596614 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36eefc3a-a74a-431c-8932-6c529484d870 None admin] REQUEST_METHOD: `GET` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:02:14.597101 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36eefc3a-a74a-431c-8932-6c529484d870 None admin] SCRIPT_NAME: `/identity` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:02:14.597575 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36eefc3a-a74a-431c-8932-6c529484d870 None admin] PATH_INFO: `/v3/roles/service` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:02:14.625846 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36eefc3a-a74a-431c-8932-6c529484d870 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 02 10:02:14.631780 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application [None req-36eefc3a-a74a-431c-8932-6c529484d870 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Aug 02 10:02:14.631780 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 02 10:02:14.631780 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 02 10:02:14.631780 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 02 10:02:14.631780 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 02 10:02:14.631780 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 02 10:02:14.631780 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 02 10:02:14.631780 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 02 10:02:14.631780 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 02 10:02:14.631780 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 02 10:02:14.631780 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 02 10:02:14.631780 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 02 10:02:14.631780 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 02 10:02:14.631780 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 02 10:02:14.631780 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 02 10:02:14.631780 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 02 10:02:14.633496 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 02 10:02:14.633496 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 02 10:02:14.633496 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 02 10:02:14.633496 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 02 10:02:14.633496 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 02 10:02:14.633496 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 02 10:02:14.633496 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 02 10:02:14.633496 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 02 10:02:14.633496 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 02 10:02:14.633496 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application return self.get_or_create( Aug 02 10:02:14.633496 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 02 10:02:14.633496 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application with Lock( Aug 02 10:02:14.633496 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 02 10:02:14.633496 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application return self._enter() Aug 02 10:02:14.633496 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 02 10:02:14.633496 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 02 10:02:14.635044 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 02 10:02:14.635044 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application return self.creator() Aug 02 10:02:14.635044 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 02 10:02:14.635044 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application created_value = creator( Aug 02 10:02:14.635044 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 02 10:02:14.635044 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 02 10:02:14.635044 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 02 10:02:14.635044 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 02 10:02:14.635044 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 02 10:02:14.635044 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 02 10:02:14.635044 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Aug 02 10:02:14.635044 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application Aug 02 10:02:14.637460 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: [pid: 80377|app: 0|req: 248/489] 10.222.0.82 () {56 vars in 1102 bytes} [Tue Aug 2 10:02:14 2022] GET /identity/v3/roles/service => generated 85 bytes in 499 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 02 10:02:14.639961 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-818e7208-78da-4172-9b18-dead68d91c94 None None] REQUEST_METHOD: `GET` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:02:14.639961 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-818e7208-78da-4172-9b18-dead68d91c94 None None] SCRIPT_NAME: `/identity` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:02:14.640290 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-818e7208-78da-4172-9b18-dead68d91c94 None None] PATH_INFO: `/` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:02:14.644655 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: [pid: 80377|app: 0|req: 249/490] 10.222.0.82 () {54 vars in 926 bytes} [Tue Aug 2 10:02:14 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 02 10:02:14.646038 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-818e7208-78da-4172-9b18-dead68d91c94 None None] Authenticating user token {{(pid=80377) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:02:14.673361 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG dbcounter [-] [80378] Writing DB stats keystone:SELECT=100 {{(pid=80378) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 10:02:14.689402 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-977164f2-09b0-411f-9f3d-57c0e5be1bbe None admin] Validating token access rules against request {{(pid=80378) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:02:14.755036 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fab6ee4a-6a45-43ae-8087-19ffa836d90a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '891005fa5ff74041b75747d16cc66798', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80378) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:02:14.756297 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fab6ee4a-6a45-43ae-8087-19ffa836d90a None admin] REQUEST_METHOD: `GET` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:02:14.757003 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fab6ee4a-6a45-43ae-8087-19ffa836d90a None admin] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:02:14.757475 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fab6ee4a-6a45-43ae-8087-19ffa836d90a None admin] PATH_INFO: `/v3/users/nova` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:02:14.774409 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fab6ee4a-6a45-43ae-8087-19ffa836d90a None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:02:14.777346 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fab6ee4a-6a45-43ae-8087-19ffa836d90a None admin] RBAC: Authorization granted {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:02:14.794752 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application [None req-fab6ee4a-6a45-43ae-8087-19ffa836d90a None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Aug 02 10:02:14.794752 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 02 10:02:14.794752 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 02 10:02:14.794752 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 02 10:02:14.794752 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 02 10:02:14.794752 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 02 10:02:14.794752 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 02 10:02:14.794752 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 02 10:02:14.794752 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 02 10:02:14.794752 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 02 10:02:14.794752 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 02 10:02:14.794752 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 02 10:02:14.794752 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 02 10:02:14.794752 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 02 10:02:14.794752 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 02 10:02:14.794752 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 02 10:02:14.796927 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 02 10:02:14.796927 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 02 10:02:14.796927 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 02 10:02:14.796927 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 02 10:02:14.796927 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 02 10:02:14.796927 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 02 10:02:14.796927 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 02 10:02:14.796927 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 02 10:02:14.796927 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 02 10:02:14.796927 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application return self.get_or_create( Aug 02 10:02:14.796927 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 02 10:02:14.796927 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application with Lock( Aug 02 10:02:14.796927 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 02 10:02:14.796927 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application return self._enter() Aug 02 10:02:14.796927 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 02 10:02:14.796927 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 02 10:02:14.796927 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 02 10:02:14.796927 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application return self.creator() Aug 02 10:02:14.799468 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 02 10:02:14.799468 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application created_value = creator( Aug 02 10:02:14.799468 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 02 10:02:14.799468 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 02 10:02:14.799468 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 02 10:02:14.799468 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 02 10:02:14.799468 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 02 10:02:14.799468 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 02 10:02:14.799468 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Aug 02 10:02:14.799468 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application Aug 02 10:02:14.799468 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 242/491] 10.222.0.82 () {56 vars in 1093 bytes} [Tue Aug 2 10:02:14 2022] GET /identity/v3/users/nova => generated 82 bytes in 306 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 02 10:02:14.799468 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa933657-2df7-4785-9263-4c6e13469274 None None] REQUEST_METHOD: `POST` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:02:14.799468 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa933657-2df7-4785-9263-4c6e13469274 None None] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:02:14.802069 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa933657-2df7-4785-9263-4c6e13469274 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:02:14.852405 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.auth.core [None req-fa933657-2df7-4785-9263-4c6e13469274 None None] MFA Rules not processed for user `891005fa5ff74041b75747d16cc66798`. Rule list: `[]` (Enabled: `True`). {{(pid=80378) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 02 10:02:14.857971 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.fernet_utils [None req-fa933657-2df7-4785-9263-4c6e13469274 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80378) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 10:02:14.868129 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-818e7208-78da-4172-9b18-dead68d91c94 None None] Validating token access rules against request {{(pid=80377) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:02:14.957063 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f202d8fd-d615-40e8-97bb-58561e5acb86 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '891005fa5ff74041b75747d16cc66798', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80377) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:02:14.958293 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f202d8fd-d615-40e8-97bb-58561e5acb86 None admin] REQUEST_METHOD: `GET` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:02:14.958982 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f202d8fd-d615-40e8-97bb-58561e5acb86 None admin] SCRIPT_NAME: `/identity` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:02:14.959614 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f202d8fd-d615-40e8-97bb-58561e5acb86 None admin] PATH_INFO: `/v3/roles` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:02:14.961119 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f202d8fd-d615-40e8-97bb-58561e5acb86 None admin] RBAC: Adding query filter params (name=service) {{(pid=80377) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 02 10:02:14.961926 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f202d8fd-d615-40e8-97bb-58561e5acb86 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:02:14.964763 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f202d8fd-d615-40e8-97bb-58561e5acb86 None admin] RBAC: Authorization granted {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:02:14.981287 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: [pid: 80377|app: 0|req: 250/492] 10.222.0.82 () {56 vars in 1103 bytes} [Tue Aug 2 10:02:14 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 336 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 02 10:02:14.983794 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f202d8fd-d615-40e8-97bb-58561e5acb86 None admin] Authenticating user token {{(pid=80377) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:02:15.017895 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 243/493] 10.222.0.82 () {58 vars in 1023 bytes} [Tue Aug 2 10:02:14 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 221 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 02 10:02:15.019893 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa933657-2df7-4785-9263-4c6e13469274 None None] Authenticating user token {{(pid=80378) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:02:15.121205 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG dbcounter [-] [80377] Writing DB stats keystone:SELECT=100 {{(pid=80377) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 10:02:15.205507 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG dbcounter [-] [80378] Writing DB stats keystone:SELECT=100 {{(pid=80378) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 10:02:15.241955 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa933657-2df7-4785-9263-4c6e13469274 None None] Validating token access rules against request {{(pid=80378) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:02:15.244048 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f202d8fd-d615-40e8-97bb-58561e5acb86 None admin] Validating token access rules against request {{(pid=80377) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:02:15.299254 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21f86cb7-0d00-42a8-bdaa-059001e3089a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '891005fa5ff74041b75747d16cc66798', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80377) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:02:15.300264 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21f86cb7-0d00-42a8-bdaa-059001e3089a None admin] REQUEST_METHOD: `GET` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:02:15.300716 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21f86cb7-0d00-42a8-bdaa-059001e3089a None admin] SCRIPT_NAME: `/identity` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:02:15.301226 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21f86cb7-0d00-42a8-bdaa-059001e3089a None admin] PATH_INFO: `/v3/users` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:02:15.302606 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21f86cb7-0d00-42a8-bdaa-059001e3089a None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=80377) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 02 10:02:15.303108 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21f86cb7-0d00-42a8-bdaa-059001e3089a None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:02:15.304570 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4591817f-20af-4c67-945e-10f257eb5d09 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '891005fa5ff74041b75747d16cc66798', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80378) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:02:15.305127 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21f86cb7-0d00-42a8-bdaa-059001e3089a None admin] RBAC: Authorization granted {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:02:15.305396 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4591817f-20af-4c67-945e-10f257eb5d09 None admin] REQUEST_METHOD: `GET` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:02:15.305888 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4591817f-20af-4c67-945e-10f257eb5d09 None admin] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:02:15.306246 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4591817f-20af-4c67-945e-10f257eb5d09 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:02:15.325500 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4591817f-20af-4c67-945e-10f257eb5d09 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:02:15.328066 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4591817f-20af-4c67-945e-10f257eb5d09 None admin] RBAC: Authorization granted {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:02:15.354762 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: [pid: 80377|app: 0|req: 251/494] 10.222.0.82 () {56 vars in 1133 bytes} [Tue Aug 2 10:02:14 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 373 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 02 10:02:15.355853 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21f86cb7-0d00-42a8-bdaa-059001e3089a None admin] Authenticating user token {{(pid=80377) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:02:15.356294 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application [None req-4591817f-20af-4c67-945e-10f257eb5d09 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 02 10:02:15.356294 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 02 10:02:15.356294 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 02 10:02:15.356294 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 02 10:02:15.356294 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 02 10:02:15.356294 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 02 10:02:15.356294 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 02 10:02:15.356294 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 02 10:02:15.356294 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 02 10:02:15.356294 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 02 10:02:15.356294 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 02 10:02:15.356294 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 02 10:02:15.356294 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 02 10:02:15.356294 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 02 10:02:15.356294 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 02 10:02:15.357159 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 02 10:02:15.357159 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 02 10:02:15.357159 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 02 10:02:15.357159 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 02 10:02:15.357159 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 02 10:02:15.357159 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 02 10:02:15.357159 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application return self.get_or_create( Aug 02 10:02:15.357159 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 02 10:02:15.357159 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application with Lock( Aug 02 10:02:15.357159 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 02 10:02:15.357159 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application return self._enter() Aug 02 10:02:15.357159 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 02 10:02:15.357159 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 02 10:02:15.357159 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 02 10:02:15.357159 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application return self.creator() Aug 02 10:02:15.357159 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 02 10:02:15.357159 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application created_value = creator( Aug 02 10:02:15.358180 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 02 10:02:15.358180 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 02 10:02:15.358180 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 02 10:02:15.358180 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application Aug 02 10:02:15.358180 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 244/495] 10.222.0.82 () {56 vars in 1108 bytes} [Tue Aug 2 10:02:15 2022] GET /identity/v3/domains/Default => generated 87 bytes in 339 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 02 10:02:15.359569 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc32a855-eb31-4e8f-bc08-a9c743097d8b None None] REQUEST_METHOD: `GET` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:02:15.359871 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc32a855-eb31-4e8f-bc08-a9c743097d8b None None] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:02:15.360194 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc32a855-eb31-4e8f-bc08-a9c743097d8b None None] PATH_INFO: `/` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:02:15.369167 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 245/496] 10.222.0.82 () {54 vars in 926 bytes} [Tue Aug 2 10:02:15 2022] GET /identity => generated 270 bytes in 11 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 02 10:02:15.370673 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc32a855-eb31-4e8f-bc08-a9c743097d8b None None] Authenticating user token {{(pid=80378) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:02:15.527369 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21f86cb7-0d00-42a8-bdaa-059001e3089a None admin] Validating token access rules against request {{(pid=80377) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:02:15.530162 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG dbcounter [-] [80377] Writing DB stats keystone:SELECT=100 {{(pid=80377) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 10:02:15.535699 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc32a855-eb31-4e8f-bc08-a9c743097d8b None None] Validating token access rules against request {{(pid=80378) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:02:15.576385 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2cc62da5-f420-4ae3-8ee4-76a79a1aa58c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '891005fa5ff74041b75747d16cc66798', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80377) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:02:15.577474 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cc62da5-f420-4ae3-8ee4-76a79a1aa58c None admin] REQUEST_METHOD: `GET` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:02:15.578021 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cc62da5-f420-4ae3-8ee4-76a79a1aa58c None admin] SCRIPT_NAME: `/identity` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:02:15.578517 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cc62da5-f420-4ae3-8ee4-76a79a1aa58c None admin] PATH_INFO: `/v3/roles/service` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:02:15.584315 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG dbcounter [-] [80378] Writing DB stats keystone:SELECT=100 {{(pid=80378) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 10:02:15.594279 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68fe3c09-0343-4ce8-b90d-53e75d063896 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '891005fa5ff74041b75747d16cc66798', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80378) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:02:15.595887 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68fe3c09-0343-4ce8-b90d-53e75d063896 None admin] REQUEST_METHOD: `GET` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:02:15.596362 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68fe3c09-0343-4ce8-b90d-53e75d063896 None admin] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:02:15.596837 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68fe3c09-0343-4ce8-b90d-53e75d063896 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:02:15.599712 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2cc62da5-f420-4ae3-8ee4-76a79a1aa58c None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 02 10:02:15.609956 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application [None req-2cc62da5-f420-4ae3-8ee4-76a79a1aa58c None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Aug 02 10:02:15.609956 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 02 10:02:15.609956 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 02 10:02:15.609956 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 02 10:02:15.609956 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 02 10:02:15.609956 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 02 10:02:15.609956 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 02 10:02:15.609956 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 02 10:02:15.609956 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 02 10:02:15.609956 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 02 10:02:15.609956 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 02 10:02:15.609956 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 02 10:02:15.609956 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 02 10:02:15.609956 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 02 10:02:15.609956 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 02 10:02:15.609956 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 02 10:02:15.611367 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 02 10:02:15.611367 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 02 10:02:15.611367 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 02 10:02:15.611367 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 02 10:02:15.611367 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 02 10:02:15.611367 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 02 10:02:15.611367 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 02 10:02:15.611367 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 02 10:02:15.611367 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 02 10:02:15.611367 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application return self.get_or_create( Aug 02 10:02:15.611367 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 02 10:02:15.611367 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application with Lock( Aug 02 10:02:15.611367 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 02 10:02:15.611367 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application return self._enter() Aug 02 10:02:15.611367 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 02 10:02:15.611367 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 02 10:02:15.612319 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 02 10:02:15.612319 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application return self.creator() Aug 02 10:02:15.612319 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 02 10:02:15.612319 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application created_value = creator( Aug 02 10:02:15.612319 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 02 10:02:15.612319 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 02 10:02:15.612319 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 02 10:02:15.612319 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 02 10:02:15.612319 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 02 10:02:15.612319 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 02 10:02:15.612319 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Aug 02 10:02:15.612319 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application Aug 02 10:02:15.614031 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: [pid: 80377|app: 0|req: 252/497] 10.222.0.82 () {56 vars in 1102 bytes} [Tue Aug 2 10:02:15 2022] GET /identity/v3/roles/service => generated 85 bytes in 258 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 02 10:02:15.616336 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8dcd5d48-2906-4ad1-968b-14cde4e311b0 None None] REQUEST_METHOD: `POST` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:02:15.616826 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8dcd5d48-2906-4ad1-968b-14cde4e311b0 None None] SCRIPT_NAME: `/identity` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:02:15.617476 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8dcd5d48-2906-4ad1-968b-14cde4e311b0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:02:15.645721 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.auth.core [None req-8dcd5d48-2906-4ad1-968b-14cde4e311b0 None None] MFA Rules not processed for user `891005fa5ff74041b75747d16cc66798`. Rule list: `[]` (Enabled: `True`). {{(pid=80377) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 02 10:02:15.646973 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.fernet_utils [None req-8dcd5d48-2906-4ad1-968b-14cde4e311b0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80377) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 10:02:15.705730 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68fe3c09-0343-4ce8-b90d-53e75d063896 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:02:15.707866 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68fe3c09-0343-4ce8-b90d-53e75d063896 None admin] RBAC: Authorization granted {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:02:15.742949 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: [pid: 80377|app: 0|req: 253/498] 10.222.0.82 () {58 vars in 1023 bytes} [Tue Aug 2 10:02:15 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 128 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 02 10:02:15.743834 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8dcd5d48-2906-4ad1-968b-14cde4e311b0 None None] Authenticating user token {{(pid=80377) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:02:15.852404 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 246/499] 10.222.0.82 () {58 vars in 1282 bytes} [Tue Aug 2 10:02:15 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 482 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 02 10:02:15.854652 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44e9363b-53de-4757-95ff-bd6bd02b756f None None] REQUEST_METHOD: `GET` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:02:15.854652 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44e9363b-53de-4757-95ff-bd6bd02b756f None None] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:02:15.855091 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44e9363b-53de-4757-95ff-bd6bd02b756f None None] PATH_INFO: `/` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:02:15.856818 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 247/500] 10.222.0.82 () {54 vars in 926 bytes} [Tue Aug 2 10:02:15 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 02 10:02:15.858659 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44e9363b-53de-4757-95ff-bd6bd02b756f None None] Authenticating user token {{(pid=80378) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:02:15.903229 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG dbcounter [-] [80378] Writing DB stats keystone:SELECT=100 {{(pid=80378) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 10:02:15.917671 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG dbcounter [-] [80377] Writing DB stats keystone:SELECT=100 {{(pid=80377) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 10:02:15.931580 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8dcd5d48-2906-4ad1-968b-14cde4e311b0 None None] Validating token access rules against request {{(pid=80377) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:02:15.977554 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26b34c52-1f48-4bca-9966-8671e683f087 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '891005fa5ff74041b75747d16cc66798', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80377) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:02:15.978356 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26b34c52-1f48-4bca-9966-8671e683f087 None admin] REQUEST_METHOD: `GET` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:02:15.978807 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26b34c52-1f48-4bca-9966-8671e683f087 None admin] SCRIPT_NAME: `/identity` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:02:15.979191 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26b34c52-1f48-4bca-9966-8671e683f087 None admin] PATH_INFO: `/v3/domains` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:02:15.980006 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26b34c52-1f48-4bca-9966-8671e683f087 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80377) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 02 10:02:15.980454 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26b34c52-1f48-4bca-9966-8671e683f087 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:02:15.982517 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26b34c52-1f48-4bca-9966-8671e683f087 None admin] RBAC: Authorization granted {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:02:16.006276 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: [pid: 80377|app: 0|req: 254/501] 10.222.0.82 () {56 vars in 1109 bytes} [Tue Aug 2 10:02:15 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 257 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 02 10:02:16.006276 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26b34c52-1f48-4bca-9966-8671e683f087 None admin] Authenticating user token {{(pid=80377) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:02:16.024236 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44e9363b-53de-4757-95ff-bd6bd02b756f None None] Validating token access rules against request {{(pid=80378) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:02:16.076421 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c28c7fe-a782-487c-83b5-3c005a103160 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '891005fa5ff74041b75747d16cc66798', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80378) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:02:16.077213 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c28c7fe-a782-487c-83b5-3c005a103160 None admin] REQUEST_METHOD: `GET` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:02:16.077680 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c28c7fe-a782-487c-83b5-3c005a103160 None admin] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:02:16.078105 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c28c7fe-a782-487c-83b5-3c005a103160 None admin] PATH_INFO: `/v3/roles` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:02:16.079763 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c28c7fe-a782-487c-83b5-3c005a103160 None admin] RBAC: Adding query filter params (name=service) {{(pid=80378) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 02 10:02:16.080220 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c28c7fe-a782-487c-83b5-3c005a103160 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:02:16.081733 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c28c7fe-a782-487c-83b5-3c005a103160 None admin] RBAC: Authorization granted {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:02:16.091957 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 248/502] 10.222.0.82 () {56 vars in 1103 bytes} [Tue Aug 2 10:02:15 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 235 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 02 10:02:16.093366 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c28c7fe-a782-487c-83b5-3c005a103160 None admin] Authenticating user token {{(pid=80378) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:02:16.129946 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26b34c52-1f48-4bca-9966-8671e683f087 None admin] Validating token access rules against request {{(pid=80377) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:02:16.174899 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80fdd917-2545-462d-8615-9fb6375c4962 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '891005fa5ff74041b75747d16cc66798', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80377) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:02:16.175762 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80fdd917-2545-462d-8615-9fb6375c4962 None admin] REQUEST_METHOD: `GET` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:02:16.176159 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80fdd917-2545-462d-8615-9fb6375c4962 None admin] SCRIPT_NAME: `/identity` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:02:16.176597 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80fdd917-2545-462d-8615-9fb6375c4962 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:02:16.192429 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80fdd917-2545-462d-8615-9fb6375c4962 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:02:16.194915 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80fdd917-2545-462d-8615-9fb6375c4962 None admin] RBAC: Authorization granted {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:02:16.217743 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application [None req-80fdd917-2545-462d-8615-9fb6375c4962 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 02 10:02:16.217743 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 02 10:02:16.217743 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 02 10:02:16.217743 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 02 10:02:16.217743 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 02 10:02:16.217743 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 02 10:02:16.217743 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 02 10:02:16.217743 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 02 10:02:16.217743 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 02 10:02:16.217743 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 02 10:02:16.217743 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 02 10:02:16.217743 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 02 10:02:16.217743 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 02 10:02:16.217743 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 02 10:02:16.217743 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 02 10:02:16.219409 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 02 10:02:16.219409 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 02 10:02:16.219409 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 02 10:02:16.219409 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 02 10:02:16.219409 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 02 10:02:16.219409 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 02 10:02:16.219409 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application return self.get_or_create( Aug 02 10:02:16.219409 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 02 10:02:16.219409 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application with Lock( Aug 02 10:02:16.219409 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 02 10:02:16.219409 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application return self._enter() Aug 02 10:02:16.219409 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 02 10:02:16.219409 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 02 10:02:16.219409 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 02 10:02:16.219409 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application return self.creator() Aug 02 10:02:16.219409 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 02 10:02:16.219409 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application created_value = creator( Aug 02 10:02:16.221222 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 02 10:02:16.221222 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 02 10:02:16.221222 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 02 10:02:16.221222 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application Aug 02 10:02:16.221222 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: [pid: 80377|app: 0|req: 255/503] 10.222.0.82 () {56 vars in 1108 bytes} [Tue Aug 2 10:02:16 2022] GET /identity/v3/domains/Default => generated 87 bytes in 218 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 02 10:02:16.221222 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1f9a3ec-c5f6-49d4-9b94-86f2178fe6ce None None] REQUEST_METHOD: `POST` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:02:16.221912 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1f9a3ec-c5f6-49d4-9b94-86f2178fe6ce None None] SCRIPT_NAME: `/identity` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:02:16.221912 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1f9a3ec-c5f6-49d4-9b94-86f2178fe6ce None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:02:16.239540 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG dbcounter [-] [80377] Writing DB stats keystone:SELECT=100 {{(pid=80377) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 10:02:16.244972 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG dbcounter [-] [80378] Writing DB stats keystone:SELECT=100 {{(pid=80378) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 10:02:16.246754 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.auth.core [None req-c1f9a3ec-c5f6-49d4-9b94-86f2178fe6ce None None] MFA Rules not processed for user `891005fa5ff74041b75747d16cc66798`. Rule list: `[]` (Enabled: `True`). {{(pid=80377) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 02 10:02:16.248427 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.fernet_utils [None req-c1f9a3ec-c5f6-49d4-9b94-86f2178fe6ce None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80377) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 10:02:16.260724 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c28c7fe-a782-487c-83b5-3c005a103160 None admin] Validating token access rules against request {{(pid=80378) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:02:16.311606 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfa4cb98-93c3-4d7f-8964-a0403e898471 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '891005fa5ff74041b75747d16cc66798', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80378) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:02:16.312898 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfa4cb98-93c3-4d7f-8964-a0403e898471 None admin] REQUEST_METHOD: `GET` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:02:16.313528 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfa4cb98-93c3-4d7f-8964-a0403e898471 None admin] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:02:16.314547 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfa4cb98-93c3-4d7f-8964-a0403e898471 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:02:16.337555 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dfa4cb98-93c3-4d7f-8964-a0403e898471 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:02:16.339915 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dfa4cb98-93c3-4d7f-8964-a0403e898471 None admin] RBAC: Authorization granted {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:02:16.343418 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: [pid: 80377|app: 0|req: 256/504] 10.222.0.82 () {58 vars in 1023 bytes} [Tue Aug 2 10:02:16 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 124 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 02 10:02:16.346326 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1f9a3ec-c5f6-49d4-9b94-86f2178fe6ce None None] Authenticating user token {{(pid=80377) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:02:16.365081 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application [None req-dfa4cb98-93c3-4d7f-8964-a0403e898471 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 02 10:02:16.365081 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 02 10:02:16.365081 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 02 10:02:16.365081 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 02 10:02:16.365081 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 02 10:02:16.365081 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 02 10:02:16.365081 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 02 10:02:16.365081 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 02 10:02:16.365081 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 02 10:02:16.365081 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 02 10:02:16.365081 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 02 10:02:16.365081 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 02 10:02:16.365081 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 02 10:02:16.365081 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 02 10:02:16.365081 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 02 10:02:16.366730 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 02 10:02:16.366730 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 02 10:02:16.366730 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 02 10:02:16.366730 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 02 10:02:16.366730 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 02 10:02:16.366730 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 02 10:02:16.366730 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application return self.get_or_create( Aug 02 10:02:16.366730 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 02 10:02:16.366730 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application with Lock( Aug 02 10:02:16.366730 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 02 10:02:16.366730 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application return self._enter() Aug 02 10:02:16.366730 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 02 10:02:16.366730 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 02 10:02:16.366730 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 02 10:02:16.366730 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application return self.creator() Aug 02 10:02:16.366730 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 02 10:02:16.366730 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application created_value = creator( Aug 02 10:02:16.369144 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 02 10:02:16.369144 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 02 10:02:16.369144 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 02 10:02:16.369144 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application Aug 02 10:02:16.369144 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 249/505] 10.222.0.82 () {56 vars in 1108 bytes} [Tue Aug 2 10:02:16 2022] GET /identity/v3/domains/Default => generated 87 bytes in 274 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 02 10:02:16.369144 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfa4cb98-93c3-4d7f-8964-a0403e898471 None admin] Authenticating user token {{(pid=80378) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:02:16.520944 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1f9a3ec-c5f6-49d4-9b94-86f2178fe6ce None None] Validating token access rules against request {{(pid=80377) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:02:16.529430 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfa4cb98-93c3-4d7f-8964-a0403e898471 None admin] Validating token access rules against request {{(pid=80378) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:02:16.575435 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3637609-dc20-41c4-a283-1befebcc2be0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '891005fa5ff74041b75747d16cc66798', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80377) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:02:16.576478 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3637609-dc20-41c4-a283-1befebcc2be0 None admin] REQUEST_METHOD: `GET` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:02:16.576961 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3637609-dc20-41c4-a283-1befebcc2be0 None admin] SCRIPT_NAME: `/identity` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:02:16.577437 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3637609-dc20-41c4-a283-1befebcc2be0 None admin] PATH_INFO: `/v3/users/cinder` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:02:16.583405 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG dbcounter [-] [80377] Writing DB stats keystone:SELECT=100 {{(pid=80377) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 10:02:16.592456 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3637609-dc20-41c4-a283-1befebcc2be0 None admin] RBAC: Authorizing `identity:get_user(user_id=cinder)` {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:02:16.594792 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3637609-dc20-41c4-a283-1befebcc2be0 None admin] RBAC: Authorization granted {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:02:16.606060 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0498621e-c5c7-46fc-85a3-635f42483cfa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '891005fa5ff74041b75747d16cc66798', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80378) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:02:16.606803 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0498621e-c5c7-46fc-85a3-635f42483cfa None admin] REQUEST_METHOD: `GET` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:02:16.607085 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0498621e-c5c7-46fc-85a3-635f42483cfa None admin] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:02:16.607373 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0498621e-c5c7-46fc-85a3-635f42483cfa None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:02:16.617936 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application [None req-b3637609-dc20-41c4-a283-1befebcc2be0 None admin] Could not find user: cinder.: keystone.exception.UserNotFound: Could not find user: cinder. Aug 02 10:02:16.617936 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 02 10:02:16.617936 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 02 10:02:16.617936 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 02 10:02:16.617936 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 02 10:02:16.617936 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 02 10:02:16.617936 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 02 10:02:16.617936 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 02 10:02:16.617936 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 02 10:02:16.617936 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 02 10:02:16.617936 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 02 10:02:16.617936 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 02 10:02:16.617936 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 02 10:02:16.617936 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 02 10:02:16.617936 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 02 10:02:16.617936 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 02 10:02:16.619605 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 02 10:02:16.619605 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 02 10:02:16.619605 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 02 10:02:16.619605 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 02 10:02:16.619605 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 02 10:02:16.619605 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 02 10:02:16.619605 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 02 10:02:16.619605 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 02 10:02:16.619605 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 02 10:02:16.619605 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application return self.get_or_create( Aug 02 10:02:16.619605 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 02 10:02:16.619605 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application with Lock( Aug 02 10:02:16.619605 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 02 10:02:16.619605 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application return self._enter() Aug 02 10:02:16.619605 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 02 10:02:16.619605 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 02 10:02:16.619605 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 02 10:02:16.619605 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application return self.creator() Aug 02 10:02:16.622020 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 02 10:02:16.622020 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application created_value = creator( Aug 02 10:02:16.622020 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 02 10:02:16.622020 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 02 10:02:16.622020 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 02 10:02:16.622020 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 02 10:02:16.622020 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 02 10:02:16.622020 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 02 10:02:16.622020 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: cinder. Aug 02 10:02:16.622020 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application Aug 02 10:02:16.622020 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: [pid: 80377|app: 0|req: 257/506] 10.222.0.82 () {56 vars in 1099 bytes} [Tue Aug 2 10:02:16 2022] GET /identity/v3/users/cinder => generated 84 bytes in 275 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 02 10:02:16.622020 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3637609-dc20-41c4-a283-1befebcc2be0 None admin] Authenticating user token {{(pid=80377) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:02:16.622695 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0498621e-c5c7-46fc-85a3-635f42483cfa None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:02:16.626627 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0498621e-c5c7-46fc-85a3-635f42483cfa None admin] RBAC: Authorization granted {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:02:16.637162 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG dbcounter [-] [80378] Writing DB stats keystone:SELECT=100 {{(pid=80378) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 10:02:16.656127 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application [None req-0498621e-c5c7-46fc-85a3-635f42483cfa None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 02 10:02:16.656127 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 02 10:02:16.656127 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 02 10:02:16.656127 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 02 10:02:16.656127 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 02 10:02:16.656127 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 02 10:02:16.656127 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 02 10:02:16.656127 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 02 10:02:16.656127 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 02 10:02:16.656127 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 02 10:02:16.656127 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 02 10:02:16.656127 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 02 10:02:16.656127 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 02 10:02:16.656127 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 02 10:02:16.656127 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 02 10:02:16.659683 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 02 10:02:16.659683 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 02 10:02:16.659683 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 02 10:02:16.659683 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 02 10:02:16.659683 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 02 10:02:16.659683 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 02 10:02:16.659683 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application return self.get_or_create( Aug 02 10:02:16.659683 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 02 10:02:16.659683 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application with Lock( Aug 02 10:02:16.659683 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 02 10:02:16.659683 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application return self._enter() Aug 02 10:02:16.659683 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 02 10:02:16.659683 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 02 10:02:16.659683 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 02 10:02:16.659683 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application return self.creator() Aug 02 10:02:16.659683 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 02 10:02:16.659683 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application created_value = creator( Aug 02 10:02:16.661926 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 02 10:02:16.661926 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 02 10:02:16.661926 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 02 10:02:16.661926 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application Aug 02 10:02:16.661926 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 250/507] 10.222.0.82 () {56 vars in 1108 bytes} [Tue Aug 2 10:02:16 2022] GET /identity/v3/domains/Default => generated 87 bytes in 291 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 02 10:02:16.661926 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0498621e-c5c7-46fc-85a3-635f42483cfa None admin] Authenticating user token {{(pid=80378) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:02:16.793392 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3637609-dc20-41c4-a283-1befebcc2be0 None admin] Validating token access rules against request {{(pid=80377) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:02:16.839155 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0498621e-c5c7-46fc-85a3-635f42483cfa None admin] Validating token access rules against request {{(pid=80378) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:02:16.846416 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3ef0201-d88f-49e0-b51c-b83c3d22f816 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '891005fa5ff74041b75747d16cc66798', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80377) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:02:16.847809 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3ef0201-d88f-49e0-b51c-b83c3d22f816 None admin] REQUEST_METHOD: `GET` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:02:16.848228 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3ef0201-d88f-49e0-b51c-b83c3d22f816 None admin] SCRIPT_NAME: `/identity` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:02:16.848798 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3ef0201-d88f-49e0-b51c-b83c3d22f816 None admin] PATH_INFO: `/v3/domains` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:02:16.850251 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3ef0201-d88f-49e0-b51c-b83c3d22f816 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80377) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 02 10:02:16.850794 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3ef0201-d88f-49e0-b51c-b83c3d22f816 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:02:16.853257 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3ef0201-d88f-49e0-b51c-b83c3d22f816 None admin] RBAC: Authorization granted {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:02:16.871386 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: [pid: 80377|app: 0|req: 258/508] 10.222.0.82 () {56 vars in 1109 bytes} [Tue Aug 2 10:02:16 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 252 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 02 10:02:16.872304 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3ef0201-d88f-49e0-b51c-b83c3d22f816 None admin] Authenticating user token {{(pid=80377) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:02:16.896415 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d518e4a2-6844-4a6a-89be-e81cbb260a39 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '891005fa5ff74041b75747d16cc66798', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80378) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:02:16.897234 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d518e4a2-6844-4a6a-89be-e81cbb260a39 None admin] REQUEST_METHOD: `GET` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:02:16.897618 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d518e4a2-6844-4a6a-89be-e81cbb260a39 None admin] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:02:16.897929 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d518e4a2-6844-4a6a-89be-e81cbb260a39 None admin] PATH_INFO: `/v3/domains` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:02:16.898911 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d518e4a2-6844-4a6a-89be-e81cbb260a39 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80378) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 02 10:02:16.899416 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d518e4a2-6844-4a6a-89be-e81cbb260a39 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:02:16.901620 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d518e4a2-6844-4a6a-89be-e81cbb260a39 None admin] RBAC: Authorization granted {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:02:16.919929 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 251/509] 10.222.0.82 () {56 vars in 1109 bytes} [Tue Aug 2 10:02:16 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 261 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 02 10:02:16.920797 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d518e4a2-6844-4a6a-89be-e81cbb260a39 None admin] Authenticating user token {{(pid=80378) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:02:16.957877 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG dbcounter [-] [80377] Writing DB stats keystone:SELECT=100 {{(pid=80377) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 10:02:16.996065 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG dbcounter [-] [80378] Writing DB stats keystone:SELECT=100 {{(pid=80378) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 10:02:17.040184 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3ef0201-d88f-49e0-b51c-b83c3d22f816 None admin] Validating token access rules against request {{(pid=80377) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:02:17.074082 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d518e4a2-6844-4a6a-89be-e81cbb260a39 None admin] Validating token access rules against request {{(pid=80378) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:02:17.106581 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f8ac78c-32be-44d5-af9d-dc8a3c964aba None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '891005fa5ff74041b75747d16cc66798', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80377) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:02:17.107295 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f8ac78c-32be-44d5-af9d-dc8a3c964aba None admin] REQUEST_METHOD: `GET` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:02:17.107659 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f8ac78c-32be-44d5-af9d-dc8a3c964aba None admin] SCRIPT_NAME: `/identity` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:02:17.108086 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f8ac78c-32be-44d5-af9d-dc8a3c964aba None admin] PATH_INFO: `/v3/roles/service` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:02:17.120390 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-369ebd13-cd3d-4472-ac2c-1e10fc8bd832 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '891005fa5ff74041b75747d16cc66798', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80378) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:02:17.121029 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-369ebd13-cd3d-4472-ac2c-1e10fc8bd832 None admin] REQUEST_METHOD: `GET` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:02:17.121304 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-369ebd13-cd3d-4472-ac2c-1e10fc8bd832 None admin] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:02:17.121720 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-369ebd13-cd3d-4472-ac2c-1e10fc8bd832 None admin] PATH_INFO: `/v3/users` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:02:17.122573 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-369ebd13-cd3d-4472-ac2c-1e10fc8bd832 None admin] RBAC: Adding query filter params (domain_id=default, name=cinder) {{(pid=80378) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 02 10:02:17.122871 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-369ebd13-cd3d-4472-ac2c-1e10fc8bd832 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:02:17.124413 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-369ebd13-cd3d-4472-ac2c-1e10fc8bd832 None admin] RBAC: Authorization granted {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:02:17.127702 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f8ac78c-32be-44d5-af9d-dc8a3c964aba None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 02 10:02:17.138087 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application [None req-9f8ac78c-32be-44d5-af9d-dc8a3c964aba None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Aug 02 10:02:17.138087 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 02 10:02:17.138087 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 02 10:02:17.138087 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 02 10:02:17.138087 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 02 10:02:17.138087 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 02 10:02:17.138087 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 02 10:02:17.138087 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 02 10:02:17.138087 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 02 10:02:17.138087 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 02 10:02:17.138087 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 02 10:02:17.138087 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 02 10:02:17.138087 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 02 10:02:17.138087 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 02 10:02:17.138087 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 02 10:02:17.138087 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 02 10:02:17.139751 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 02 10:02:17.139751 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 02 10:02:17.139751 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 02 10:02:17.139751 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 02 10:02:17.139751 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 02 10:02:17.139751 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 02 10:02:17.139751 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 02 10:02:17.139751 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 02 10:02:17.139751 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 02 10:02:17.139751 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application return self.get_or_create( Aug 02 10:02:17.139751 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 02 10:02:17.139751 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application with Lock( Aug 02 10:02:17.139751 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 02 10:02:17.139751 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application return self._enter() Aug 02 10:02:17.139751 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 02 10:02:17.139751 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 02 10:02:17.141211 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 02 10:02:17.141211 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application return self.creator() Aug 02 10:02:17.141211 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 02 10:02:17.141211 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application created_value = creator( Aug 02 10:02:17.141211 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 02 10:02:17.141211 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 02 10:02:17.141211 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 02 10:02:17.141211 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 02 10:02:17.141211 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 02 10:02:17.141211 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 02 10:02:17.141211 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Aug 02 10:02:17.141211 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application Aug 02 10:02:17.146063 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: [pid: 80377|app: 0|req: 259/510] 10.222.0.82 () {56 vars in 1102 bytes} [Tue Aug 2 10:02:16 2022] GET /identity/v3/roles/service => generated 85 bytes in 274 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 02 10:02:17.147801 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f8ac78c-32be-44d5-af9d-dc8a3c964aba None admin] Authenticating user token {{(pid=80377) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:02:17.188186 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 252/511] 10.222.0.82 () {56 vars in 1137 bytes} [Tue Aug 2 10:02:16 2022] GET /identity/v3/users?domain_id=default&name=cinder => generated 372 bytes in 268 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 02 10:02:17.189632 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-369ebd13-cd3d-4472-ac2c-1e10fc8bd832 None admin] Authenticating user token {{(pid=80378) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:02:17.333229 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f8ac78c-32be-44d5-af9d-dc8a3c964aba None admin] Validating token access rules against request {{(pid=80377) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:02:17.346207 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-369ebd13-cd3d-4472-ac2c-1e10fc8bd832 None admin] Validating token access rules against request {{(pid=80378) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:02:17.367185 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG dbcounter [-] [80377] Writing DB stats keystone:SELECT=100 {{(pid=80377) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 10:02:17.367460 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG dbcounter [-] [80378] Writing DB stats keystone:SELECT=100 {{(pid=80378) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 10:02:17.389716 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ac4999d-5da5-475f-afaf-ff181fce465a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '891005fa5ff74041b75747d16cc66798', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80377) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:02:17.392552 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ac4999d-5da5-475f-afaf-ff181fce465a None admin] REQUEST_METHOD: `GET` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:02:17.392552 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ac4999d-5da5-475f-afaf-ff181fce465a None admin] SCRIPT_NAME: `/identity` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:02:17.392552 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ac4999d-5da5-475f-afaf-ff181fce465a None admin] PATH_INFO: `/v3/domains` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:02:17.392552 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ac4999d-5da5-475f-afaf-ff181fce465a None admin] RBAC: Adding query filter params (name=Default) {{(pid=80377) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 02 10:02:17.392552 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ac4999d-5da5-475f-afaf-ff181fce465a None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:02:17.393551 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ac4999d-5da5-475f-afaf-ff181fce465a None admin] RBAC: Authorization granted {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:02:17.397835 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-591c24d1-b528-4e6e-9832-79796f9744c4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '891005fa5ff74041b75747d16cc66798', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80378) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:02:17.398196 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-591c24d1-b528-4e6e-9832-79796f9744c4 None admin] REQUEST_METHOD: `POST` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:02:17.398480 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-591c24d1-b528-4e6e-9832-79796f9744c4 None admin] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:02:17.398814 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-591c24d1-b528-4e6e-9832-79796f9744c4 None admin] PATH_INFO: `/v3/users` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:02:17.399509 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-591c24d1-b528-4e6e-9832-79796f9744c4 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:02:17.400984 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-591c24d1-b528-4e6e-9832-79796f9744c4 None admin] RBAC: Authorization granted {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:02:17.406893 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: [pid: 80377|app: 0|req: 260/512] 10.222.0.82 () {56 vars in 1109 bytes} [Tue Aug 2 10:02:17 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 260 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 02 10:02:17.408686 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ac4999d-5da5-475f-afaf-ff181fce465a None admin] Authenticating user token {{(pid=80377) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:02:17.426925 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 253/513] 10.222.0.82 () {60 vars in 1132 bytes} [Tue Aug 2 10:02:17 2022] POST /identity/v3/users => generated 247 bytes in 238 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 02 10:02:17.427952 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-591c24d1-b528-4e6e-9832-79796f9744c4 None admin] Authenticating user token {{(pid=80378) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:02:17.582025 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ac4999d-5da5-475f-afaf-ff181fce465a None admin] Validating token access rules against request {{(pid=80377) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:02:17.630471 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-591c24d1-b528-4e6e-9832-79796f9744c4 None admin] Validating token access rules against request {{(pid=80378) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:02:17.650094 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebeae562-e261-4305-9b79-6ff8559d8d0b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '891005fa5ff74041b75747d16cc66798', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80377) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:02:17.651358 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebeae562-e261-4305-9b79-6ff8559d8d0b None admin] REQUEST_METHOD: `GET` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:02:17.651878 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebeae562-e261-4305-9b79-6ff8559d8d0b None admin] SCRIPT_NAME: `/identity` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:02:17.653556 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebeae562-e261-4305-9b79-6ff8559d8d0b None admin] PATH_INFO: `/v3/projects/service` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:02:17.660964 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ebeae562-e261-4305-9b79-6ff8559d8d0b None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:02:17.663267 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ebeae562-e261-4305-9b79-6ff8559d8d0b None admin] RBAC: Authorization granted {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:02:17.685009 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application [None req-ebeae562-e261-4305-9b79-6ff8559d8d0b None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 02 10:02:17.685009 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 02 10:02:17.685009 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 02 10:02:17.685009 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 02 10:02:17.685009 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 02 10:02:17.685009 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 02 10:02:17.685009 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 02 10:02:17.685009 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 02 10:02:17.685009 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 02 10:02:17.685009 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 02 10:02:17.685009 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 02 10:02:17.685009 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 02 10:02:17.685009 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 02 10:02:17.685009 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 02 10:02:17.685009 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 02 10:02:17.686614 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 02 10:02:17.686614 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 02 10:02:17.686614 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 02 10:02:17.686614 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 02 10:02:17.686614 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 02 10:02:17.686614 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 02 10:02:17.686614 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application return self.get_or_create( Aug 02 10:02:17.686614 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 02 10:02:17.686614 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application with Lock( Aug 02 10:02:17.686614 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 02 10:02:17.686614 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application return self._enter() Aug 02 10:02:17.686614 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 02 10:02:17.686614 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 02 10:02:17.686614 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 02 10:02:17.686614 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application return self.creator() Aug 02 10:02:17.686614 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 02 10:02:17.686614 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application created_value = creator( Aug 02 10:02:17.687959 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 02 10:02:17.687959 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 02 10:02:17.687959 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 02 10:02:17.687959 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 02 10:02:17.687959 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 02 10:02:17.687959 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 02 10:02:17.687959 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 02 10:02:17.687959 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application Aug 02 10:02:17.687959 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: [pid: 80377|app: 0|req: 261/514] 10.222.0.82 () {56 vars in 1111 bytes} [Tue Aug 2 10:02:17 2022] GET /identity/v3/projects/service => generated 88 bytes in 280 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 02 10:02:17.689566 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebeae562-e261-4305-9b79-6ff8559d8d0b None admin] Authenticating user token {{(pid=80377) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:02:17.698637 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce4e9fa2-271d-4e2a-a3fa-ebd9a0919891 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '891005fa5ff74041b75747d16cc66798', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80378) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:02:17.699697 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce4e9fa2-271d-4e2a-a3fa-ebd9a0919891 None admin] REQUEST_METHOD: `GET` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:02:17.700179 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce4e9fa2-271d-4e2a-a3fa-ebd9a0919891 None admin] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:02:17.700683 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce4e9fa2-271d-4e2a-a3fa-ebd9a0919891 None admin] PATH_INFO: `/v3/roles` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:02:17.701994 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce4e9fa2-271d-4e2a-a3fa-ebd9a0919891 None admin] RBAC: Adding query filter params (name=service) {{(pid=80378) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 02 10:02:17.702558 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce4e9fa2-271d-4e2a-a3fa-ebd9a0919891 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:02:17.704401 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce4e9fa2-271d-4e2a-a3fa-ebd9a0919891 None admin] RBAC: Authorization granted {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:02:17.717616 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 254/515] 10.222.0.82 () {56 vars in 1103 bytes} [Tue Aug 2 10:02:17 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 290 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 02 10:02:17.718996 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce4e9fa2-271d-4e2a-a3fa-ebd9a0919891 None admin] Authenticating user token {{(pid=80378) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:02:17.756015 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG dbcounter [-] [80377] Writing DB stats keystone:SELECT=100 {{(pid=80377) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 10:02:17.763816 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG dbcounter [-] [80378] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80378) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 10:02:17.863486 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebeae562-e261-4305-9b79-6ff8559d8d0b None admin] Validating token access rules against request {{(pid=80377) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:02:17.876956 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce4e9fa2-271d-4e2a-a3fa-ebd9a0919891 None admin] Validating token access rules against request {{(pid=80378) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:02:17.921489 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8c5822d-45fb-48df-8baa-95daf1e94624 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '891005fa5ff74041b75747d16cc66798', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80378) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:02:17.921767 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f699f5d-d852-4877-9f42-9c4d6bddeac2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '891005fa5ff74041b75747d16cc66798', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80377) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:02:17.922446 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f699f5d-d852-4877-9f42-9c4d6bddeac2 None admin] REQUEST_METHOD: `GET` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:02:17.922604 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8c5822d-45fb-48df-8baa-95daf1e94624 None admin] REQUEST_METHOD: `GET` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:02:17.922736 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f699f5d-d852-4877-9f42-9c4d6bddeac2 None admin] SCRIPT_NAME: `/identity` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:02:17.923049 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8c5822d-45fb-48df-8baa-95daf1e94624 None admin] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:02:17.923201 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f699f5d-d852-4877-9f42-9c4d6bddeac2 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:02:17.923454 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8c5822d-45fb-48df-8baa-95daf1e94624 None admin] PATH_INFO: `/v3/users/glance` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:02:17.934090 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8c5822d-45fb-48df-8baa-95daf1e94624 None admin] RBAC: Authorizing `identity:get_user(user_id=glance)` {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:02:17.935577 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8c5822d-45fb-48df-8baa-95daf1e94624 None admin] RBAC: Authorization granted {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:02:17.958046 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application [None req-c8c5822d-45fb-48df-8baa-95daf1e94624 None admin] Could not find user: glance.: keystone.exception.UserNotFound: Could not find user: glance. Aug 02 10:02:17.958046 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 02 10:02:17.958046 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 02 10:02:17.958046 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 02 10:02:17.958046 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 02 10:02:17.958046 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 02 10:02:17.958046 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 02 10:02:17.958046 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 02 10:02:17.958046 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 02 10:02:17.958046 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 02 10:02:17.958046 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 02 10:02:17.958046 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 02 10:02:17.958046 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 02 10:02:17.958046 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 02 10:02:17.958046 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 02 10:02:17.958046 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 02 10:02:17.959302 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 02 10:02:17.959302 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 02 10:02:17.959302 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 02 10:02:17.959302 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 02 10:02:17.959302 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 02 10:02:17.959302 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 02 10:02:17.959302 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 02 10:02:17.959302 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 02 10:02:17.959302 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 02 10:02:17.959302 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application return self.get_or_create( Aug 02 10:02:17.959302 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 02 10:02:17.959302 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application with Lock( Aug 02 10:02:17.959302 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 02 10:02:17.959302 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application return self._enter() Aug 02 10:02:17.959302 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 02 10:02:17.959302 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 02 10:02:17.959302 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 02 10:02:17.959302 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application return self.creator() Aug 02 10:02:17.960307 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 02 10:02:17.960307 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application created_value = creator( Aug 02 10:02:17.960307 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 02 10:02:17.960307 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 02 10:02:17.960307 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 02 10:02:17.960307 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 02 10:02:17.960307 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 02 10:02:17.960307 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 02 10:02:17.960307 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance. Aug 02 10:02:17.960307 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application Aug 02 10:02:17.960307 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 255/516] 10.222.0.82 () {56 vars in 1099 bytes} [Tue Aug 2 10:02:17 2022] GET /identity/v3/users/glance => generated 84 bytes in 242 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 02 10:02:17.961763 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8c5822d-45fb-48df-8baa-95daf1e94624 None admin] Authenticating user token {{(pid=80378) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:02:18.033502 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f699f5d-d852-4877-9f42-9c4d6bddeac2 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:02:18.035146 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f699f5d-d852-4877-9f42-9c4d6bddeac2 None admin] RBAC: Authorization granted {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:02:18.082081 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG dbcounter [-] [80377] Writing DB stats keystone:SELECT=100 {{(pid=80377) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 10:02:18.104058 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG dbcounter [-] [80378] Writing DB stats keystone:SELECT=100 {{(pid=80378) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 10:02:18.115139 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8c5822d-45fb-48df-8baa-95daf1e94624 None admin] Validating token access rules against request {{(pid=80378) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:02:18.174650 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73364569-2019-4a9d-b5e1-aa84b81c7c9a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '891005fa5ff74041b75747d16cc66798', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80378) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:02:18.175806 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73364569-2019-4a9d-b5e1-aa84b81c7c9a None admin] REQUEST_METHOD: `GET` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:02:18.176092 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73364569-2019-4a9d-b5e1-aa84b81c7c9a None admin] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:02:18.176376 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73364569-2019-4a9d-b5e1-aa84b81c7c9a None admin] PATH_INFO: `/v3/projects` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:02:18.177336 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73364569-2019-4a9d-b5e1-aa84b81c7c9a None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80378) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 02 10:02:18.177706 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73364569-2019-4a9d-b5e1-aa84b81c7c9a None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:02:18.180394 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73364569-2019-4a9d-b5e1-aa84b81c7c9a None admin] RBAC: Authorization granted {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:02:18.198979 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: [pid: 80377|app: 0|req: 262/517] 10.222.0.82 () {58 vars in 1282 bytes} [Tue Aug 2 10:02:17 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 506 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 02 10:02:18.198979 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f699f5d-d852-4877-9f42-9c4d6bddeac2 None admin] Authenticating user token {{(pid=80377) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:02:18.210529 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 256/518] 10.222.0.82 () {56 vars in 1148 bytes} [Tue Aug 2 10:02:17 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 250 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 02 10:02:18.211726 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73364569-2019-4a9d-b5e1-aa84b81c7c9a None admin] Authenticating user token {{(pid=80378) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:02:18.360964 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73364569-2019-4a9d-b5e1-aa84b81c7c9a None admin] Validating token access rules against request {{(pid=80378) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:02:18.363107 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f699f5d-d852-4877-9f42-9c4d6bddeac2 None admin] Validating token access rules against request {{(pid=80377) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:02:18.404400 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9085ca3-4cbb-41a7-a123-91297ecbf1c6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '891005fa5ff74041b75747d16cc66798', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80377) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:02:18.405228 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9085ca3-4cbb-41a7-a123-91297ecbf1c6 None admin] REQUEST_METHOD: `GET` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:02:18.405624 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9085ca3-4cbb-41a7-a123-91297ecbf1c6 None admin] SCRIPT_NAME: `/identity` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:02:18.405937 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9085ca3-4cbb-41a7-a123-91297ecbf1c6 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:02:18.409649 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a8ef234-a99b-4061-b35d-f532cb1be1b1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '891005fa5ff74041b75747d16cc66798', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80378) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:02:18.410645 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a8ef234-a99b-4061-b35d-f532cb1be1b1 None admin] REQUEST_METHOD: `GET` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:02:18.410902 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG dbcounter [-] [80377] Writing DB stats keystone:SELECT=100 {{(pid=80377) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 10:02:18.411078 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a8ef234-a99b-4061-b35d-f532cb1be1b1 None admin] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:02:18.411236 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a8ef234-a99b-4061-b35d-f532cb1be1b1 None admin] PATH_INFO: `/v3/users` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:02:18.412061 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a8ef234-a99b-4061-b35d-f532cb1be1b1 None admin] RBAC: Adding query filter params (domain_id=default, name=glance) {{(pid=80378) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 02 10:02:18.412438 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a8ef234-a99b-4061-b35d-f532cb1be1b1 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:02:18.414251 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a8ef234-a99b-4061-b35d-f532cb1be1b1 None admin] RBAC: Authorization granted {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:02:18.424758 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9085ca3-4cbb-41a7-a123-91297ecbf1c6 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:02:18.425375 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9085ca3-4cbb-41a7-a123-91297ecbf1c6 None admin] RBAC: Authorization granted {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:02:18.445408 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 257/519] 10.222.0.82 () {56 vars in 1137 bytes} [Tue Aug 2 10:02:18 2022] GET /identity/v3/users?domain_id=default&name=glance => generated 372 bytes in 227 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 02 10:02:18.445408 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a8ef234-a99b-4061-b35d-f532cb1be1b1 None admin] Authenticating user token {{(pid=80378) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:02:18.447242 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application [None req-a9085ca3-4cbb-41a7-a123-91297ecbf1c6 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 02 10:02:18.447242 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 02 10:02:18.447242 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 02 10:02:18.447242 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 02 10:02:18.447242 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 02 10:02:18.447242 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 02 10:02:18.447242 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 02 10:02:18.447242 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 02 10:02:18.447242 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 02 10:02:18.447242 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 02 10:02:18.447242 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 02 10:02:18.447242 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 02 10:02:18.447242 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 02 10:02:18.447242 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 02 10:02:18.447242 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 02 10:02:18.449007 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 02 10:02:18.449007 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 02 10:02:18.449007 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 02 10:02:18.449007 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 02 10:02:18.449007 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 02 10:02:18.449007 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 02 10:02:18.449007 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application return self.get_or_create( Aug 02 10:02:18.449007 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 02 10:02:18.449007 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application with Lock( Aug 02 10:02:18.449007 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 02 10:02:18.449007 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application return self._enter() Aug 02 10:02:18.449007 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 02 10:02:18.449007 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 02 10:02:18.449007 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 02 10:02:18.449007 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application return self.creator() Aug 02 10:02:18.449007 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 02 10:02:18.449007 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application created_value = creator( Aug 02 10:02:18.450943 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 02 10:02:18.450943 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 02 10:02:18.450943 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 02 10:02:18.450943 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application Aug 02 10:02:18.454263 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: [pid: 80377|app: 0|req: 263/520] 10.222.0.82 () {56 vars in 1108 bytes} [Tue Aug 2 10:02:18 2022] GET /identity/v3/domains/Default => generated 87 bytes in 258 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 02 10:02:18.454263 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9085ca3-4cbb-41a7-a123-91297ecbf1c6 None admin] Authenticating user token {{(pid=80377) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:02:18.468942 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG dbcounter [-] [80378] Writing DB stats keystone:SELECT=100 {{(pid=80378) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 10:02:18.593798 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a8ef234-a99b-4061-b35d-f532cb1be1b1 None admin] Validating token access rules against request {{(pid=80378) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:02:18.608938 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9085ca3-4cbb-41a7-a123-91297ecbf1c6 None admin] Validating token access rules against request {{(pid=80377) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:02:18.642743 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d2546c9-030c-4013-95f4-1e964cda6cee None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '891005fa5ff74041b75747d16cc66798', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80378) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:02:18.643882 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d2546c9-030c-4013-95f4-1e964cda6cee None admin] REQUEST_METHOD: `GET` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:02:18.644371 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d2546c9-030c-4013-95f4-1e964cda6cee None admin] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:02:18.644782 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d2546c9-030c-4013-95f4-1e964cda6cee None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:02:18.662812 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-438bcae0-96b5-4968-8594-43a7adcf3d7e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '891005fa5ff74041b75747d16cc66798', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80377) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:02:18.663682 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d2546c9-030c-4013-95f4-1e964cda6cee None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:02:18.663802 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-438bcae0-96b5-4968-8594-43a7adcf3d7e None admin] REQUEST_METHOD: `GET` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:02:18.664220 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-438bcae0-96b5-4968-8594-43a7adcf3d7e None admin] SCRIPT_NAME: `/identity` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:02:18.664689 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-438bcae0-96b5-4968-8594-43a7adcf3d7e None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:02:18.666056 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-438bcae0-96b5-4968-8594-43a7adcf3d7e None admin] RBAC: Adding query filter params (role.id=02003a0be60f47f2a8dfea17053af52c, scope.project.id=f1b38ebb804c41c69772c998aae72c41, user.id=88af518196674fe593ad88d30111fb7d) {{(pid=80377) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 02 10:02:18.666580 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d2546c9-030c-4013-95f4-1e964cda6cee None admin] RBAC: Authorization granted {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:02:18.666683 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-438bcae0-96b5-4968-8594-43a7adcf3d7e None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:02:18.669133 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-438bcae0-96b5-4968-8594-43a7adcf3d7e None admin] RBAC: Authorization granted {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:02:18.689919 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: [pid: 80377|app: 0|req: 264/521] 10.222.0.82 () {56 vars in 1374 bytes} [Tue Aug 2 10:02:18 2022] GET /identity/v3/role_assignments?user.id=88af518196674fe593ad88d30111fb7d&scope.project.id=f1b38ebb804c41c69772c998aae72c41&role.id=02003a0be60f47f2a8dfea17053af52c => generated 259 bytes in 226 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 02 10:02:18.689919 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-438bcae0-96b5-4968-8594-43a7adcf3d7e None admin] Authenticating user token {{(pid=80377) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:02:18.698318 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application [None req-1d2546c9-030c-4013-95f4-1e964cda6cee None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 02 10:02:18.698318 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 02 10:02:18.698318 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 02 10:02:18.698318 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 02 10:02:18.698318 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 02 10:02:18.698318 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 02 10:02:18.698318 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 02 10:02:18.698318 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 02 10:02:18.698318 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 02 10:02:18.698318 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 02 10:02:18.698318 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 02 10:02:18.698318 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 02 10:02:18.698318 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 02 10:02:18.698318 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 02 10:02:18.698318 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 02 10:02:18.699411 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 02 10:02:18.699411 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 02 10:02:18.699411 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 02 10:02:18.699411 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 02 10:02:18.699411 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 02 10:02:18.699411 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 02 10:02:18.699411 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application return self.get_or_create( Aug 02 10:02:18.699411 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 02 10:02:18.699411 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application with Lock( Aug 02 10:02:18.699411 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 02 10:02:18.699411 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application return self._enter() Aug 02 10:02:18.699411 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 02 10:02:18.699411 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 02 10:02:18.699411 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 02 10:02:18.699411 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application return self.creator() Aug 02 10:02:18.699411 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 02 10:02:18.699411 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application created_value = creator( Aug 02 10:02:18.700469 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 02 10:02:18.700469 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 02 10:02:18.700469 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 02 10:02:18.700469 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application Aug 02 10:02:18.703080 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 258/522] 10.222.0.82 () {56 vars in 1108 bytes} [Tue Aug 2 10:02:18 2022] GET /identity/v3/domains/Default => generated 87 bytes in 263 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 02 10:02:18.703080 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d2546c9-030c-4013-95f4-1e964cda6cee None admin] Authenticating user token {{(pid=80378) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:02:18.781176 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG dbcounter [-] [80377] Writing DB stats keystone:SELECT=100 {{(pid=80377) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 10:02:18.796972 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG dbcounter [-] [80378] Writing DB stats keystone:SELECT=100 {{(pid=80378) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 10:02:18.864332 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d2546c9-030c-4013-95f4-1e964cda6cee None admin] Validating token access rules against request {{(pid=80378) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:02:18.889135 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-438bcae0-96b5-4968-8594-43a7adcf3d7e None admin] Validating token access rules against request {{(pid=80377) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:02:18.909163 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e2e4456-f4fb-4166-8c34-f2bff7b7f6f4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '891005fa5ff74041b75747d16cc66798', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80378) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:02:18.909830 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e2e4456-f4fb-4166-8c34-f2bff7b7f6f4 None admin] REQUEST_METHOD: `GET` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:02:18.910119 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e2e4456-f4fb-4166-8c34-f2bff7b7f6f4 None admin] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:02:18.910393 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e2e4456-f4fb-4166-8c34-f2bff7b7f6f4 None admin] PATH_INFO: `/v3/domains` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:02:18.911174 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e2e4456-f4fb-4166-8c34-f2bff7b7f6f4 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80378) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 02 10:02:18.911487 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e2e4456-f4fb-4166-8c34-f2bff7b7f6f4 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:02:18.912899 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e2e4456-f4fb-4166-8c34-f2bff7b7f6f4 None admin] RBAC: Authorization granted {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:02:18.926743 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 259/523] 10.222.0.82 () {56 vars in 1109 bytes} [Tue Aug 2 10:02:18 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 224 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 02 10:02:18.927628 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e2e4456-f4fb-4166-8c34-f2bff7b7f6f4 None admin] Authenticating user token {{(pid=80378) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:02:18.941356 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-544887a1-5652-4326-bb45-1ffb7040dce7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '891005fa5ff74041b75747d16cc66798', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80377) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:02:18.941921 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-544887a1-5652-4326-bb45-1ffb7040dce7 None admin] REQUEST_METHOD: `GET` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:02:18.942195 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-544887a1-5652-4326-bb45-1ffb7040dce7 None admin] SCRIPT_NAME: `/identity` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:02:18.942473 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-544887a1-5652-4326-bb45-1ffb7040dce7 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:02:19.024362 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-544887a1-5652-4326-bb45-1ffb7040dce7 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:02:19.025862 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-544887a1-5652-4326-bb45-1ffb7040dce7 None admin] RBAC: Authorization granted {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:02:19.062728 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e2e4456-f4fb-4166-8c34-f2bff7b7f6f4 None admin] Validating token access rules against request {{(pid=80378) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:02:19.091266 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG dbcounter [-] [80378] Writing DB stats keystone:SELECT=100 {{(pid=80378) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 10:02:19.092514 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG dbcounter [-] [80377] Writing DB stats keystone:SELECT=100 {{(pid=80377) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 10:02:19.109838 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48c05354-6265-4acf-8a7e-3980a052efab None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '891005fa5ff74041b75747d16cc66798', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80378) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:02:19.111004 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48c05354-6265-4acf-8a7e-3980a052efab None admin] REQUEST_METHOD: `GET` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:02:19.111289 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48c05354-6265-4acf-8a7e-3980a052efab None admin] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:02:19.111564 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48c05354-6265-4acf-8a7e-3980a052efab None admin] PATH_INFO: `/v3/domains` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:02:19.112280 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48c05354-6265-4acf-8a7e-3980a052efab None admin] RBAC: Adding query filter params (name=Default) {{(pid=80378) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 02 10:02:19.112905 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48c05354-6265-4acf-8a7e-3980a052efab None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:02:19.115040 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48c05354-6265-4acf-8a7e-3980a052efab None admin] RBAC: Authorization granted {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:02:19.132974 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 260/524] 10.222.0.82 () {56 vars in 1109 bytes} [Tue Aug 2 10:02:18 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 205 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 02 10:02:19.134114 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48c05354-6265-4acf-8a7e-3980a052efab None admin] Authenticating user token {{(pid=80378) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:02:19.161315 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: [pid: 80377|app: 0|req: 265/525] 10.222.0.82 () {58 vars in 1282 bytes} [Tue Aug 2 10:02:18 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 482 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 02 10:02:19.162681 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-544887a1-5652-4326-bb45-1ffb7040dce7 None admin] Authenticating user token {{(pid=80377) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:02:19.299159 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48c05354-6265-4acf-8a7e-3980a052efab None admin] Validating token access rules against request {{(pid=80378) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:02:19.338655 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-544887a1-5652-4326-bb45-1ffb7040dce7 None admin] Validating token access rules against request {{(pid=80377) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:02:19.342469 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf2ec37f-61e0-4a48-9e6f-e3777c748f2f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '891005fa5ff74041b75747d16cc66798', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80378) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:02:19.343709 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf2ec37f-61e0-4a48-9e6f-e3777c748f2f None admin] REQUEST_METHOD: `GET` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:02:19.344220 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf2ec37f-61e0-4a48-9e6f-e3777c748f2f None admin] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:02:19.345465 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf2ec37f-61e0-4a48-9e6f-e3777c748f2f None admin] PATH_INFO: `/v3/users/neutron` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:02:19.358140 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf2ec37f-61e0-4a48-9e6f-e3777c748f2f None admin] RBAC: Authorizing `identity:get_user(user_id=neutron)` {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:02:19.360007 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf2ec37f-61e0-4a48-9e6f-e3777c748f2f None admin] RBAC: Authorization granted {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:02:19.375048 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application [None req-cf2ec37f-61e0-4a48-9e6f-e3777c748f2f None admin] Could not find user: neutron.: keystone.exception.UserNotFound: Could not find user: neutron. Aug 02 10:02:19.375048 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 02 10:02:19.375048 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 02 10:02:19.375048 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 02 10:02:19.375048 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 02 10:02:19.375048 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 02 10:02:19.375048 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 02 10:02:19.375048 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 02 10:02:19.375048 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 02 10:02:19.375048 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 02 10:02:19.375048 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 02 10:02:19.375048 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 02 10:02:19.375048 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 02 10:02:19.375048 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 02 10:02:19.375048 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 02 10:02:19.375048 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 02 10:02:19.376844 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 02 10:02:19.376844 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 02 10:02:19.376844 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 02 10:02:19.376844 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 02 10:02:19.376844 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 02 10:02:19.376844 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 02 10:02:19.376844 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 02 10:02:19.376844 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 02 10:02:19.376844 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 02 10:02:19.376844 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application return self.get_or_create( Aug 02 10:02:19.376844 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 02 10:02:19.376844 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application with Lock( Aug 02 10:02:19.376844 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 02 10:02:19.376844 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application return self._enter() Aug 02 10:02:19.376844 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 02 10:02:19.376844 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 02 10:02:19.376844 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 02 10:02:19.376844 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application return self.creator() Aug 02 10:02:19.378840 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 02 10:02:19.378840 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application created_value = creator( Aug 02 10:02:19.378840 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 02 10:02:19.378840 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 02 10:02:19.378840 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 02 10:02:19.378840 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 02 10:02:19.378840 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 02 10:02:19.378840 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 02 10:02:19.378840 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: neutron. Aug 02 10:02:19.378840 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application Aug 02 10:02:19.378840 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 261/526] 10.222.0.82 () {56 vars in 1102 bytes} [Tue Aug 2 10:02:19 2022] GET /identity/v3/users/neutron => generated 85 bytes in 244 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 02 10:02:19.379493 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf2ec37f-61e0-4a48-9e6f-e3777c748f2f None admin] Authenticating user token {{(pid=80378) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:02:19.387354 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f45eac4c-cd9d-4d69-b293-c72fc5295888 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '891005fa5ff74041b75747d16cc66798', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80377) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:02:19.388124 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f45eac4c-cd9d-4d69-b293-c72fc5295888 None admin] REQUEST_METHOD: `GET` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:02:19.388435 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f45eac4c-cd9d-4d69-b293-c72fc5295888 None admin] SCRIPT_NAME: `/identity` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:02:19.388709 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f45eac4c-cd9d-4d69-b293-c72fc5295888 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:02:19.408909 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f45eac4c-cd9d-4d69-b293-c72fc5295888 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:02:19.409890 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f45eac4c-cd9d-4d69-b293-c72fc5295888 None admin] RBAC: Authorization granted {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:02:19.425809 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application [None req-f45eac4c-cd9d-4d69-b293-c72fc5295888 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 02 10:02:19.425809 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 02 10:02:19.425809 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 02 10:02:19.425809 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 02 10:02:19.425809 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 02 10:02:19.425809 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 02 10:02:19.425809 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 02 10:02:19.425809 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 02 10:02:19.425809 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 02 10:02:19.425809 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 02 10:02:19.425809 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 02 10:02:19.425809 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 02 10:02:19.425809 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 02 10:02:19.425809 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 02 10:02:19.425809 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 02 10:02:19.426751 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 02 10:02:19.426751 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 02 10:02:19.426751 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 02 10:02:19.426751 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 02 10:02:19.426751 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 02 10:02:19.426751 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 02 10:02:19.426751 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application return self.get_or_create( Aug 02 10:02:19.426751 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 02 10:02:19.426751 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application with Lock( Aug 02 10:02:19.426751 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 02 10:02:19.426751 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application return self._enter() Aug 02 10:02:19.426751 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 02 10:02:19.426751 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 02 10:02:19.426751 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 02 10:02:19.426751 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application return self.creator() Aug 02 10:02:19.426751 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 02 10:02:19.426751 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application created_value = creator( Aug 02 10:02:19.427619 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 02 10:02:19.427619 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 02 10:02:19.427619 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 02 10:02:19.427619 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 02 10:02:19.427619 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 02 10:02:19.427619 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 02 10:02:19.427619 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 02 10:02:19.427619 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application Aug 02 10:02:19.427619 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: [pid: 80377|app: 0|req: 266/527] 10.222.0.82 () {56 vars in 1111 bytes} [Tue Aug 2 10:02:19 2022] GET /identity/v3/projects/service => generated 88 bytes in 265 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 02 10:02:19.428560 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f45eac4c-cd9d-4d69-b293-c72fc5295888 None admin] Authenticating user token {{(pid=80377) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:02:19.454707 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG dbcounter [-] [80378] Writing DB stats keystone:SELECT=100 {{(pid=80378) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 10:02:19.457571 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG dbcounter [-] [80377] Writing DB stats keystone:SELECT=100 {{(pid=80377) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 10:02:19.542980 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf2ec37f-61e0-4a48-9e6f-e3777c748f2f None admin] Validating token access rules against request {{(pid=80378) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:02:19.584924 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-565c1863-17cd-458b-978f-2869c4ac1949 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '891005fa5ff74041b75747d16cc66798', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80378) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:02:19.585981 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-565c1863-17cd-458b-978f-2869c4ac1949 None admin] REQUEST_METHOD: `GET` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:02:19.586548 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-565c1863-17cd-458b-978f-2869c4ac1949 None admin] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:02:19.587041 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-565c1863-17cd-458b-978f-2869c4ac1949 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:02:19.601503 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-565c1863-17cd-458b-978f-2869c4ac1949 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:02:19.603420 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-565c1863-17cd-458b-978f-2869c4ac1949 None admin] RBAC: Authorization granted {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:02:19.612068 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f45eac4c-cd9d-4d69-b293-c72fc5295888 None admin] Validating token access rules against request {{(pid=80377) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:02:19.620978 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application [None req-565c1863-17cd-458b-978f-2869c4ac1949 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 02 10:02:19.620978 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 02 10:02:19.620978 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 02 10:02:19.620978 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 02 10:02:19.620978 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 02 10:02:19.620978 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 02 10:02:19.620978 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 02 10:02:19.620978 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 02 10:02:19.620978 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 02 10:02:19.620978 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 02 10:02:19.620978 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 02 10:02:19.620978 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 02 10:02:19.620978 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 02 10:02:19.620978 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 02 10:02:19.620978 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 02 10:02:19.621953 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 02 10:02:19.621953 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 02 10:02:19.621953 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 02 10:02:19.621953 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 02 10:02:19.621953 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 02 10:02:19.621953 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 02 10:02:19.621953 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application return self.get_or_create( Aug 02 10:02:19.621953 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 02 10:02:19.621953 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application with Lock( Aug 02 10:02:19.621953 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 02 10:02:19.621953 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application return self._enter() Aug 02 10:02:19.621953 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 02 10:02:19.621953 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 02 10:02:19.621953 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 02 10:02:19.621953 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application return self.creator() Aug 02 10:02:19.621953 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 02 10:02:19.621953 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application created_value = creator( Aug 02 10:02:19.622795 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 02 10:02:19.622795 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 02 10:02:19.622795 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 02 10:02:19.622795 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application Aug 02 10:02:19.625122 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 262/528] 10.222.0.82 () {56 vars in 1108 bytes} [Tue Aug 2 10:02:19 2022] GET /identity/v3/domains/Default => generated 87 bytes in 246 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 02 10:02:19.626311 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-565c1863-17cd-458b-978f-2869c4ac1949 None admin] Authenticating user token {{(pid=80378) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:02:19.673736 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9137ca5-85d0-45bf-9c92-02c7089ed0c7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '891005fa5ff74041b75747d16cc66798', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80377) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:02:19.674967 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9137ca5-85d0-45bf-9c92-02c7089ed0c7 None admin] REQUEST_METHOD: `GET` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:02:19.675515 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9137ca5-85d0-45bf-9c92-02c7089ed0c7 None admin] SCRIPT_NAME: `/identity` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:02:19.676035 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9137ca5-85d0-45bf-9c92-02c7089ed0c7 None admin] PATH_INFO: `/v3/users` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:02:19.677561 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9137ca5-85d0-45bf-9c92-02c7089ed0c7 None admin] RBAC: Adding query filter params (domain_id=default, name=neutron) {{(pid=80377) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 02 10:02:19.678168 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9137ca5-85d0-45bf-9c92-02c7089ed0c7 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:02:19.680305 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9137ca5-85d0-45bf-9c92-02c7089ed0c7 None admin] RBAC: Authorization granted {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:02:19.708614 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: [pid: 80377|app: 0|req: 267/529] 10.222.0.82 () {56 vars in 1139 bytes} [Tue Aug 2 10:02:19 2022] GET /identity/v3/users?domain_id=default&name=neutron => generated 374 bytes in 275 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 02 10:02:19.708614 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9137ca5-85d0-45bf-9c92-02c7089ed0c7 None admin] Authenticating user token {{(pid=80377) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:02:19.769986 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG dbcounter [-] [80378] Writing DB stats keystone:SELECT=100 {{(pid=80378) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 10:02:19.784120 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-565c1863-17cd-458b-978f-2869c4ac1949 None admin] Validating token access rules against request {{(pid=80378) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:02:19.806979 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG dbcounter [-] [80377] Writing DB stats keystone:SELECT=100 {{(pid=80377) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 10:02:19.832603 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-691aa13b-71b3-43a0-801b-3df2bfca5c8d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '891005fa5ff74041b75747d16cc66798', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80378) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:02:19.833356 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-691aa13b-71b3-43a0-801b-3df2bfca5c8d None admin] REQUEST_METHOD: `GET` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:02:19.833764 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-691aa13b-71b3-43a0-801b-3df2bfca5c8d None admin] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:02:19.834562 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-691aa13b-71b3-43a0-801b-3df2bfca5c8d None admin] PATH_INFO: `/v3/projects` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:02:19.835382 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-691aa13b-71b3-43a0-801b-3df2bfca5c8d None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80378) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 02 10:02:19.835640 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-691aa13b-71b3-43a0-801b-3df2bfca5c8d None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:02:19.837682 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-691aa13b-71b3-43a0-801b-3df2bfca5c8d None admin] RBAC: Authorization granted {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:02:19.853744 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9137ca5-85d0-45bf-9c92-02c7089ed0c7 None admin] Validating token access rules against request {{(pid=80377) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:02:19.855033 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 263/530] 10.222.0.82 () {56 vars in 1148 bytes} [Tue Aug 2 10:02:19 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 229 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 02 10:02:19.856296 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-691aa13b-71b3-43a0-801b-3df2bfca5c8d None admin] Authenticating user token {{(pid=80378) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:02:19.907932 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5de71f5-af01-4f7d-9312-5020fa01f78e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '891005fa5ff74041b75747d16cc66798', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80377) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:02:19.908927 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5de71f5-af01-4f7d-9312-5020fa01f78e None admin] REQUEST_METHOD: `GET` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:02:19.909432 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5de71f5-af01-4f7d-9312-5020fa01f78e None admin] SCRIPT_NAME: `/identity` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:02:19.909982 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5de71f5-af01-4f7d-9312-5020fa01f78e None admin] PATH_INFO: `/v3/domains` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:02:19.911439 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5de71f5-af01-4f7d-9312-5020fa01f78e None admin] RBAC: Adding query filter params (name=Default) {{(pid=80377) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 02 10:02:19.911949 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5de71f5-af01-4f7d-9312-5020fa01f78e None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:02:19.914259 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5de71f5-af01-4f7d-9312-5020fa01f78e None admin] RBAC: Authorization granted {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:02:19.932033 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: [pid: 80377|app: 0|req: 268/531] 10.222.0.82 () {56 vars in 1109 bytes} [Tue Aug 2 10:02:19 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 225 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 02 10:02:19.933880 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5de71f5-af01-4f7d-9312-5020fa01f78e None admin] Authenticating user token {{(pid=80377) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:02:20.002556 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-691aa13b-71b3-43a0-801b-3df2bfca5c8d None admin] Validating token access rules against request {{(pid=80378) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:02:20.040876 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dab71f35-0f19-4f42-a28a-12dbe1d170f4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '891005fa5ff74041b75747d16cc66798', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80378) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:02:20.041509 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dab71f35-0f19-4f42-a28a-12dbe1d170f4 None admin] REQUEST_METHOD: `GET` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:02:20.041809 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dab71f35-0f19-4f42-a28a-12dbe1d170f4 None admin] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:02:20.042096 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dab71f35-0f19-4f42-a28a-12dbe1d170f4 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:02:20.067718 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG dbcounter [-] [80378] Writing DB stats keystone:SELECT=100 {{(pid=80378) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 10:02:20.087227 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5de71f5-af01-4f7d-9312-5020fa01f78e None admin] Validating token access rules against request {{(pid=80377) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:02:20.117335 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dab71f35-0f19-4f42-a28a-12dbe1d170f4 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:02:20.118885 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dab71f35-0f19-4f42-a28a-12dbe1d170f4 None admin] RBAC: Authorization granted {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:02:20.123807 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG dbcounter [-] [80377] Writing DB stats keystone:SELECT=100 {{(pid=80377) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 10:02:20.138384 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bee35f2-68ad-4262-8b42-b292e28fe07c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '891005fa5ff74041b75747d16cc66798', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80377) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:02:20.138999 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bee35f2-68ad-4262-8b42-b292e28fe07c None admin] REQUEST_METHOD: `GET` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:02:20.139287 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bee35f2-68ad-4262-8b42-b292e28fe07c None admin] SCRIPT_NAME: `/identity` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:02:20.139563 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bee35f2-68ad-4262-8b42-b292e28fe07c None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:02:20.140367 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7bee35f2-68ad-4262-8b42-b292e28fe07c None admin] RBAC: Adding query filter params (role.id=02003a0be60f47f2a8dfea17053af52c, scope.project.id=f1b38ebb804c41c69772c998aae72c41, user.id=1023de281ae74e9d8a6d6c312c1c97a7) {{(pid=80377) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 02 10:02:20.140676 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7bee35f2-68ad-4262-8b42-b292e28fe07c None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:02:20.142716 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7bee35f2-68ad-4262-8b42-b292e28fe07c None admin] RBAC: Authorization granted {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:02:20.149942 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: [pid: 80377|app: 0|req: 269/532] 10.222.0.82 () {56 vars in 1374 bytes} [Tue Aug 2 10:02:19 2022] GET /identity/v3/role_assignments?user.id=1023de281ae74e9d8a6d6c312c1c97a7&scope.project.id=f1b38ebb804c41c69772c998aae72c41&role.id=02003a0be60f47f2a8dfea17053af52c => generated 259 bytes in 216 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 02 10:02:20.149942 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bee35f2-68ad-4262-8b42-b292e28fe07c None admin] Authenticating user token {{(pid=80377) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:02:20.244429 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 264/533] 10.222.0.82 () {58 vars in 1282 bytes} [Tue Aug 2 10:02:19 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 389 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 02 10:02:20.246133 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f9641ae-edbb-4d4c-895e-b7a6b13b6b96 None None] REQUEST_METHOD: `GET` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:02:20.246597 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f9641ae-edbb-4d4c-895e-b7a6b13b6b96 None None] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:02:20.247007 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f9641ae-edbb-4d4c-895e-b7a6b13b6b96 None None] PATH_INFO: `/` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:02:20.248494 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 265/534] 10.222.0.82 () {54 vars in 926 bytes} [Tue Aug 2 10:02:20 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 02 10:02:20.251538 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f9641ae-edbb-4d4c-895e-b7a6b13b6b96 None None] Authenticating user token {{(pid=80378) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:02:20.326884 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG dbcounter [-] [80378] Writing DB stats keystone:SELECT=100 {{(pid=80378) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 10:02:20.339455 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bee35f2-68ad-4262-8b42-b292e28fe07c None admin] Validating token access rules against request {{(pid=80377) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:02:20.385735 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f9641ae-edbb-4d4c-895e-b7a6b13b6b96 None None] Validating token access rules against request {{(pid=80378) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:02:20.393580 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a67d7119-d323-439f-b24c-dbc13b927160 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '891005fa5ff74041b75747d16cc66798', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80377) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:02:20.400727 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a67d7119-d323-439f-b24c-dbc13b927160 None admin] REQUEST_METHOD: `GET` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:02:20.400727 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a67d7119-d323-439f-b24c-dbc13b927160 None admin] SCRIPT_NAME: `/identity` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:02:20.400727 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a67d7119-d323-439f-b24c-dbc13b927160 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:02:20.413550 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a67d7119-d323-439f-b24c-dbc13b927160 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:02:20.416480 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a67d7119-d323-439f-b24c-dbc13b927160 None admin] RBAC: Authorization granted {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:02:20.441255 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application [None req-a67d7119-d323-439f-b24c-dbc13b927160 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 02 10:02:20.441255 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 02 10:02:20.441255 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 02 10:02:20.441255 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 02 10:02:20.441255 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 02 10:02:20.441255 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 02 10:02:20.441255 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 02 10:02:20.441255 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 02 10:02:20.441255 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 02 10:02:20.441255 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 02 10:02:20.441255 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 02 10:02:20.441255 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 02 10:02:20.441255 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 02 10:02:20.441255 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 02 10:02:20.441255 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 02 10:02:20.442416 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 02 10:02:20.442416 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 02 10:02:20.442416 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 02 10:02:20.442416 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 02 10:02:20.442416 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 02 10:02:20.442416 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 02 10:02:20.442416 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application return self.get_or_create( Aug 02 10:02:20.442416 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 02 10:02:20.442416 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application with Lock( Aug 02 10:02:20.442416 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 02 10:02:20.442416 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application return self._enter() Aug 02 10:02:20.442416 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 02 10:02:20.442416 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 02 10:02:20.442416 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 02 10:02:20.442416 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application return self.creator() Aug 02 10:02:20.442416 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 02 10:02:20.442416 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application created_value = creator( Aug 02 10:02:20.443580 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 02 10:02:20.443580 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 02 10:02:20.443580 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 02 10:02:20.443580 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 02 10:02:20.443580 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 02 10:02:20.443580 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 02 10:02:20.443580 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 02 10:02:20.443580 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application Aug 02 10:02:20.447420 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e639b251-ed8f-4a1e-b3fb-44885e3dac0c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '891005fa5ff74041b75747d16cc66798', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80378) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:02:20.448682 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e639b251-ed8f-4a1e-b3fb-44885e3dac0c None admin] REQUEST_METHOD: `GET` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:02:20.449268 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e639b251-ed8f-4a1e-b3fb-44885e3dac0c None admin] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:02:20.449881 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e639b251-ed8f-4a1e-b3fb-44885e3dac0c None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:02:20.450472 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: [pid: 80377|app: 0|req: 270/535] 10.222.0.82 () {56 vars in 1111 bytes} [Tue Aug 2 10:02:20 2022] GET /identity/v3/projects/service => generated 88 bytes in 296 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 02 10:02:20.452026 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e382d83b-c8fc-421f-8b7c-0b2cc26e7a21 None None] REQUEST_METHOD: `POST` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:02:20.452328 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e382d83b-c8fc-421f-8b7c-0b2cc26e7a21 None None] SCRIPT_NAME: `/identity` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:02:20.452670 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e382d83b-c8fc-421f-8b7c-0b2cc26e7a21 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:02:20.464805 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e639b251-ed8f-4a1e-b3fb-44885e3dac0c None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:02:20.466450 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e639b251-ed8f-4a1e-b3fb-44885e3dac0c None admin] RBAC: Authorization granted {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:02:20.477616 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.auth.core [None req-e382d83b-c8fc-421f-8b7c-0b2cc26e7a21 None None] MFA Rules not processed for user `891005fa5ff74041b75747d16cc66798`. Rule list: `[]` (Enabled: `True`). {{(pid=80377) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 02 10:02:20.478883 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.fernet_utils [None req-e382d83b-c8fc-421f-8b7c-0b2cc26e7a21 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80377) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 10:02:20.483533 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application [None req-e639b251-ed8f-4a1e-b3fb-44885e3dac0c None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 02 10:02:20.483533 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 02 10:02:20.483533 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 02 10:02:20.483533 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 02 10:02:20.483533 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 02 10:02:20.483533 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 02 10:02:20.483533 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 02 10:02:20.483533 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 02 10:02:20.483533 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 02 10:02:20.483533 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 02 10:02:20.483533 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 02 10:02:20.483533 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 02 10:02:20.483533 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 02 10:02:20.483533 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 02 10:02:20.483533 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 02 10:02:20.486522 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 02 10:02:20.486522 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 02 10:02:20.486522 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 02 10:02:20.486522 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 02 10:02:20.486522 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 02 10:02:20.486522 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 02 10:02:20.486522 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application return self.get_or_create( Aug 02 10:02:20.486522 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 02 10:02:20.486522 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application with Lock( Aug 02 10:02:20.486522 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 02 10:02:20.486522 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application return self._enter() Aug 02 10:02:20.486522 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 02 10:02:20.486522 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 02 10:02:20.486522 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 02 10:02:20.486522 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application return self.creator() Aug 02 10:02:20.486522 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 02 10:02:20.486522 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application created_value = creator( Aug 02 10:02:20.489038 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 02 10:02:20.489038 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 02 10:02:20.489038 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 02 10:02:20.489038 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application Aug 02 10:02:20.489038 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 266/536] 10.222.0.82 () {56 vars in 1108 bytes} [Tue Aug 2 10:02:20 2022] GET /identity/v3/domains/Default => generated 87 bytes in 236 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 02 10:02:20.489038 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e639b251-ed8f-4a1e-b3fb-44885e3dac0c None admin] Authenticating user token {{(pid=80378) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:02:20.563980 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG dbcounter [-] [80377] Writing DB stats keystone:SELECT=100 {{(pid=80377) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 10:02:20.606909 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: [pid: 80377|app: 0|req: 271/537] 10.222.0.82 () {58 vars in 1023 bytes} [Tue Aug 2 10:02:20 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 147 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 02 10:02:20.606909 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e382d83b-c8fc-421f-8b7c-0b2cc26e7a21 None None] Authenticating user token {{(pid=80377) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:02:20.691787 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e639b251-ed8f-4a1e-b3fb-44885e3dac0c None admin] Validating token access rules against request {{(pid=80378) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:02:20.707303 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG dbcounter [-] [80378] Writing DB stats keystone:SELECT=100 {{(pid=80378) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 10:02:20.739259 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e382d83b-c8fc-421f-8b7c-0b2cc26e7a21 None None] Validating token access rules against request {{(pid=80377) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:02:20.762599 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfcbd619-560a-4554-9ce6-19a0bbb5ad86 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '891005fa5ff74041b75747d16cc66798', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80378) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:02:20.764532 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfcbd619-560a-4554-9ce6-19a0bbb5ad86 None admin] REQUEST_METHOD: `GET` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:02:20.765049 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfcbd619-560a-4554-9ce6-19a0bbb5ad86 None admin] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:02:20.765866 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfcbd619-560a-4554-9ce6-19a0bbb5ad86 None admin] PATH_INFO: `/v3/projects` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:02:20.768041 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bfcbd619-560a-4554-9ce6-19a0bbb5ad86 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80378) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 02 10:02:20.768974 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bfcbd619-560a-4554-9ce6-19a0bbb5ad86 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:02:20.773173 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bfcbd619-560a-4554-9ce6-19a0bbb5ad86 None admin] RBAC: Authorization granted {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:02:20.783336 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6780adc9-cbbb-4535-974c-bf93a8f4d9cb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '891005fa5ff74041b75747d16cc66798', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80377) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:02:20.784254 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6780adc9-cbbb-4535-974c-bf93a8f4d9cb None admin] REQUEST_METHOD: `GET` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:02:20.784706 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6780adc9-cbbb-4535-974c-bf93a8f4d9cb None admin] SCRIPT_NAME: `/identity` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:02:20.785172 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6780adc9-cbbb-4535-974c-bf93a8f4d9cb None admin] PATH_INFO: `/v3/domains` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:02:20.786062 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6780adc9-cbbb-4535-974c-bf93a8f4d9cb None admin] RBAC: Adding query filter params (name=Default) {{(pid=80377) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 02 10:02:20.786546 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6780adc9-cbbb-4535-974c-bf93a8f4d9cb None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:02:20.788125 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6780adc9-cbbb-4535-974c-bf93a8f4d9cb None admin] RBAC: Authorization granted {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:02:20.806248 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 267/538] 10.222.0.82 () {56 vars in 1148 bytes} [Tue Aug 2 10:02:20 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 318 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 02 10:02:20.807474 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfcbd619-560a-4554-9ce6-19a0bbb5ad86 None admin] Authenticating user token {{(pid=80378) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:02:20.808396 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: [pid: 80377|app: 0|req: 272/539] 10.222.0.82 () {56 vars in 1109 bytes} [Tue Aug 2 10:02:20 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 205 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 02 10:02:20.809899 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6780adc9-cbbb-4535-974c-bf93a8f4d9cb None admin] Authenticating user token {{(pid=80377) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:02:20.873393 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG dbcounter [-] [80377] Writing DB stats keystone:SELECT=100 {{(pid=80377) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 10:02:20.944737 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6780adc9-cbbb-4535-974c-bf93a8f4d9cb None admin] Validating token access rules against request {{(pid=80377) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:02:20.953718 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfcbd619-560a-4554-9ce6-19a0bbb5ad86 None admin] Validating token access rules against request {{(pid=80378) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:02:20.982220 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ee67c7e-e895-4af6-9041-ab8d93796a4e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '891005fa5ff74041b75747d16cc66798', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80377) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:02:20.983251 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ee67c7e-e895-4af6-9041-ab8d93796a4e None admin] REQUEST_METHOD: `GET` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:02:20.983741 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ee67c7e-e895-4af6-9041-ab8d93796a4e None admin] SCRIPT_NAME: `/identity` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:02:20.984296 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ee67c7e-e895-4af6-9041-ab8d93796a4e None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:02:20.985616 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ee67c7e-e895-4af6-9041-ab8d93796a4e None admin] RBAC: Adding query filter params (role.id=02003a0be60f47f2a8dfea17053af52c, scope.project.id=f1b38ebb804c41c69772c998aae72c41, user.id=fa964af53a2b4fb9ace6afb544d54d40) {{(pid=80377) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 02 10:02:20.986317 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ee67c7e-e895-4af6-9041-ab8d93796a4e None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:02:20.987991 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ee67c7e-e895-4af6-9041-ab8d93796a4e None admin] RBAC: Authorization granted {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:02:21.002131 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: [pid: 80377|app: 0|req: 273/540] 10.222.0.82 () {56 vars in 1374 bytes} [Tue Aug 2 10:02:20 2022] GET /identity/v3/role_assignments?user.id=fa964af53a2b4fb9ace6afb544d54d40&scope.project.id=f1b38ebb804c41c69772c998aae72c41&role.id=02003a0be60f47f2a8dfea17053af52c => generated 259 bytes in 186 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 02 10:02:21.003600 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ee67c7e-e895-4af6-9041-ab8d93796a4e None admin] Authenticating user token {{(pid=80377) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:02:21.008716 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47f6ba1e-12f1-4618-abb5-0fd8d7c95c2f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '891005fa5ff74041b75747d16cc66798', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80378) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:02:21.009177 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47f6ba1e-12f1-4618-abb5-0fd8d7c95c2f None admin] REQUEST_METHOD: `GET` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:02:21.009553 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47f6ba1e-12f1-4618-abb5-0fd8d7c95c2f None admin] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:02:21.009863 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47f6ba1e-12f1-4618-abb5-0fd8d7c95c2f None admin] PATH_INFO: `/v3/roles/service` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:02:21.023083 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47f6ba1e-12f1-4618-abb5-0fd8d7c95c2f None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 02 10:02:21.033994 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application [None req-47f6ba1e-12f1-4618-abb5-0fd8d7c95c2f None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Aug 02 10:02:21.033994 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 02 10:02:21.033994 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 02 10:02:21.033994 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 02 10:02:21.033994 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 02 10:02:21.033994 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 02 10:02:21.033994 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 02 10:02:21.033994 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 02 10:02:21.033994 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 02 10:02:21.033994 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 02 10:02:21.033994 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 02 10:02:21.033994 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 02 10:02:21.033994 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 02 10:02:21.033994 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 02 10:02:21.033994 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 02 10:02:21.033994 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 02 10:02:21.035386 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 02 10:02:21.035386 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 02 10:02:21.035386 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 02 10:02:21.035386 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 02 10:02:21.035386 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 02 10:02:21.035386 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 02 10:02:21.035386 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 02 10:02:21.035386 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 02 10:02:21.035386 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 02 10:02:21.035386 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application return self.get_or_create( Aug 02 10:02:21.035386 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 02 10:02:21.035386 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application with Lock( Aug 02 10:02:21.035386 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 02 10:02:21.035386 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application return self._enter() Aug 02 10:02:21.035386 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 02 10:02:21.035386 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 02 10:02:21.036220 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 02 10:02:21.036220 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application return self.creator() Aug 02 10:02:21.036220 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 02 10:02:21.036220 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application created_value = creator( Aug 02 10:02:21.036220 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 02 10:02:21.036220 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 02 10:02:21.036220 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 02 10:02:21.036220 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 02 10:02:21.036220 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 02 10:02:21.036220 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 02 10:02:21.036220 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Aug 02 10:02:21.036220 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application Aug 02 10:02:21.036220 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 268/541] 10.222.0.82 () {56 vars in 1102 bytes} [Tue Aug 2 10:02:20 2022] GET /identity/v3/roles/service => generated 85 bytes in 223 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 02 10:02:21.039472 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47f6ba1e-12f1-4618-abb5-0fd8d7c95c2f None admin] Authenticating user token {{(pid=80378) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:02:21.112358 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG dbcounter [-] [80378] Writing DB stats keystone:SELECT=100 {{(pid=80378) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 10:02:21.171674 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ee67c7e-e895-4af6-9041-ab8d93796a4e None admin] Validating token access rules against request {{(pid=80377) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:02:21.183992 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG dbcounter [-] [80377] Writing DB stats keystone:SELECT=100 {{(pid=80377) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 10:02:21.217480 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2671f25d-402d-475d-88f9-78172d80b32f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '891005fa5ff74041b75747d16cc66798', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80377) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:02:21.218462 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2671f25d-402d-475d-88f9-78172d80b32f None admin] REQUEST_METHOD: `GET` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:02:21.218728 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2671f25d-402d-475d-88f9-78172d80b32f None admin] SCRIPT_NAME: `/identity` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:02:21.218988 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2671f25d-402d-475d-88f9-78172d80b32f None admin] PATH_INFO: `/v3/projects/service` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:02:21.228624 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2671f25d-402d-475d-88f9-78172d80b32f None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:02:21.230485 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2671f25d-402d-475d-88f9-78172d80b32f None admin] RBAC: Authorization granted {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:02:21.244929 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application [None req-2671f25d-402d-475d-88f9-78172d80b32f None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 02 10:02:21.244929 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 02 10:02:21.244929 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 02 10:02:21.244929 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 02 10:02:21.244929 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 02 10:02:21.244929 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 02 10:02:21.244929 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 02 10:02:21.244929 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 02 10:02:21.244929 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 02 10:02:21.244929 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 02 10:02:21.244929 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 02 10:02:21.244929 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 02 10:02:21.244929 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 02 10:02:21.244929 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 02 10:02:21.244929 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 02 10:02:21.245913 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 02 10:02:21.245913 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 02 10:02:21.245913 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 02 10:02:21.245913 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 02 10:02:21.245913 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 02 10:02:21.245913 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 02 10:02:21.245913 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application return self.get_or_create( Aug 02 10:02:21.245913 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 02 10:02:21.245913 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application with Lock( Aug 02 10:02:21.245913 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 02 10:02:21.245913 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application return self._enter() Aug 02 10:02:21.245913 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 02 10:02:21.245913 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 02 10:02:21.245913 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 02 10:02:21.245913 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application return self.creator() Aug 02 10:02:21.245913 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 02 10:02:21.245913 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application created_value = creator( Aug 02 10:02:21.246763 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 02 10:02:21.246763 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 02 10:02:21.246763 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 02 10:02:21.246763 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 02 10:02:21.246763 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 02 10:02:21.246763 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 02 10:02:21.246763 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 02 10:02:21.246763 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application Aug 02 10:02:21.246763 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: [pid: 80377|app: 0|req: 274/542] 10.222.0.82 () {56 vars in 1111 bytes} [Tue Aug 2 10:02:21 2022] GET /identity/v3/projects/service => generated 88 bytes in 243 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 02 10:02:21.252758 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2671f25d-402d-475d-88f9-78172d80b32f None admin] Authenticating user token {{(pid=80377) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:02:21.320730 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47f6ba1e-12f1-4618-abb5-0fd8d7c95c2f None admin] Validating token access rules against request {{(pid=80378) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:02:21.370796 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-133b2217-0417-4af1-b9a4-43ec7b60173e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '891005fa5ff74041b75747d16cc66798', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80378) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:02:21.371587 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-133b2217-0417-4af1-b9a4-43ec7b60173e None admin] REQUEST_METHOD: `GET` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:02:21.371878 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-133b2217-0417-4af1-b9a4-43ec7b60173e None admin] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:02:21.372160 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-133b2217-0417-4af1-b9a4-43ec7b60173e None admin] PATH_INFO: `/v3/roles` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:02:21.372896 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-133b2217-0417-4af1-b9a4-43ec7b60173e None admin] RBAC: Adding query filter params (name=service) {{(pid=80378) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 02 10:02:21.373464 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-133b2217-0417-4af1-b9a4-43ec7b60173e None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:02:21.375453 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-133b2217-0417-4af1-b9a4-43ec7b60173e None admin] RBAC: Authorization granted {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:02:21.386874 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 269/543] 10.222.0.82 () {56 vars in 1103 bytes} [Tue Aug 2 10:02:21 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 348 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 02 10:02:21.387751 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2671f25d-402d-475d-88f9-78172d80b32f None admin] Validating token access rules against request {{(pid=80377) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:02:21.393958 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-133b2217-0417-4af1-b9a4-43ec7b60173e None admin] Authenticating user token {{(pid=80378) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:02:21.503866 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7abd92b-2384-49fc-8b11-f068e9e0dc4d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '891005fa5ff74041b75747d16cc66798', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80377) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:02:21.504900 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7abd92b-2384-49fc-8b11-f068e9e0dc4d None admin] REQUEST_METHOD: `GET` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:02:21.505403 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7abd92b-2384-49fc-8b11-f068e9e0dc4d None admin] SCRIPT_NAME: `/identity` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:02:21.505951 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7abd92b-2384-49fc-8b11-f068e9e0dc4d None admin] PATH_INFO: `/v3/projects` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:02:21.506931 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7abd92b-2384-49fc-8b11-f068e9e0dc4d None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80377) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 02 10:02:21.507600 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7abd92b-2384-49fc-8b11-f068e9e0dc4d None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:02:21.509396 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7abd92b-2384-49fc-8b11-f068e9e0dc4d None admin] RBAC: Authorization granted {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:02:21.542546 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG dbcounter [-] [80378] Writing DB stats keystone:SELECT=100 {{(pid=80378) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 10:02:21.546466 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: [pid: 80377|app: 0|req: 275/544] 10.222.0.82 () {56 vars in 1148 bytes} [Tue Aug 2 10:02:21 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 294 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 02 10:02:21.548380 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac9ea0d1-1cb4-4c1f-9ac3-9fa00251f24b None None] REQUEST_METHOD: `GET` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:02:21.549841 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac9ea0d1-1cb4-4c1f-9ac3-9fa00251f24b None None] SCRIPT_NAME: `/identity` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:02:21.549841 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac9ea0d1-1cb4-4c1f-9ac3-9fa00251f24b None None] PATH_INFO: `/` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:02:21.550813 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: [pid: 80377|app: 0|req: 276/545] 10.222.0.82 () {54 vars in 926 bytes} [Tue Aug 2 10:02:21 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 02 10:02:21.554846 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac9ea0d1-1cb4-4c1f-9ac3-9fa00251f24b None None] Authenticating user token {{(pid=80377) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:02:21.562829 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-133b2217-0417-4af1-b9a4-43ec7b60173e None admin] Validating token access rules against request {{(pid=80378) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:02:21.605931 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG dbcounter [-] [80377] Writing DB stats keystone:SELECT=100 {{(pid=80377) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 10:02:21.607548 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-814d3cd8-b982-4384-a5c3-f81dd81c1f75 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '891005fa5ff74041b75747d16cc66798', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80378) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:02:21.608111 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-814d3cd8-b982-4384-a5c3-f81dd81c1f75 None admin] REQUEST_METHOD: `GET` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:02:21.608418 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-814d3cd8-b982-4384-a5c3-f81dd81c1f75 None admin] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:02:21.608700 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-814d3cd8-b982-4384-a5c3-f81dd81c1f75 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:02:21.622194 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-814d3cd8-b982-4384-a5c3-f81dd81c1f75 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:02:21.624100 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-814d3cd8-b982-4384-a5c3-f81dd81c1f75 None admin] RBAC: Authorization granted {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:02:21.646724 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application [None req-814d3cd8-b982-4384-a5c3-f81dd81c1f75 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 02 10:02:21.646724 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 02 10:02:21.646724 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 02 10:02:21.646724 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 02 10:02:21.646724 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 02 10:02:21.646724 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 02 10:02:21.646724 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 02 10:02:21.646724 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 02 10:02:21.646724 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 02 10:02:21.646724 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 02 10:02:21.646724 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 02 10:02:21.646724 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 02 10:02:21.646724 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 02 10:02:21.646724 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 02 10:02:21.646724 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 02 10:02:21.647669 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 02 10:02:21.647669 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 02 10:02:21.647669 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 02 10:02:21.647669 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 02 10:02:21.647669 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 02 10:02:21.647669 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 02 10:02:21.647669 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application return self.get_or_create( Aug 02 10:02:21.647669 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 02 10:02:21.647669 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application with Lock( Aug 02 10:02:21.647669 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 02 10:02:21.647669 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application return self._enter() Aug 02 10:02:21.647669 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 02 10:02:21.647669 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 02 10:02:21.647669 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 02 10:02:21.647669 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application return self.creator() Aug 02 10:02:21.647669 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 02 10:02:21.647669 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application created_value = creator( Aug 02 10:02:21.648657 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 02 10:02:21.648657 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 02 10:02:21.648657 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 02 10:02:21.648657 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application Aug 02 10:02:21.653456 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 270/546] 10.222.0.82 () {56 vars in 1108 bytes} [Tue Aug 2 10:02:21 2022] GET /identity/v3/domains/Default => generated 87 bytes in 255 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 02 10:02:21.655242 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f185f5e-4e67-422b-aace-7de064e9e98d None None] REQUEST_METHOD: `POST` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:02:21.655537 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f185f5e-4e67-422b-aace-7de064e9e98d None None] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:02:21.655821 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f185f5e-4e67-422b-aace-7de064e9e98d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:02:21.681430 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.auth.core [None req-3f185f5e-4e67-422b-aace-7de064e9e98d None None] MFA Rules not processed for user `891005fa5ff74041b75747d16cc66798`. Rule list: `[]` (Enabled: `True`). {{(pid=80378) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 02 10:02:21.683388 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.fernet_utils [None req-3f185f5e-4e67-422b-aace-7de064e9e98d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80378) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 10:02:21.723453 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac9ea0d1-1cb4-4c1f-9ac3-9fa00251f24b None None] Validating token access rules against request {{(pid=80377) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:02:21.781232 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b726c70c-9dc3-4a05-8508-8af2ab4a49fc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '891005fa5ff74041b75747d16cc66798', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80377) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:02:21.781795 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b726c70c-9dc3-4a05-8508-8af2ab4a49fc None admin] REQUEST_METHOD: `GET` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:02:21.782086 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b726c70c-9dc3-4a05-8508-8af2ab4a49fc None admin] SCRIPT_NAME: `/identity` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:02:21.783394 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b726c70c-9dc3-4a05-8508-8af2ab4a49fc None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:02:21.783956 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b726c70c-9dc3-4a05-8508-8af2ab4a49fc None admin] RBAC: Adding query filter params (role.id=02003a0be60f47f2a8dfea17053af52c, scope.project.id=f1b38ebb804c41c69772c998aae72c41, user.id=33b87170f0634b67bce60750847b5cbe) {{(pid=80377) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 02 10:02:21.784274 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b726c70c-9dc3-4a05-8508-8af2ab4a49fc None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:02:21.785742 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b726c70c-9dc3-4a05-8508-8af2ab4a49fc None admin] RBAC: Authorization granted {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:02:21.793629 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 271/547] 10.222.0.82 () {58 vars in 1023 bytes} [Tue Aug 2 10:02:21 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 133 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 02 10:02:21.793629 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f185f5e-4e67-422b-aace-7de064e9e98d None None] Authenticating user token {{(pid=80378) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:02:21.804163 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: [pid: 80377|app: 0|req: 277/548] 10.222.0.82 () {56 vars in 1374 bytes} [Tue Aug 2 10:02:21 2022] GET /identity/v3/role_assignments?user.id=33b87170f0634b67bce60750847b5cbe&scope.project.id=f1b38ebb804c41c69772c998aae72c41&role.id=02003a0be60f47f2a8dfea17053af52c => generated 259 bytes in 250 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 02 10:02:21.814852 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b726c70c-9dc3-4a05-8508-8af2ab4a49fc None admin] Authenticating user token {{(pid=80377) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:02:21.999022 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG dbcounter [-] [80377] Writing DB stats keystone:SELECT=100 {{(pid=80377) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 10:02:21.999247 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG dbcounter [-] [80378] Writing DB stats keystone:SELECT=100 {{(pid=80378) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 10:02:22.016653 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b726c70c-9dc3-4a05-8508-8af2ab4a49fc None admin] Validating token access rules against request {{(pid=80377) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:02:22.071041 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a69d2984-5559-474f-bfeb-f5af5654c925 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '891005fa5ff74041b75747d16cc66798', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80377) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:02:22.071408 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f185f5e-4e67-422b-aace-7de064e9e98d None None] Validating token access rules against request {{(pid=80378) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:02:22.071608 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a69d2984-5559-474f-bfeb-f5af5654c925 None admin] REQUEST_METHOD: `GET` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:02:22.071895 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a69d2984-5559-474f-bfeb-f5af5654c925 None admin] SCRIPT_NAME: `/identity` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:02:22.072329 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a69d2984-5559-474f-bfeb-f5af5654c925 None admin] PATH_INFO: `/v3/roles/service` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:02:22.086850 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a69d2984-5559-474f-bfeb-f5af5654c925 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 02 10:02:22.093896 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application [None req-a69d2984-5559-474f-bfeb-f5af5654c925 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Aug 02 10:02:22.093896 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 02 10:02:22.093896 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 02 10:02:22.093896 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 02 10:02:22.093896 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 02 10:02:22.093896 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 02 10:02:22.093896 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 02 10:02:22.093896 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 02 10:02:22.093896 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 02 10:02:22.093896 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 02 10:02:22.093896 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 02 10:02:22.093896 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 02 10:02:22.093896 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 02 10:02:22.093896 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 02 10:02:22.093896 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 02 10:02:22.093896 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 02 10:02:22.095090 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 02 10:02:22.095090 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 02 10:02:22.095090 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 02 10:02:22.095090 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 02 10:02:22.095090 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 02 10:02:22.095090 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 02 10:02:22.095090 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 02 10:02:22.095090 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 02 10:02:22.095090 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 02 10:02:22.095090 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application return self.get_or_create( Aug 02 10:02:22.095090 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 02 10:02:22.095090 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application with Lock( Aug 02 10:02:22.095090 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 02 10:02:22.095090 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application return self._enter() Aug 02 10:02:22.095090 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 02 10:02:22.095090 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 02 10:02:22.095901 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 02 10:02:22.095901 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application return self.creator() Aug 02 10:02:22.095901 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 02 10:02:22.095901 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application created_value = creator( Aug 02 10:02:22.095901 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 02 10:02:22.095901 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 02 10:02:22.095901 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 02 10:02:22.095901 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 02 10:02:22.095901 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 02 10:02:22.095901 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 02 10:02:22.095901 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Aug 02 10:02:22.095901 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application Aug 02 10:02:22.095901 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: [pid: 80377|app: 0|req: 278/549] 10.222.0.82 () {56 vars in 1102 bytes} [Tue Aug 2 10:02:21 2022] GET /identity/v3/roles/service => generated 85 bytes in 281 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 02 10:02:22.103188 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a69d2984-5559-474f-bfeb-f5af5654c925 None admin] Authenticating user token {{(pid=80377) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:02:22.208286 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ddeb48c0-c329-4db8-ba4c-ff0f1010720d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '891005fa5ff74041b75747d16cc66798', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80378) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:02:22.209097 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddeb48c0-c329-4db8-ba4c-ff0f1010720d None admin] REQUEST_METHOD: `GET` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:02:22.209552 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddeb48c0-c329-4db8-ba4c-ff0f1010720d None admin] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:02:22.209924 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddeb48c0-c329-4db8-ba4c-ff0f1010720d None admin] PATH_INFO: `/v3/domains` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:02:22.210936 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ddeb48c0-c329-4db8-ba4c-ff0f1010720d None admin] RBAC: Adding query filter params (name=Default) {{(pid=80378) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 02 10:02:22.211366 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ddeb48c0-c329-4db8-ba4c-ff0f1010720d None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:02:22.212907 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ddeb48c0-c329-4db8-ba4c-ff0f1010720d None admin] RBAC: Authorization granted {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:02:22.243169 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a69d2984-5559-474f-bfeb-f5af5654c925 None admin] Validating token access rules against request {{(pid=80377) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:02:22.248203 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 272/550] 10.222.0.82 () {56 vars in 1109 bytes} [Tue Aug 2 10:02:21 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 458 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 02 10:02:22.256435 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ddeb48c0-c329-4db8-ba4c-ff0f1010720d None admin] Authenticating user token {{(pid=80378) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:02:22.290353 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-368a0052-5cfd-4a29-828e-5dd3f4310ccf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '891005fa5ff74041b75747d16cc66798', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80377) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:02:22.291421 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-368a0052-5cfd-4a29-828e-5dd3f4310ccf None admin] REQUEST_METHOD: `GET` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:02:22.292094 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-368a0052-5cfd-4a29-828e-5dd3f4310ccf None admin] SCRIPT_NAME: `/identity` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:02:22.292598 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-368a0052-5cfd-4a29-828e-5dd3f4310ccf None admin] PATH_INFO: `/v3/roles` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:02:22.293353 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-368a0052-5cfd-4a29-828e-5dd3f4310ccf None admin] RBAC: Adding query filter params (name=service) {{(pid=80377) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 02 10:02:22.293666 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-368a0052-5cfd-4a29-828e-5dd3f4310ccf None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:02:22.295095 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-368a0052-5cfd-4a29-828e-5dd3f4310ccf None admin] RBAC: Authorization granted {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:02:22.305864 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: [pid: 80377|app: 0|req: 279/551] 10.222.0.82 () {56 vars in 1103 bytes} [Tue Aug 2 10:02:22 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 203 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 02 10:02:22.312587 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-368a0052-5cfd-4a29-828e-5dd3f4310ccf None admin] Authenticating user token {{(pid=80377) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:02:22.343523 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG dbcounter [-] [80377] Writing DB stats keystone:SELECT=100 {{(pid=80377) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 10:02:22.510155 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ddeb48c0-c329-4db8-ba4c-ff0f1010720d None admin] Validating token access rules against request {{(pid=80378) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:02:22.546720 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG dbcounter [-] [80378] Writing DB stats keystone:SELECT=100 {{(pid=80378) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 10:02:22.557069 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-528b3e0d-6dde-4964-be1f-d46f23a8c9ea None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '891005fa5ff74041b75747d16cc66798', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80378) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:02:22.557741 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-528b3e0d-6dde-4964-be1f-d46f23a8c9ea None admin] REQUEST_METHOD: `GET` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:02:22.558048 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-528b3e0d-6dde-4964-be1f-d46f23a8c9ea None admin] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:02:22.558327 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-528b3e0d-6dde-4964-be1f-d46f23a8c9ea None admin] PATH_INFO: `/v3/users/swift` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:02:22.571125 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-368a0052-5cfd-4a29-828e-5dd3f4310ccf None admin] Validating token access rules against request {{(pid=80377) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:02:22.580402 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-528b3e0d-6dde-4964-be1f-d46f23a8c9ea None admin] RBAC: Authorizing `identity:get_user(user_id=swift)` {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:02:22.581917 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-528b3e0d-6dde-4964-be1f-d46f23a8c9ea None admin] RBAC: Authorization granted {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:02:22.600168 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application [None req-528b3e0d-6dde-4964-be1f-d46f23a8c9ea None admin] Could not find user: swift.: keystone.exception.UserNotFound: Could not find user: swift. Aug 02 10:02:22.600168 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 02 10:02:22.600168 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 02 10:02:22.600168 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 02 10:02:22.600168 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 02 10:02:22.600168 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 02 10:02:22.600168 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 02 10:02:22.600168 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 02 10:02:22.600168 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 02 10:02:22.600168 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 02 10:02:22.600168 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 02 10:02:22.600168 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 02 10:02:22.600168 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 02 10:02:22.600168 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 02 10:02:22.600168 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 02 10:02:22.600168 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 02 10:02:22.601283 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 02 10:02:22.601283 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 02 10:02:22.601283 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 02 10:02:22.601283 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 02 10:02:22.601283 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 02 10:02:22.601283 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 02 10:02:22.601283 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 02 10:02:22.601283 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 02 10:02:22.601283 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 02 10:02:22.601283 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application return self.get_or_create( Aug 02 10:02:22.601283 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 02 10:02:22.601283 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application with Lock( Aug 02 10:02:22.601283 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 02 10:02:22.601283 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application return self._enter() Aug 02 10:02:22.601283 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 02 10:02:22.601283 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 02 10:02:22.601283 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 02 10:02:22.601283 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application return self.creator() Aug 02 10:02:22.602636 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 02 10:02:22.602636 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application created_value = creator( Aug 02 10:02:22.602636 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 02 10:02:22.602636 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 02 10:02:22.602636 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 02 10:02:22.602636 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 02 10:02:22.602636 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 02 10:02:22.602636 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 02 10:02:22.602636 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: swift. Aug 02 10:02:22.602636 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application Aug 02 10:02:22.602636 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 273/552] 10.222.0.82 () {56 vars in 1096 bytes} [Tue Aug 2 10:02:22 2022] GET /identity/v3/users/swift => generated 83 bytes in 345 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 02 10:02:22.611820 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-528b3e0d-6dde-4964-be1f-d46f23a8c9ea None admin] Authenticating user token {{(pid=80378) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:02:22.643340 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-518ded35-5ac1-4c84-9318-d90f10fe9fa0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '891005fa5ff74041b75747d16cc66798', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80377) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:02:22.644007 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-518ded35-5ac1-4c84-9318-d90f10fe9fa0 None admin] REQUEST_METHOD: `GET` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:02:22.644290 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-518ded35-5ac1-4c84-9318-d90f10fe9fa0 None admin] SCRIPT_NAME: `/identity` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:02:22.644561 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-518ded35-5ac1-4c84-9318-d90f10fe9fa0 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:02:22.669076 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-518ded35-5ac1-4c84-9318-d90f10fe9fa0 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:02:22.670612 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-518ded35-5ac1-4c84-9318-d90f10fe9fa0 None admin] RBAC: Authorization granted {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:02:22.700082 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application [None req-518ded35-5ac1-4c84-9318-d90f10fe9fa0 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 02 10:02:22.700082 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 02 10:02:22.700082 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 02 10:02:22.700082 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 02 10:02:22.700082 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 02 10:02:22.700082 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 02 10:02:22.700082 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 02 10:02:22.700082 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 02 10:02:22.700082 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 02 10:02:22.700082 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 02 10:02:22.700082 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 02 10:02:22.700082 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 02 10:02:22.700082 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 02 10:02:22.700082 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 02 10:02:22.700082 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 02 10:02:22.701261 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 02 10:02:22.701261 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 02 10:02:22.701261 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 02 10:02:22.701261 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 02 10:02:22.701261 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 02 10:02:22.701261 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 02 10:02:22.701261 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application return self.get_or_create( Aug 02 10:02:22.701261 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 02 10:02:22.701261 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application with Lock( Aug 02 10:02:22.701261 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 02 10:02:22.701261 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application return self._enter() Aug 02 10:02:22.701261 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 02 10:02:22.701261 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 02 10:02:22.701261 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 02 10:02:22.701261 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application return self.creator() Aug 02 10:02:22.701261 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 02 10:02:22.701261 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application created_value = creator( Aug 02 10:02:22.702391 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 02 10:02:22.702391 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 02 10:02:22.702391 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 02 10:02:22.702391 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application Aug 02 10:02:22.705883 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: [pid: 80377|app: 0|req: 280/553] 10.222.0.82 () {56 vars in 1108 bytes} [Tue Aug 2 10:02:22 2022] GET /identity/v3/domains/Default => generated 87 bytes in 394 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 02 10:02:22.712195 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-518ded35-5ac1-4c84-9318-d90f10fe9fa0 None admin] Authenticating user token {{(pid=80377) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:02:22.855704 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-528b3e0d-6dde-4964-be1f-d46f23a8c9ea None admin] Validating token access rules against request {{(pid=80378) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:02:22.866209 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG dbcounter [-] [80377] Writing DB stats keystone:SELECT=100 {{(pid=80377) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 10:02:22.932686 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-518ded35-5ac1-4c84-9318-d90f10fe9fa0 None admin] Validating token access rules against request {{(pid=80377) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:02:22.934123 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f525630-ea2f-4cf1-a433-916b6c859a8f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '891005fa5ff74041b75747d16cc66798', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80378) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:02:22.934876 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f525630-ea2f-4cf1-a433-916b6c859a8f None admin] REQUEST_METHOD: `GET` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:02:22.935251 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f525630-ea2f-4cf1-a433-916b6c859a8f None admin] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:02:22.935626 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f525630-ea2f-4cf1-a433-916b6c859a8f None admin] PATH_INFO: `/v3/users` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:02:22.936520 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f525630-ea2f-4cf1-a433-916b6c859a8f None admin] RBAC: Adding query filter params (domain_id=default, name=swift) {{(pid=80378) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 02 10:02:22.936930 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f525630-ea2f-4cf1-a433-916b6c859a8f None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:02:22.938619 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f525630-ea2f-4cf1-a433-916b6c859a8f None admin] RBAC: Authorization granted {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:02:22.980337 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 274/554] 10.222.0.82 () {56 vars in 1135 bytes} [Tue Aug 2 10:02:22 2022] GET /identity/v3/users?domain_id=default&name=swift => generated 370 bytes in 369 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 02 10:02:22.986301 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f525630-ea2f-4cf1-a433-916b6c859a8f None admin] Authenticating user token {{(pid=80378) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:02:23.012858 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6f89e72-2bb6-4ca8-a430-dc6843f93ddb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '891005fa5ff74041b75747d16cc66798', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80377) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:02:23.013617 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6f89e72-2bb6-4ca8-a430-dc6843f93ddb None admin] REQUEST_METHOD: `GET` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:02:23.014078 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6f89e72-2bb6-4ca8-a430-dc6843f93ddb None admin] SCRIPT_NAME: `/identity` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:02:23.014369 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6f89e72-2bb6-4ca8-a430-dc6843f93ddb None admin] PATH_INFO: `/v3/domains` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:02:23.015330 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6f89e72-2bb6-4ca8-a430-dc6843f93ddb None admin] RBAC: Adding query filter params (name=Default) {{(pid=80377) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 02 10:02:23.015771 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6f89e72-2bb6-4ca8-a430-dc6843f93ddb None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:02:23.019860 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6f89e72-2bb6-4ca8-a430-dc6843f93ddb None admin] RBAC: Authorization granted {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:02:23.059416 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: [pid: 80377|app: 0|req: 281/555] 10.222.0.82 () {56 vars in 1109 bytes} [Tue Aug 2 10:02:22 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 348 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 02 10:02:23.061455 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6f89e72-2bb6-4ca8-a430-dc6843f93ddb None admin] Authenticating user token {{(pid=80377) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:02:23.102299 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG dbcounter [-] [80378] Writing DB stats keystone:SELECT=100 {{(pid=80378) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 10:02:23.169560 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f525630-ea2f-4cf1-a433-916b6c859a8f None admin] Validating token access rules against request {{(pid=80378) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:02:23.210867 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30f7d4a1-312b-4fd5-a7b8-58f680180016 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '891005fa5ff74041b75747d16cc66798', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80378) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:02:23.211846 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30f7d4a1-312b-4fd5-a7b8-58f680180016 None admin] REQUEST_METHOD: `GET` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:02:23.212196 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30f7d4a1-312b-4fd5-a7b8-58f680180016 None admin] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:02:23.212559 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30f7d4a1-312b-4fd5-a7b8-58f680180016 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:02:23.295137 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30f7d4a1-312b-4fd5-a7b8-58f680180016 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:02:23.296794 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30f7d4a1-312b-4fd5-a7b8-58f680180016 None admin] RBAC: Authorization granted {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:02:23.323129 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6f89e72-2bb6-4ca8-a430-dc6843f93ddb None admin] Validating token access rules against request {{(pid=80377) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:02:23.366415 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG dbcounter [-] [80378] Writing DB stats keystone:SELECT=100 {{(pid=80378) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 10:02:23.384315 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG dbcounter [-] [80377] Writing DB stats keystone:SELECT=100 {{(pid=80377) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 10:02:23.393826 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b00d4d4-95f3-47da-a951-622f50851805 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '891005fa5ff74041b75747d16cc66798', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80377) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:02:23.394771 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b00d4d4-95f3-47da-a951-622f50851805 None admin] REQUEST_METHOD: `GET` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:02:23.395168 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b00d4d4-95f3-47da-a951-622f50851805 None admin] SCRIPT_NAME: `/identity` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:02:23.395556 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b00d4d4-95f3-47da-a951-622f50851805 None admin] PATH_INFO: `/v3/users/nova` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:02:23.408569 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b00d4d4-95f3-47da-a951-622f50851805 None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:02:23.410883 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b00d4d4-95f3-47da-a951-622f50851805 None admin] RBAC: Authorization granted {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:02:23.421775 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 275/556] 10.222.0.82 () {58 vars in 1282 bytes} [Tue Aug 2 10:02:22 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 436 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 02 10:02:23.423091 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66bfd077-e143-4184-a30c-fbb1ec063f01 None None] REQUEST_METHOD: `GET` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:02:23.423380 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66bfd077-e143-4184-a30c-fbb1ec063f01 None None] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:02:23.423653 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66bfd077-e143-4184-a30c-fbb1ec063f01 None None] PATH_INFO: `/` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:02:23.425381 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 276/557] 10.222.0.82 () {54 vars in 926 bytes} [Tue Aug 2 10:02:23 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 02 10:02:23.431647 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4542384b-89f6-45e9-b00c-0d9e279e48c7 None None] REQUEST_METHOD: `POST` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:02:23.432054 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4542384b-89f6-45e9-b00c-0d9e279e48c7 None None] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:02:23.432325 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4542384b-89f6-45e9-b00c-0d9e279e48c7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:02:23.440902 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application [None req-7b00d4d4-95f3-47da-a951-622f50851805 None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Aug 02 10:02:23.440902 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 02 10:02:23.440902 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 02 10:02:23.440902 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 02 10:02:23.440902 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 02 10:02:23.440902 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 02 10:02:23.440902 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 02 10:02:23.440902 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 02 10:02:23.440902 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 02 10:02:23.440902 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 02 10:02:23.440902 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 02 10:02:23.440902 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 02 10:02:23.440902 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 02 10:02:23.440902 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 02 10:02:23.440902 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 02 10:02:23.440902 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 02 10:02:23.442525 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 02 10:02:23.442525 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 02 10:02:23.442525 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 02 10:02:23.442525 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 02 10:02:23.442525 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 02 10:02:23.442525 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 02 10:02:23.442525 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 02 10:02:23.442525 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 02 10:02:23.442525 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 02 10:02:23.442525 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application return self.get_or_create( Aug 02 10:02:23.442525 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 02 10:02:23.442525 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application with Lock( Aug 02 10:02:23.442525 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 02 10:02:23.442525 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application return self._enter() Aug 02 10:02:23.442525 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 02 10:02:23.442525 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 02 10:02:23.442525 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 02 10:02:23.442525 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application return self.creator() Aug 02 10:02:23.443467 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 02 10:02:23.443467 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application created_value = creator( Aug 02 10:02:23.443467 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 02 10:02:23.443467 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 02 10:02:23.443467 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 02 10:02:23.443467 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 02 10:02:23.443467 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 02 10:02:23.443467 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 02 10:02:23.443467 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Aug 02 10:02:23.443467 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application Aug 02 10:02:23.443467 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: [pid: 80377|app: 0|req: 282/558] 10.222.0.82 () {56 vars in 1093 bytes} [Tue Aug 2 10:02:23 2022] GET /identity/v3/users/nova => generated 82 bytes in 382 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 02 10:02:23.443467 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b00d4d4-95f3-47da-a951-622f50851805 None admin] Authenticating user token {{(pid=80377) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:02:23.458026 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.auth.core [None req-4542384b-89f6-45e9-b00c-0d9e279e48c7 None None] MFA Rules not processed for user `891005fa5ff74041b75747d16cc66798`. Rule list: `[]` (Enabled: `True`). {{(pid=80378) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 02 10:02:23.459936 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.fernet_utils [None req-4542384b-89f6-45e9-b00c-0d9e279e48c7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80378) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 10:02:23.553582 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 277/559] 10.222.0.82 () {58 vars in 1023 bytes} [Tue Aug 2 10:02:23 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 123 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Aug 02 10:02:23.554812 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4542384b-89f6-45e9-b00c-0d9e279e48c7 None None] Authenticating user token {{(pid=80378) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:02:23.575420 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b00d4d4-95f3-47da-a951-622f50851805 None admin] Validating token access rules against request {{(pid=80377) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:02:23.635056 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0c677d9-8575-4f94-827b-b0e6ae5753dc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '891005fa5ff74041b75747d16cc66798', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80377) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:02:23.637692 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0c677d9-8575-4f94-827b-b0e6ae5753dc None admin] REQUEST_METHOD: `GET` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:02:23.638627 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0c677d9-8575-4f94-827b-b0e6ae5753dc None admin] SCRIPT_NAME: `/identity` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:02:23.639744 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0c677d9-8575-4f94-827b-b0e6ae5753dc None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:02:23.655833 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0c677d9-8575-4f94-827b-b0e6ae5753dc None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:02:23.658034 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0c677d9-8575-4f94-827b-b0e6ae5753dc None admin] RBAC: Authorization granted {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:02:23.673465 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG dbcounter [-] [80378] Writing DB stats keystone:SELECT=100 {{(pid=80378) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 10:02:23.679306 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application [None req-a0c677d9-8575-4f94-827b-b0e6ae5753dc None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 02 10:02:23.679306 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 02 10:02:23.679306 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 02 10:02:23.679306 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 02 10:02:23.679306 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 02 10:02:23.679306 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 02 10:02:23.679306 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 02 10:02:23.679306 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 02 10:02:23.679306 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 02 10:02:23.679306 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 02 10:02:23.679306 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 02 10:02:23.679306 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 02 10:02:23.679306 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 02 10:02:23.679306 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 02 10:02:23.679306 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 02 10:02:23.681136 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 02 10:02:23.681136 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 02 10:02:23.681136 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 02 10:02:23.681136 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 02 10:02:23.681136 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 02 10:02:23.681136 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 02 10:02:23.681136 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application return self.get_or_create( Aug 02 10:02:23.681136 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 02 10:02:23.681136 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application with Lock( Aug 02 10:02:23.681136 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 02 10:02:23.681136 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application return self._enter() Aug 02 10:02:23.681136 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 02 10:02:23.681136 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 02 10:02:23.681136 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 02 10:02:23.681136 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application return self.creator() Aug 02 10:02:23.681136 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 02 10:02:23.681136 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application created_value = creator( Aug 02 10:02:23.682924 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 02 10:02:23.682924 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 02 10:02:23.682924 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 02 10:02:23.682924 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application Aug 02 10:02:23.682924 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: [pid: 80377|app: 0|req: 283/560] 10.222.0.82 () {56 vars in 1108 bytes} [Tue Aug 2 10:02:23 2022] GET /identity/v3/domains/Default => generated 87 bytes in 238 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 02 10:02:23.682924 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0c677d9-8575-4f94-827b-b0e6ae5753dc None admin] Authenticating user token {{(pid=80377) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:02:23.722638 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4542384b-89f6-45e9-b00c-0d9e279e48c7 None None] Validating token access rules against request {{(pid=80378) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:02:23.757963 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG dbcounter [-] [80377] Writing DB stats keystone:SELECT=100 {{(pid=80377) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 10:02:23.784186 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-671b4d44-c94e-48e2-b63c-e5acf4a82ffe None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '891005fa5ff74041b75747d16cc66798', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80378) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:02:23.785052 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-671b4d44-c94e-48e2-b63c-e5acf4a82ffe None admin] REQUEST_METHOD: `GET` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:02:23.785363 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-671b4d44-c94e-48e2-b63c-e5acf4a82ffe None admin] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:02:23.785669 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-671b4d44-c94e-48e2-b63c-e5acf4a82ffe None admin] PATH_INFO: `/v3/users` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:02:23.786383 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-671b4d44-c94e-48e2-b63c-e5acf4a82ffe None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=80378) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 02 10:02:23.786944 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-671b4d44-c94e-48e2-b63c-e5acf4a82ffe None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:02:23.788312 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-671b4d44-c94e-48e2-b63c-e5acf4a82ffe None admin] RBAC: Authorization granted {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:02:23.812664 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 278/561] 10.222.0.82 () {56 vars in 1133 bytes} [Tue Aug 2 10:02:23 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 258 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 02 10:02:23.814737 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-671b4d44-c94e-48e2-b63c-e5acf4a82ffe None admin] Authenticating user token {{(pid=80378) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:02:23.877700 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0c677d9-8575-4f94-827b-b0e6ae5753dc None admin] Validating token access rules against request {{(pid=80377) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:02:23.927013 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d880676e-7208-4412-8817-220f172c3a88 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '891005fa5ff74041b75747d16cc66798', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80377) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:02:23.928273 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d880676e-7208-4412-8817-220f172c3a88 None admin] REQUEST_METHOD: `GET` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:02:23.928754 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d880676e-7208-4412-8817-220f172c3a88 None admin] SCRIPT_NAME: `/identity` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:02:23.929227 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d880676e-7208-4412-8817-220f172c3a88 None admin] PATH_INFO: `/v3/roles/service` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:02:23.951085 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d880676e-7208-4412-8817-220f172c3a88 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 02 10:02:23.961455 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application [None req-d880676e-7208-4412-8817-220f172c3a88 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Aug 02 10:02:23.961455 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 02 10:02:23.961455 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 02 10:02:23.961455 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 02 10:02:23.961455 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 02 10:02:23.961455 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 02 10:02:23.961455 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 02 10:02:23.961455 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 02 10:02:23.961455 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 02 10:02:23.961455 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 02 10:02:23.961455 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 02 10:02:23.961455 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 02 10:02:23.961455 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 02 10:02:23.961455 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 02 10:02:23.961455 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 02 10:02:23.961455 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 02 10:02:23.963761 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 02 10:02:23.963761 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 02 10:02:23.963761 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 02 10:02:23.963761 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 02 10:02:23.963761 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 02 10:02:23.963761 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 02 10:02:23.963761 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 02 10:02:23.963761 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 02 10:02:23.963761 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 02 10:02:23.963761 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application return self.get_or_create( Aug 02 10:02:23.963761 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 02 10:02:23.963761 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application with Lock( Aug 02 10:02:23.963761 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 02 10:02:23.963761 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application return self._enter() Aug 02 10:02:23.963761 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 02 10:02:23.963761 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 02 10:02:23.964902 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 02 10:02:23.964902 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application return self.creator() Aug 02 10:02:23.964902 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 02 10:02:23.964902 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application created_value = creator( Aug 02 10:02:23.964902 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 02 10:02:23.964902 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 02 10:02:23.964902 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 02 10:02:23.964902 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 02 10:02:23.964902 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 02 10:02:23.964902 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 02 10:02:23.964902 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Aug 02 10:02:23.964902 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application Aug 02 10:02:23.964902 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: [pid: 80377|app: 0|req: 284/562] 10.222.0.82 () {56 vars in 1102 bytes} [Tue Aug 2 10:02:23 2022] GET /identity/v3/roles/service => generated 85 bytes in 282 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 02 10:02:23.964902 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d880676e-7208-4412-8817-220f172c3a88 None admin] Authenticating user token {{(pid=80377) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:02:24.003308 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-671b4d44-c94e-48e2-b63c-e5acf4a82ffe None admin] Validating token access rules against request {{(pid=80378) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:02:24.053220 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad6b9125-c89c-4e91-9c19-e6e62822f9d4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '891005fa5ff74041b75747d16cc66798', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80378) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:02:24.053877 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad6b9125-c89c-4e91-9c19-e6e62822f9d4 None admin] REQUEST_METHOD: `GET` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:02:24.054171 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad6b9125-c89c-4e91-9c19-e6e62822f9d4 None admin] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:02:24.054434 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad6b9125-c89c-4e91-9c19-e6e62822f9d4 None admin] PATH_INFO: `/v3/domains` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:02:24.055610 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad6b9125-c89c-4e91-9c19-e6e62822f9d4 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80378) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 02 10:02:24.056231 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad6b9125-c89c-4e91-9c19-e6e62822f9d4 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:02:24.057728 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad6b9125-c89c-4e91-9c19-e6e62822f9d4 None admin] RBAC: Authorization granted {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:02:24.060904 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG dbcounter [-] [80378] Writing DB stats keystone:SELECT=100 {{(pid=80378) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 10:02:24.078168 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 279/563] 10.222.0.82 () {56 vars in 1109 bytes} [Tue Aug 2 10:02:23 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 265 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 02 10:02:24.079999 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-206bb566-9e81-4d37-8489-875d04d45b94 None None] REQUEST_METHOD: `GET` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:02:24.080288 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-206bb566-9e81-4d37-8489-875d04d45b94 None None] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:02:24.080608 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-206bb566-9e81-4d37-8489-875d04d45b94 None None] PATH_INFO: `/` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:02:24.087536 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 280/564] 10.222.0.82 () {54 vars in 926 bytes} [Tue Aug 2 10:02:24 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 02 10:02:24.087536 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-206bb566-9e81-4d37-8489-875d04d45b94 None None] Authenticating user token {{(pid=80378) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:02:24.134159 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG dbcounter [-] [80377] Writing DB stats keystone:SELECT=100 {{(pid=80377) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 10:02:24.138539 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d880676e-7208-4412-8817-220f172c3a88 None admin] Validating token access rules against request {{(pid=80377) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:02:24.184834 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50795ed7-7c41-4abb-97ef-e1bbc848fc11 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '891005fa5ff74041b75747d16cc66798', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80377) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:02:24.185897 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50795ed7-7c41-4abb-97ef-e1bbc848fc11 None admin] REQUEST_METHOD: `GET` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:02:24.185897 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50795ed7-7c41-4abb-97ef-e1bbc848fc11 None admin] SCRIPT_NAME: `/identity` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:02:24.186182 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50795ed7-7c41-4abb-97ef-e1bbc848fc11 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:02:24.200746 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50795ed7-7c41-4abb-97ef-e1bbc848fc11 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:02:24.202402 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50795ed7-7c41-4abb-97ef-e1bbc848fc11 None admin] RBAC: Authorization granted {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:02:24.213169 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-206bb566-9e81-4d37-8489-875d04d45b94 None None] Validating token access rules against request {{(pid=80378) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:02:24.220314 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application [None req-50795ed7-7c41-4abb-97ef-e1bbc848fc11 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 02 10:02:24.220314 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 02 10:02:24.220314 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 02 10:02:24.220314 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 02 10:02:24.220314 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 02 10:02:24.220314 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 02 10:02:24.220314 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 02 10:02:24.220314 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 02 10:02:24.220314 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 02 10:02:24.220314 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 02 10:02:24.220314 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 02 10:02:24.220314 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 02 10:02:24.220314 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 02 10:02:24.220314 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 02 10:02:24.220314 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 02 10:02:24.223938 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 02 10:02:24.223938 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 02 10:02:24.223938 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 02 10:02:24.223938 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 02 10:02:24.223938 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 02 10:02:24.223938 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 02 10:02:24.223938 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application return self.get_or_create( Aug 02 10:02:24.223938 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 02 10:02:24.223938 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application with Lock( Aug 02 10:02:24.223938 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 02 10:02:24.223938 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application return self._enter() Aug 02 10:02:24.223938 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 02 10:02:24.223938 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 02 10:02:24.223938 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 02 10:02:24.223938 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application return self.creator() Aug 02 10:02:24.223938 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 02 10:02:24.223938 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application created_value = creator( Aug 02 10:02:24.225519 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 02 10:02:24.225519 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 02 10:02:24.225519 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 02 10:02:24.225519 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application Aug 02 10:02:24.225519 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: [pid: 80377|app: 0|req: 285/565] 10.222.0.82 () {56 vars in 1108 bytes} [Tue Aug 2 10:02:23 2022] GET /identity/v3/domains/Default => generated 87 bytes in 258 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 02 10:02:24.225519 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50795ed7-7c41-4abb-97ef-e1bbc848fc11 None admin] Authenticating user token {{(pid=80377) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:02:24.284094 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7e9c0bd-99bc-44ea-9d3a-f99f15e5a0cc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '891005fa5ff74041b75747d16cc66798', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80378) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:02:24.284657 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7e9c0bd-99bc-44ea-9d3a-f99f15e5a0cc None admin] REQUEST_METHOD: `GET` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:02:24.284988 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7e9c0bd-99bc-44ea-9d3a-f99f15e5a0cc None admin] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:02:24.285529 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7e9c0bd-99bc-44ea-9d3a-f99f15e5a0cc None admin] PATH_INFO: `/v3/roles` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:02:24.286592 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7e9c0bd-99bc-44ea-9d3a-f99f15e5a0cc None admin] RBAC: Adding query filter params (name=service) {{(pid=80378) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 02 10:02:24.287038 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7e9c0bd-99bc-44ea-9d3a-f99f15e5a0cc None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:02:24.289899 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7e9c0bd-99bc-44ea-9d3a-f99f15e5a0cc None admin] RBAC: Authorization granted {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:02:24.298749 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 281/566] 10.222.0.82 () {56 vars in 1103 bytes} [Tue Aug 2 10:02:24 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 216 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 02 10:02:24.304615 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6783c340-8e45-400c-b588-94fc66f13b26 None None] REQUEST_METHOD: `POST` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:02:24.304829 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6783c340-8e45-400c-b588-94fc66f13b26 None None] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:02:24.304993 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6783c340-8e45-400c-b588-94fc66f13b26 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:02:24.325932 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.auth.core [None req-6783c340-8e45-400c-b588-94fc66f13b26 None None] MFA Rules not processed for user `891005fa5ff74041b75747d16cc66798`. Rule list: `[]` (Enabled: `True`). {{(pid=80378) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 02 10:02:24.327130 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.fernet_utils [None req-6783c340-8e45-400c-b588-94fc66f13b26 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80378) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 10:02:24.389297 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50795ed7-7c41-4abb-97ef-e1bbc848fc11 None admin] Validating token access rules against request {{(pid=80377) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:02:24.408552 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG dbcounter [-] [80378] Writing DB stats keystone:SELECT=100 {{(pid=80378) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 10:02:24.441970 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 282/567] 10.222.0.82 () {58 vars in 1023 bytes} [Tue Aug 2 10:02:24 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 137 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 02 10:02:24.441970 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6783c340-8e45-400c-b588-94fc66f13b26 None None] Authenticating user token {{(pid=80378) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:02:24.450181 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a756bef-7d99-4a9b-9e17-99b45199cc4e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '891005fa5ff74041b75747d16cc66798', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80377) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:02:24.450939 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a756bef-7d99-4a9b-9e17-99b45199cc4e None admin] REQUEST_METHOD: `GET` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:02:24.451286 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a756bef-7d99-4a9b-9e17-99b45199cc4e None admin] SCRIPT_NAME: `/identity` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:02:24.451626 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a756bef-7d99-4a9b-9e17-99b45199cc4e None admin] PATH_INFO: `/v3/projects/service` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:02:24.460488 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a756bef-7d99-4a9b-9e17-99b45199cc4e None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:02:24.462126 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a756bef-7d99-4a9b-9e17-99b45199cc4e None admin] RBAC: Authorization granted {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:02:24.479315 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application [None req-3a756bef-7d99-4a9b-9e17-99b45199cc4e None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 02 10:02:24.479315 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 02 10:02:24.479315 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 02 10:02:24.479315 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 02 10:02:24.479315 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 02 10:02:24.479315 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 02 10:02:24.479315 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 02 10:02:24.479315 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 02 10:02:24.479315 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 02 10:02:24.479315 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 02 10:02:24.479315 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 02 10:02:24.479315 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 02 10:02:24.479315 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 02 10:02:24.479315 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 02 10:02:24.479315 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 02 10:02:24.481054 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 02 10:02:24.481054 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 02 10:02:24.481054 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 02 10:02:24.481054 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 02 10:02:24.481054 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 02 10:02:24.481054 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 02 10:02:24.481054 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application return self.get_or_create( Aug 02 10:02:24.481054 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 02 10:02:24.481054 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application with Lock( Aug 02 10:02:24.481054 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 02 10:02:24.481054 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application return self._enter() Aug 02 10:02:24.481054 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 02 10:02:24.481054 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 02 10:02:24.481054 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 02 10:02:24.481054 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application return self.creator() Aug 02 10:02:24.481054 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 02 10:02:24.481054 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application created_value = creator( Aug 02 10:02:24.482901 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 02 10:02:24.482901 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 02 10:02:24.482901 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 02 10:02:24.482901 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 02 10:02:24.482901 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 02 10:02:24.482901 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 02 10:02:24.482901 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 02 10:02:24.482901 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application Aug 02 10:02:24.482901 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: [pid: 80377|app: 0|req: 286/568] 10.222.0.82 () {56 vars in 1111 bytes} [Tue Aug 2 10:02:24 2022] GET /identity/v3/projects/service => generated 88 bytes in 258 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 02 10:02:24.482901 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a756bef-7d99-4a9b-9e17-99b45199cc4e None admin] Authenticating user token {{(pid=80377) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:02:24.498197 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG dbcounter [-] [80377] Writing DB stats keystone:SELECT=100 {{(pid=80377) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 10:02:24.614888 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6783c340-8e45-400c-b588-94fc66f13b26 None None] Validating token access rules against request {{(pid=80378) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:02:24.636854 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a756bef-7d99-4a9b-9e17-99b45199cc4e None admin] Validating token access rules against request {{(pid=80377) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:02:24.664231 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7c087a7-9c39-4fae-a1e1-818a63043f68 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '891005fa5ff74041b75747d16cc66798', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80378) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:02:24.664861 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7c087a7-9c39-4fae-a1e1-818a63043f68 None admin] REQUEST_METHOD: `GET` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:02:24.665142 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7c087a7-9c39-4fae-a1e1-818a63043f68 None admin] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:02:24.665415 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7c087a7-9c39-4fae-a1e1-818a63043f68 None admin] PATH_INFO: `/v3/domains` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:02:24.666201 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7c087a7-9c39-4fae-a1e1-818a63043f68 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80378) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 02 10:02:24.666558 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7c087a7-9c39-4fae-a1e1-818a63043f68 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:02:24.668810 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7c087a7-9c39-4fae-a1e1-818a63043f68 None admin] RBAC: Authorization granted {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:02:24.681044 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51c495cd-0c13-4be3-b5f4-1b9356fb6fec None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '891005fa5ff74041b75747d16cc66798', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80377) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:02:24.681693 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51c495cd-0c13-4be3-b5f4-1b9356fb6fec None admin] REQUEST_METHOD: `GET` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:02:24.682003 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51c495cd-0c13-4be3-b5f4-1b9356fb6fec None admin] SCRIPT_NAME: `/identity` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:02:24.682277 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51c495cd-0c13-4be3-b5f4-1b9356fb6fec None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:02:24.682582 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 283/569] 10.222.0.82 () {56 vars in 1109 bytes} [Tue Aug 2 10:02:24 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 246 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 02 10:02:24.683588 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7c087a7-9c39-4fae-a1e1-818a63043f68 None admin] Authenticating user token {{(pid=80378) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:02:24.698805 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51c495cd-0c13-4be3-b5f4-1b9356fb6fec None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:02:24.700223 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51c495cd-0c13-4be3-b5f4-1b9356fb6fec None admin] RBAC: Authorization granted {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:02:24.716310 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application [None req-51c495cd-0c13-4be3-b5f4-1b9356fb6fec None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 02 10:02:24.716310 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 02 10:02:24.716310 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 02 10:02:24.716310 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 02 10:02:24.716310 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 02 10:02:24.716310 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 02 10:02:24.716310 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 02 10:02:24.716310 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 02 10:02:24.716310 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 02 10:02:24.716310 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 02 10:02:24.716310 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 02 10:02:24.716310 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 02 10:02:24.716310 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 02 10:02:24.716310 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 02 10:02:24.716310 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 02 10:02:24.718120 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 02 10:02:24.718120 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 02 10:02:24.718120 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 02 10:02:24.718120 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 02 10:02:24.718120 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 02 10:02:24.718120 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 02 10:02:24.718120 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application return self.get_or_create( Aug 02 10:02:24.718120 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 02 10:02:24.718120 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application with Lock( Aug 02 10:02:24.718120 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 02 10:02:24.718120 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application return self._enter() Aug 02 10:02:24.718120 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 02 10:02:24.718120 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 02 10:02:24.718120 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 02 10:02:24.718120 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application return self.creator() Aug 02 10:02:24.718120 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 02 10:02:24.718120 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application created_value = creator( Aug 02 10:02:24.720414 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 02 10:02:24.720414 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 02 10:02:24.720414 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 02 10:02:24.720414 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application Aug 02 10:02:24.720414 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: [pid: 80377|app: 0|req: 287/570] 10.222.0.82 () {56 vars in 1108 bytes} [Tue Aug 2 10:02:24 2022] GET /identity/v3/domains/Default => generated 87 bytes in 236 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 02 10:02:24.720414 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51c495cd-0c13-4be3-b5f4-1b9356fb6fec None admin] Authenticating user token {{(pid=80377) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:02:24.754098 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG dbcounter [-] [80378] Writing DB stats keystone:SELECT=100 {{(pid=80378) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 10:02:24.804941 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG dbcounter [-] [80377] Writing DB stats keystone:SELECT=100 {{(pid=80377) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 10:02:24.831239 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7c087a7-9c39-4fae-a1e1-818a63043f68 None admin] Validating token access rules against request {{(pid=80378) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:02:24.859268 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51c495cd-0c13-4be3-b5f4-1b9356fb6fec None admin] Validating token access rules against request {{(pid=80377) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:02:24.896780 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5ae3c07-14b0-492b-9cd1-9706bd3fc131 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '891005fa5ff74041b75747d16cc66798', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80378) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:02:24.898056 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5ae3c07-14b0-492b-9cd1-9706bd3fc131 None admin] REQUEST_METHOD: `GET` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:02:24.898561 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5ae3c07-14b0-492b-9cd1-9706bd3fc131 None admin] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:02:24.899049 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5ae3c07-14b0-492b-9cd1-9706bd3fc131 None admin] PATH_INFO: `/v3/roles/service` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:02:24.900206 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd1e79c8-38d2-4a77-be14-b243a9acc1c0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '891005fa5ff74041b75747d16cc66798', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80377) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:02:24.900833 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd1e79c8-38d2-4a77-be14-b243a9acc1c0 None admin] REQUEST_METHOD: `GET` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:02:24.901121 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd1e79c8-38d2-4a77-be14-b243a9acc1c0 None admin] SCRIPT_NAME: `/identity` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:02:24.901400 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd1e79c8-38d2-4a77-be14-b243a9acc1c0 None admin] PATH_INFO: `/v3/projects` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:02:24.902202 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd1e79c8-38d2-4a77-be14-b243a9acc1c0 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80377) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 02 10:02:24.902510 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd1e79c8-38d2-4a77-be14-b243a9acc1c0 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:02:24.903995 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd1e79c8-38d2-4a77-be14-b243a9acc1c0 None admin] RBAC: Authorization granted {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:02:24.913154 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5ae3c07-14b0-492b-9cd1-9706bd3fc131 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 02 10:02:24.917584 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: [pid: 80377|app: 0|req: 288/571] 10.222.0.82 () {56 vars in 1148 bytes} [Tue Aug 2 10:02:24 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 200 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 02 10:02:24.918859 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c15f2f8f-004d-4356-99ed-c39fa413ba66 None None] REQUEST_METHOD: `GET` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:02:24.919150 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c15f2f8f-004d-4356-99ed-c39fa413ba66 None None] SCRIPT_NAME: `/identity` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:02:24.919434 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c15f2f8f-004d-4356-99ed-c39fa413ba66 None None] PATH_INFO: `/` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:02:24.919772 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application [None req-d5ae3c07-14b0-492b-9cd1-9706bd3fc131 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Aug 02 10:02:24.919772 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 02 10:02:24.919772 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 02 10:02:24.919772 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 02 10:02:24.919772 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 02 10:02:24.919772 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 02 10:02:24.919772 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 02 10:02:24.919772 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 02 10:02:24.919772 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 02 10:02:24.919772 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 02 10:02:24.919772 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 02 10:02:24.919772 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 02 10:02:24.919772 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 02 10:02:24.919772 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 02 10:02:24.919772 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 02 10:02:24.919772 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 02 10:02:24.926415 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 02 10:02:24.926415 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 02 10:02:24.926415 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 02 10:02:24.926415 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 02 10:02:24.926415 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 02 10:02:24.926415 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 02 10:02:24.926415 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 02 10:02:24.926415 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 02 10:02:24.926415 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 02 10:02:24.926415 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application return self.get_or_create( Aug 02 10:02:24.926415 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 02 10:02:24.926415 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application with Lock( Aug 02 10:02:24.926415 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 02 10:02:24.926415 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application return self._enter() Aug 02 10:02:24.926415 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 02 10:02:24.926415 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 02 10:02:24.927268 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 02 10:02:24.927268 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application return self.creator() Aug 02 10:02:24.927268 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 02 10:02:24.927268 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application created_value = creator( Aug 02 10:02:24.927268 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 02 10:02:24.927268 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 02 10:02:24.927268 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 02 10:02:24.927268 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 02 10:02:24.927268 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 02 10:02:24.927268 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 02 10:02:24.927268 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Aug 02 10:02:24.927268 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application Aug 02 10:02:24.927904 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: [pid: 80377|app: 0|req: 289/572] 10.222.0.82 () {54 vars in 926 bytes} [Tue Aug 2 10:02:24 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 02 10:02:24.927904 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c15f2f8f-004d-4356-99ed-c39fa413ba66 None None] Authenticating user token {{(pid=80377) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:02:24.928039 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 284/573] 10.222.0.82 () {56 vars in 1102 bytes} [Tue Aug 2 10:02:24 2022] GET /identity/v3/roles/service => generated 85 bytes in 239 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 02 10:02:24.928039 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5ae3c07-14b0-492b-9cd1-9706bd3fc131 None admin] Authenticating user token {{(pid=80378) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:02:25.050637 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5ae3c07-14b0-492b-9cd1-9706bd3fc131 None admin] Validating token access rules against request {{(pid=80378) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:02:25.060075 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG dbcounter [-] [80378] Writing DB stats keystone:SELECT=100 {{(pid=80378) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 10:02:25.074300 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c15f2f8f-004d-4356-99ed-c39fa413ba66 None None] Validating token access rules against request {{(pid=80377) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:02:25.097339 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e974335d-6ceb-447a-a90d-046c509e3cc7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '891005fa5ff74041b75747d16cc66798', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80378) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:02:25.098148 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e974335d-6ceb-447a-a90d-046c509e3cc7 None admin] REQUEST_METHOD: `GET` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:02:25.098471 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e974335d-6ceb-447a-a90d-046c509e3cc7 None admin] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:02:25.098784 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e974335d-6ceb-447a-a90d-046c509e3cc7 None admin] PATH_INFO: `/v3/domains` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:02:25.099526 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e974335d-6ceb-447a-a90d-046c509e3cc7 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80378) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 02 10:02:25.099950 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e974335d-6ceb-447a-a90d-046c509e3cc7 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:02:25.101429 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e974335d-6ceb-447a-a90d-046c509e3cc7 None admin] RBAC: Authorization granted {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:02:25.102714 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG dbcounter [-] [80377] Writing DB stats keystone:SELECT=100 {{(pid=80377) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 10:02:25.115394 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 285/574] 10.222.0.82 () {56 vars in 1109 bytes} [Tue Aug 2 10:02:24 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 193 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 02 10:02:25.116234 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e974335d-6ceb-447a-a90d-046c509e3cc7 None admin] Authenticating user token {{(pid=80378) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:02:25.124443 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3e583a0-031a-4c45-a684-e18e8e559f97 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '891005fa5ff74041b75747d16cc66798', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80377) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:02:25.125478 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3e583a0-031a-4c45-a684-e18e8e559f97 None admin] REQUEST_METHOD: `GET` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:02:25.126006 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3e583a0-031a-4c45-a684-e18e8e559f97 None admin] SCRIPT_NAME: `/identity` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:02:25.127885 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3e583a0-031a-4c45-a684-e18e8e559f97 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:02:25.139521 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3e583a0-031a-4c45-a684-e18e8e559f97 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:02:25.141989 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3e583a0-031a-4c45-a684-e18e8e559f97 None admin] RBAC: Authorization granted {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:02:25.163714 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application [None req-e3e583a0-031a-4c45-a684-e18e8e559f97 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 02 10:02:25.163714 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 02 10:02:25.163714 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 02 10:02:25.163714 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 02 10:02:25.163714 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 02 10:02:25.163714 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 02 10:02:25.163714 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 02 10:02:25.163714 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 02 10:02:25.163714 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 02 10:02:25.163714 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 02 10:02:25.163714 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 02 10:02:25.163714 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 02 10:02:25.163714 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 02 10:02:25.163714 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 02 10:02:25.163714 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 02 10:02:25.165266 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 02 10:02:25.165266 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 02 10:02:25.165266 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 02 10:02:25.165266 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 02 10:02:25.165266 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 02 10:02:25.165266 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 02 10:02:25.165266 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application return self.get_or_create( Aug 02 10:02:25.165266 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 02 10:02:25.165266 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application with Lock( Aug 02 10:02:25.165266 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 02 10:02:25.165266 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application return self._enter() Aug 02 10:02:25.165266 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 02 10:02:25.165266 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 02 10:02:25.165266 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 02 10:02:25.165266 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application return self.creator() Aug 02 10:02:25.165266 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 02 10:02:25.165266 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application created_value = creator( Aug 02 10:02:25.166467 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 02 10:02:25.166467 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 02 10:02:25.166467 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 02 10:02:25.166467 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 02 10:02:25.166467 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 02 10:02:25.166467 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 02 10:02:25.166467 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 02 10:02:25.166467 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application Aug 02 10:02:25.166467 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: [pid: 80377|app: 0|req: 290/575] 10.222.0.82 () {56 vars in 1111 bytes} [Tue Aug 2 10:02:24 2022] GET /identity/v3/projects/service => generated 88 bytes in 244 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 02 10:02:25.167390 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93059851-a558-4d56-a2c2-a73512062142 None None] REQUEST_METHOD: `POST` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:02:25.167751 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93059851-a558-4d56-a2c2-a73512062142 None None] SCRIPT_NAME: `/identity` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:02:25.168218 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93059851-a558-4d56-a2c2-a73512062142 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:02:25.190795 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.auth.core [None req-93059851-a558-4d56-a2c2-a73512062142 None None] MFA Rules not processed for user `891005fa5ff74041b75747d16cc66798`. Rule list: `[]` (Enabled: `True`). {{(pid=80377) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 02 10:02:25.192011 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.fernet_utils [None req-93059851-a558-4d56-a2c2-a73512062142 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80377) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 10:02:25.275393 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e974335d-6ceb-447a-a90d-046c509e3cc7 None admin] Validating token access rules against request {{(pid=80378) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:02:25.275603 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: [pid: 80377|app: 0|req: 291/576] 10.222.0.82 () {58 vars in 1023 bytes} [Tue Aug 2 10:02:25 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 110 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 02 10:02:25.276361 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93059851-a558-4d56-a2c2-a73512062142 None None] Authenticating user token {{(pid=80377) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:02:25.331320 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8178b99e-287b-4691-b67b-8ae536143dc1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '891005fa5ff74041b75747d16cc66798', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80378) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:02:25.331862 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8178b99e-287b-4691-b67b-8ae536143dc1 None admin] REQUEST_METHOD: `GET` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:02:25.332141 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8178b99e-287b-4691-b67b-8ae536143dc1 None admin] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:02:25.332412 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8178b99e-287b-4691-b67b-8ae536143dc1 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:02:25.333205 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8178b99e-287b-4691-b67b-8ae536143dc1 None admin] RBAC: Adding query filter params (role.id=02003a0be60f47f2a8dfea17053af52c, scope.project.id=f1b38ebb804c41c69772c998aae72c41, user.id=bef1b2e714cb45c4a51eb6d3e15aad6d) {{(pid=80378) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 02 10:02:25.333517 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8178b99e-287b-4691-b67b-8ae536143dc1 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:02:25.335109 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8178b99e-287b-4691-b67b-8ae536143dc1 None admin] RBAC: Authorization granted {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:02:25.340770 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 286/577] 10.222.0.82 () {56 vars in 1374 bytes} [Tue Aug 2 10:02:25 2022] GET /identity/v3/role_assignments?user.id=bef1b2e714cb45c4a51eb6d3e15aad6d&scope.project.id=f1b38ebb804c41c69772c998aae72c41&role.id=02003a0be60f47f2a8dfea17053af52c => generated 259 bytes in 225 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 02 10:02:25.341555 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8178b99e-287b-4691-b67b-8ae536143dc1 None admin] Authenticating user token {{(pid=80378) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:02:25.397092 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG dbcounter [-] [80378] Writing DB stats keystone:SELECT=100 {{(pid=80378) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 10:02:25.435617 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93059851-a558-4d56-a2c2-a73512062142 None None] Validating token access rules against request {{(pid=80377) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:02:25.455213 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG dbcounter [-] [80377] Writing DB stats keystone:SELECT=100 {{(pid=80377) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 10:02:25.493030 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2935bb0-5c82-49cb-9d36-c815777a3ea4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '891005fa5ff74041b75747d16cc66798', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80377) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:02:25.494334 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2935bb0-5c82-49cb-9d36-c815777a3ea4 None admin] REQUEST_METHOD: `GET` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:02:25.494855 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2935bb0-5c82-49cb-9d36-c815777a3ea4 None admin] SCRIPT_NAME: `/identity` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:02:25.495387 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2935bb0-5c82-49cb-9d36-c815777a3ea4 None admin] PATH_INFO: `/v3/roles` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:02:25.496507 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2935bb0-5c82-49cb-9d36-c815777a3ea4 None admin] RBAC: Adding query filter params (name=service) {{(pid=80377) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 02 10:02:25.496918 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2935bb0-5c82-49cb-9d36-c815777a3ea4 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:02:25.498856 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2935bb0-5c82-49cb-9d36-c815777a3ea4 None admin] RBAC: Authorization granted {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:02:25.503329 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8178b99e-287b-4691-b67b-8ae536143dc1 None admin] Validating token access rules against request {{(pid=80378) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:02:25.522878 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: [pid: 80377|app: 0|req: 292/578] 10.222.0.82 () {56 vars in 1103 bytes} [Tue Aug 2 10:02:25 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 246 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 02 10:02:25.523336 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2935bb0-5c82-49cb-9d36-c815777a3ea4 None admin] Authenticating user token {{(pid=80377) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:02:25.548107 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99652f83-97e8-421e-927d-80afe35d5e62 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '891005fa5ff74041b75747d16cc66798', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80378) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:02:25.549158 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99652f83-97e8-421e-927d-80afe35d5e62 None admin] REQUEST_METHOD: `GET` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:02:25.549601 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99652f83-97e8-421e-927d-80afe35d5e62 None admin] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:02:25.549992 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99652f83-97e8-421e-927d-80afe35d5e62 None admin] PATH_INFO: `/v3/users/cinder` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:02:25.560627 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99652f83-97e8-421e-927d-80afe35d5e62 None admin] RBAC: Authorizing `identity:get_user(user_id=cinder)` {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:02:25.562677 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99652f83-97e8-421e-927d-80afe35d5e62 None admin] RBAC: Authorization granted {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:02:25.580930 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application [None req-99652f83-97e8-421e-927d-80afe35d5e62 None admin] Could not find user: cinder.: keystone.exception.UserNotFound: Could not find user: cinder. Aug 02 10:02:25.580930 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 02 10:02:25.580930 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 02 10:02:25.580930 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 02 10:02:25.580930 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 02 10:02:25.580930 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 02 10:02:25.580930 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 02 10:02:25.580930 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 02 10:02:25.580930 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 02 10:02:25.580930 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 02 10:02:25.580930 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 02 10:02:25.580930 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 02 10:02:25.580930 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 02 10:02:25.580930 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 02 10:02:25.580930 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 02 10:02:25.580930 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 02 10:02:25.582055 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 02 10:02:25.582055 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 02 10:02:25.582055 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 02 10:02:25.582055 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 02 10:02:25.582055 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 02 10:02:25.582055 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 02 10:02:25.582055 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 02 10:02:25.582055 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 02 10:02:25.582055 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 02 10:02:25.582055 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application return self.get_or_create( Aug 02 10:02:25.582055 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 02 10:02:25.582055 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application with Lock( Aug 02 10:02:25.582055 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 02 10:02:25.582055 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application return self._enter() Aug 02 10:02:25.582055 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 02 10:02:25.582055 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 02 10:02:25.582055 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 02 10:02:25.582055 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application return self.creator() Aug 02 10:02:25.583037 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 02 10:02:25.583037 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application created_value = creator( Aug 02 10:02:25.583037 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 02 10:02:25.583037 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 02 10:02:25.583037 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 02 10:02:25.583037 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 02 10:02:25.583037 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 02 10:02:25.583037 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 02 10:02:25.583037 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: cinder. Aug 02 10:02:25.583037 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application Aug 02 10:02:25.583037 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 287/579] 10.222.0.82 () {56 vars in 1099 bytes} [Tue Aug 2 10:02:25 2022] GET /identity/v3/users/cinder => generated 84 bytes in 241 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 02 10:02:25.583634 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99652f83-97e8-421e-927d-80afe35d5e62 None admin] Authenticating user token {{(pid=80378) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:02:25.669011 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2935bb0-5c82-49cb-9d36-c815777a3ea4 None admin] Validating token access rules against request {{(pid=80377) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:02:25.700397 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG dbcounter [-] [80378] Writing DB stats keystone:SELECT=100 {{(pid=80378) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 10:02:25.715445 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99652f83-97e8-421e-927d-80afe35d5e62 None admin] Validating token access rules against request {{(pid=80378) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:02:25.721742 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62d7ded0-88f1-4c4d-ba65-ddf8a38c2e2b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '891005fa5ff74041b75747d16cc66798', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80377) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:02:25.722727 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62d7ded0-88f1-4c4d-ba65-ddf8a38c2e2b None admin] REQUEST_METHOD: `GET` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:02:25.723080 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62d7ded0-88f1-4c4d-ba65-ddf8a38c2e2b None admin] SCRIPT_NAME: `/identity` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:02:25.723365 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62d7ded0-88f1-4c4d-ba65-ddf8a38c2e2b None admin] PATH_INFO: `/v3/projects` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:02:25.724140 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62d7ded0-88f1-4c4d-ba65-ddf8a38c2e2b None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80377) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 02 10:02:25.724457 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62d7ded0-88f1-4c4d-ba65-ddf8a38c2e2b None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:02:25.726056 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62d7ded0-88f1-4c4d-ba65-ddf8a38c2e2b None admin] RBAC: Authorization granted {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:02:25.748011 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: [pid: 80377|app: 0|req: 293/580] 10.222.0.82 () {56 vars in 1148 bytes} [Tue Aug 2 10:02:25 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 224 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 02 10:02:25.748909 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62d7ded0-88f1-4c4d-ba65-ddf8a38c2e2b None admin] Authenticating user token {{(pid=80377) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:02:25.762874 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b817f7aa-f7b4-40b5-92ac-1788b84e20a8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '891005fa5ff74041b75747d16cc66798', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80378) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:02:25.763685 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b817f7aa-f7b4-40b5-92ac-1788b84e20a8 None admin] REQUEST_METHOD: `GET` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:02:25.763975 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b817f7aa-f7b4-40b5-92ac-1788b84e20a8 None admin] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:02:25.764248 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b817f7aa-f7b4-40b5-92ac-1788b84e20a8 None admin] PATH_INFO: `/v3/roles/service` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:02:25.776582 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b817f7aa-f7b4-40b5-92ac-1788b84e20a8 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 02 10:02:25.782818 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application [None req-b817f7aa-f7b4-40b5-92ac-1788b84e20a8 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Aug 02 10:02:25.782818 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 02 10:02:25.782818 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 02 10:02:25.782818 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 02 10:02:25.782818 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 02 10:02:25.782818 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 02 10:02:25.782818 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 02 10:02:25.782818 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 02 10:02:25.782818 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 02 10:02:25.782818 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 02 10:02:25.782818 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 02 10:02:25.782818 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 02 10:02:25.782818 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 02 10:02:25.782818 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 02 10:02:25.782818 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 02 10:02:25.782818 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 02 10:02:25.787080 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 02 10:02:25.787080 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 02 10:02:25.787080 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 02 10:02:25.787080 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 02 10:02:25.787080 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 02 10:02:25.787080 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 02 10:02:25.787080 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 02 10:02:25.787080 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 02 10:02:25.787080 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 02 10:02:25.787080 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application return self.get_or_create( Aug 02 10:02:25.787080 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 02 10:02:25.787080 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application with Lock( Aug 02 10:02:25.787080 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 02 10:02:25.787080 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application return self._enter() Aug 02 10:02:25.787080 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 02 10:02:25.787080 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 02 10:02:25.789621 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 02 10:02:25.789621 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application return self.creator() Aug 02 10:02:25.789621 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 02 10:02:25.789621 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application created_value = creator( Aug 02 10:02:25.789621 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 02 10:02:25.789621 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 02 10:02:25.789621 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 02 10:02:25.789621 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 02 10:02:25.789621 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 02 10:02:25.789621 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 02 10:02:25.789621 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Aug 02 10:02:25.789621 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application Aug 02 10:02:25.789621 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 288/581] 10.222.0.82 () {56 vars in 1102 bytes} [Tue Aug 2 10:02:25 2022] GET /identity/v3/roles/service => generated 85 bytes in 201 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 02 10:02:25.789621 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b817f7aa-f7b4-40b5-92ac-1788b84e20a8 None admin] Authenticating user token {{(pid=80378) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:02:25.802788 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG dbcounter [-] [80377] Writing DB stats keystone:SELECT=100 {{(pid=80377) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 10:02:25.890148 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62d7ded0-88f1-4c4d-ba65-ddf8a38c2e2b None admin] Validating token access rules against request {{(pid=80377) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:02:25.934974 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46354d50-bfc6-4014-a3e8-218b8262b000 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '891005fa5ff74041b75747d16cc66798', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80377) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:02:25.935633 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46354d50-bfc6-4014-a3e8-218b8262b000 None admin] REQUEST_METHOD: `GET` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:02:25.935908 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46354d50-bfc6-4014-a3e8-218b8262b000 None admin] SCRIPT_NAME: `/identity` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:02:25.936182 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46354d50-bfc6-4014-a3e8-218b8262b000 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:02:25.942089 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b817f7aa-f7b4-40b5-92ac-1788b84e20a8 None admin] Validating token access rules against request {{(pid=80378) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:02:25.948156 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46354d50-bfc6-4014-a3e8-218b8262b000 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:02:25.950131 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46354d50-bfc6-4014-a3e8-218b8262b000 None admin] RBAC: Authorization granted {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:02:25.976999 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application [None req-46354d50-bfc6-4014-a3e8-218b8262b000 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 02 10:02:25.976999 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 02 10:02:25.976999 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 02 10:02:25.976999 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 02 10:02:25.976999 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 02 10:02:25.976999 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 02 10:02:25.976999 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 02 10:02:25.976999 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 02 10:02:25.976999 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 02 10:02:25.976999 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 02 10:02:25.976999 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 02 10:02:25.976999 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 02 10:02:25.976999 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 02 10:02:25.976999 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 02 10:02:25.976999 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 02 10:02:25.978575 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 02 10:02:25.978575 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 02 10:02:25.978575 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 02 10:02:25.978575 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 02 10:02:25.978575 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 02 10:02:25.978575 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 02 10:02:25.978575 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application return self.get_or_create( Aug 02 10:02:25.978575 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 02 10:02:25.978575 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application with Lock( Aug 02 10:02:25.978575 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 02 10:02:25.978575 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application return self._enter() Aug 02 10:02:25.978575 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 02 10:02:25.978575 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 02 10:02:25.978575 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 02 10:02:25.978575 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application return self.creator() Aug 02 10:02:25.978575 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 02 10:02:25.978575 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application created_value = creator( Aug 02 10:02:25.979891 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 02 10:02:25.979891 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 02 10:02:25.979891 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 02 10:02:25.979891 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application Aug 02 10:02:25.979891 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: [pid: 80377|app: 0|req: 294/582] 10.222.0.82 () {56 vars in 1108 bytes} [Tue Aug 2 10:02:25 2022] GET /identity/v3/domains/Default => generated 87 bytes in 229 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 02 10:02:25.979891 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46354d50-bfc6-4014-a3e8-218b8262b000 None admin] Authenticating user token {{(pid=80377) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:02:25.994030 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-164fcdb0-9c3f-4664-ab59-76f44a400ac7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '891005fa5ff74041b75747d16cc66798', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80378) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:02:25.995132 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-164fcdb0-9c3f-4664-ab59-76f44a400ac7 None admin] REQUEST_METHOD: `GET` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:02:25.995613 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-164fcdb0-9c3f-4664-ab59-76f44a400ac7 None admin] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:02:25.996092 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-164fcdb0-9c3f-4664-ab59-76f44a400ac7 None admin] PATH_INFO: `/v3/users` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:02:25.997610 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-164fcdb0-9c3f-4664-ab59-76f44a400ac7 None admin] RBAC: Adding query filter params (domain_id=default, name=cinder) {{(pid=80378) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 02 10:02:25.998149 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-164fcdb0-9c3f-4664-ab59-76f44a400ac7 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:02:26.000586 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-164fcdb0-9c3f-4664-ab59-76f44a400ac7 None admin] RBAC: Authorization granted {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:02:26.039758 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 289/583] 10.222.0.82 () {56 vars in 1137 bytes} [Tue Aug 2 10:02:25 2022] GET /identity/v3/users?domain_id=default&name=cinder => generated 372 bytes in 250 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 02 10:02:26.039758 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-164fcdb0-9c3f-4664-ab59-76f44a400ac7 None admin] Authenticating user token {{(pid=80378) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:02:26.062890 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG dbcounter [-] [80378] Writing DB stats keystone:SELECT=100 {{(pid=80378) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 10:02:26.114948 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG dbcounter [-] [80377] Writing DB stats keystone:SELECT=100 {{(pid=80377) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 10:02:26.136500 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46354d50-bfc6-4014-a3e8-218b8262b000 None admin] Validating token access rules against request {{(pid=80377) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:02:26.183908 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbe826ac-8087-4253-826a-178dc87feb1f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '891005fa5ff74041b75747d16cc66798', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80377) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:02:26.184615 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbe826ac-8087-4253-826a-178dc87feb1f None admin] REQUEST_METHOD: `PUT` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:02:26.184895 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbe826ac-8087-4253-826a-178dc87feb1f None admin] SCRIPT_NAME: `/identity` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:02:26.185178 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbe826ac-8087-4253-826a-178dc87feb1f None admin] PATH_INFO: `/v3/projects/f1b38ebb804c41c69772c998aae72c41/users/88af518196674fe593ad88d30111fb7d/roles/02003a0be60f47f2a8dfea17053af52c` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:02:26.194531 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-164fcdb0-9c3f-4664-ab59-76f44a400ac7 None admin] Validating token access rules against request {{(pid=80378) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:02:26.209924 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dbe826ac-8087-4253-826a-178dc87feb1f None admin] RBAC: Authorizing `identity:create_grant(project_id=f1b38ebb804c41c69772c998aae72c41, user_id=88af518196674fe593ad88d30111fb7d, role_id=02003a0be60f47f2a8dfea17053af52c)` {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:02:26.212152 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dbe826ac-8087-4253-826a-178dc87feb1f None admin] RBAC: Authorization granted {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:02:26.235063 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: [pid: 80377|app: 0|req: 295/584] 10.222.0.82 () {58 vars in 1439 bytes} [Tue Aug 2 10:02:25 2022] PUT /identity/v3/projects/f1b38ebb804c41c69772c998aae72c41/users/88af518196674fe593ad88d30111fb7d/roles/02003a0be60f47f2a8dfea17053af52c => generated 0 bytes in 257 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 02 10:02:26.236478 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbe826ac-8087-4253-826a-178dc87feb1f None admin] Authenticating user token {{(pid=80377) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:02:26.256134 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc270a0b-8e71-4f95-a208-6bdb489d2a9d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '891005fa5ff74041b75747d16cc66798', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80378) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:02:26.257436 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc270a0b-8e71-4f95-a208-6bdb489d2a9d None admin] REQUEST_METHOD: `GET` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:02:26.257982 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc270a0b-8e71-4f95-a208-6bdb489d2a9d None admin] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:02:26.258531 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc270a0b-8e71-4f95-a208-6bdb489d2a9d None admin] PATH_INFO: `/v3/roles` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:02:26.259734 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc270a0b-8e71-4f95-a208-6bdb489d2a9d None admin] RBAC: Adding query filter params (name=service) {{(pid=80378) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 02 10:02:26.260645 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc270a0b-8e71-4f95-a208-6bdb489d2a9d None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:02:26.262497 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc270a0b-8e71-4f95-a208-6bdb489d2a9d None admin] RBAC: Authorization granted {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:02:26.276143 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 290/585] 10.222.0.82 () {56 vars in 1103 bytes} [Tue Aug 2 10:02:26 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 240 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 02 10:02:26.277453 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc270a0b-8e71-4f95-a208-6bdb489d2a9d None admin] Authenticating user token {{(pid=80378) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:02:26.421238 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG dbcounter [-] [80378] Writing DB stats keystone:SELECT=100 {{(pid=80378) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 10:02:26.433404 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbe826ac-8087-4253-826a-178dc87feb1f None admin] Validating token access rules against request {{(pid=80377) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:02:26.474211 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc270a0b-8e71-4f95-a208-6bdb489d2a9d None admin] Validating token access rules against request {{(pid=80378) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:02:26.490239 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2855a97c-c872-4756-ad6e-19e922513157 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '891005fa5ff74041b75747d16cc66798', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80377) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:02:26.491670 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2855a97c-c872-4756-ad6e-19e922513157 None admin] REQUEST_METHOD: `GET` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:02:26.492020 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2855a97c-c872-4756-ad6e-19e922513157 None admin] SCRIPT_NAME: `/identity` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:02:26.492285 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2855a97c-c872-4756-ad6e-19e922513157 None admin] PATH_INFO: `/v3/domains` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:02:26.493016 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2855a97c-c872-4756-ad6e-19e922513157 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80377) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 02 10:02:26.493338 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2855a97c-c872-4756-ad6e-19e922513157 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:02:26.498250 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2855a97c-c872-4756-ad6e-19e922513157 None admin] RBAC: Authorization granted {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:02:26.498250 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG dbcounter [-] [80377] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80377) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 10:02:26.510193 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: [pid: 80377|app: 0|req: 296/586] 10.222.0.82 () {56 vars in 1109 bytes} [Tue Aug 2 10:02:26 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 274 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 02 10:02:26.511077 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2855a97c-c872-4756-ad6e-19e922513157 None admin] Authenticating user token {{(pid=80377) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:02:26.550819 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c223a204-3e2e-4375-b40d-4160ca28308e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '891005fa5ff74041b75747d16cc66798', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80378) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:02:26.552607 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c223a204-3e2e-4375-b40d-4160ca28308e None admin] REQUEST_METHOD: `GET` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:02:26.553139 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c223a204-3e2e-4375-b40d-4160ca28308e None admin] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:02:26.554201 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c223a204-3e2e-4375-b40d-4160ca28308e None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:02:26.580947 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c223a204-3e2e-4375-b40d-4160ca28308e None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:02:26.584659 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c223a204-3e2e-4375-b40d-4160ca28308e None admin] RBAC: Authorization granted {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:02:26.621766 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application [None req-c223a204-3e2e-4375-b40d-4160ca28308e None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 02 10:02:26.621766 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 02 10:02:26.621766 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 02 10:02:26.621766 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 02 10:02:26.621766 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 02 10:02:26.621766 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 02 10:02:26.621766 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 02 10:02:26.621766 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 02 10:02:26.621766 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 02 10:02:26.621766 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 02 10:02:26.621766 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 02 10:02:26.621766 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 02 10:02:26.621766 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 02 10:02:26.621766 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 02 10:02:26.621766 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 02 10:02:26.622779 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 02 10:02:26.622779 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 02 10:02:26.622779 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 02 10:02:26.622779 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 02 10:02:26.622779 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 02 10:02:26.622779 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 02 10:02:26.622779 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application return self.get_or_create( Aug 02 10:02:26.622779 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 02 10:02:26.622779 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application with Lock( Aug 02 10:02:26.622779 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 02 10:02:26.622779 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application return self._enter() Aug 02 10:02:26.622779 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 02 10:02:26.622779 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 02 10:02:26.622779 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 02 10:02:26.622779 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application return self.creator() Aug 02 10:02:26.622779 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 02 10:02:26.622779 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application created_value = creator( Aug 02 10:02:26.623677 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 02 10:02:26.623677 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 02 10:02:26.623677 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 02 10:02:26.623677 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application Aug 02 10:02:26.624742 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 291/587] 10.222.0.82 () {56 vars in 1108 bytes} [Tue Aug 2 10:02:26 2022] GET /identity/v3/domains/Default => generated 87 bytes in 347 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 02 10:02:26.625846 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c223a204-3e2e-4375-b40d-4160ca28308e None admin] Authenticating user token {{(pid=80378) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:02:26.643758 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2855a97c-c872-4756-ad6e-19e922513157 None admin] Validating token access rules against request {{(pid=80377) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:02:26.698567 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b51a66a8-ae25-4533-9d94-e7a910d14187 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '891005fa5ff74041b75747d16cc66798', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80377) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:02:26.699769 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b51a66a8-ae25-4533-9d94-e7a910d14187 None admin] REQUEST_METHOD: `GET` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:02:26.700372 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b51a66a8-ae25-4533-9d94-e7a910d14187 None admin] SCRIPT_NAME: `/identity` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:02:26.700931 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b51a66a8-ae25-4533-9d94-e7a910d14187 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:02:26.714794 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b51a66a8-ae25-4533-9d94-e7a910d14187 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:02:26.716264 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b51a66a8-ae25-4533-9d94-e7a910d14187 None admin] RBAC: Authorization granted {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:02:26.732514 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application [None req-b51a66a8-ae25-4533-9d94-e7a910d14187 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 02 10:02:26.732514 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 02 10:02:26.732514 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 02 10:02:26.732514 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 02 10:02:26.732514 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 02 10:02:26.732514 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 02 10:02:26.732514 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 02 10:02:26.732514 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 02 10:02:26.732514 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 02 10:02:26.732514 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 02 10:02:26.732514 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 02 10:02:26.732514 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 02 10:02:26.732514 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 02 10:02:26.732514 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 02 10:02:26.732514 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 02 10:02:26.734052 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 02 10:02:26.734052 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 02 10:02:26.734052 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 02 10:02:26.734052 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 02 10:02:26.734052 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 02 10:02:26.734052 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 02 10:02:26.734052 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application return self.get_or_create( Aug 02 10:02:26.734052 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 02 10:02:26.734052 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application with Lock( Aug 02 10:02:26.734052 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 02 10:02:26.734052 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application return self._enter() Aug 02 10:02:26.734052 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 02 10:02:26.734052 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 02 10:02:26.734052 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 02 10:02:26.734052 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application return self.creator() Aug 02 10:02:26.734052 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 02 10:02:26.734052 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application created_value = creator( Aug 02 10:02:26.735688 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 02 10:02:26.735688 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 02 10:02:26.735688 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 02 10:02:26.735688 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application Aug 02 10:02:26.735688 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: [pid: 80377|app: 0|req: 297/588] 10.222.0.82 () {56 vars in 1108 bytes} [Tue Aug 2 10:02:26 2022] GET /identity/v3/domains/Default => generated 87 bytes in 223 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 02 10:02:26.735688 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b51a66a8-ae25-4533-9d94-e7a910d14187 None admin] Authenticating user token {{(pid=80377) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:02:26.781035 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c223a204-3e2e-4375-b40d-4160ca28308e None admin] Validating token access rules against request {{(pid=80378) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:02:26.808670 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG dbcounter [-] [80377] Writing DB stats keystone:SELECT=100 {{(pid=80377) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 10:02:26.813545 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG dbcounter [-] [80378] Writing DB stats keystone:SELECT=100 {{(pid=80378) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 10:02:26.827773 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4260c81-bba2-4c10-9139-8f4bc363bb40 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '891005fa5ff74041b75747d16cc66798', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80378) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:02:26.828496 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4260c81-bba2-4c10-9139-8f4bc363bb40 None admin] REQUEST_METHOD: `GET` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:02:26.828781 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4260c81-bba2-4c10-9139-8f4bc363bb40 None admin] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:02:26.829120 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4260c81-bba2-4c10-9139-8f4bc363bb40 None admin] PATH_INFO: `/v3/users/glance` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:02:26.839644 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4260c81-bba2-4c10-9139-8f4bc363bb40 None admin] RBAC: Authorizing `identity:get_user(user_id=glance)` {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:02:26.841099 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4260c81-bba2-4c10-9139-8f4bc363bb40 None admin] RBAC: Authorization granted {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:02:26.859563 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application [None req-d4260c81-bba2-4c10-9139-8f4bc363bb40 None admin] Could not find user: glance.: keystone.exception.UserNotFound: Could not find user: glance. Aug 02 10:02:26.859563 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 02 10:02:26.859563 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 02 10:02:26.859563 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 02 10:02:26.859563 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 02 10:02:26.859563 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 02 10:02:26.859563 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 02 10:02:26.859563 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 02 10:02:26.859563 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 02 10:02:26.859563 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 02 10:02:26.859563 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 02 10:02:26.859563 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 02 10:02:26.859563 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 02 10:02:26.859563 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 02 10:02:26.859563 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 02 10:02:26.859563 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 02 10:02:26.861823 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 02 10:02:26.861823 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 02 10:02:26.861823 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 02 10:02:26.861823 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 02 10:02:26.861823 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 02 10:02:26.861823 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 02 10:02:26.861823 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 02 10:02:26.861823 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 02 10:02:26.861823 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 02 10:02:26.861823 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application return self.get_or_create( Aug 02 10:02:26.861823 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 02 10:02:26.861823 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application with Lock( Aug 02 10:02:26.861823 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 02 10:02:26.861823 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application return self._enter() Aug 02 10:02:26.861823 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 02 10:02:26.861823 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 02 10:02:26.861823 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 02 10:02:26.861823 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application return self.creator() Aug 02 10:02:26.863345 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 02 10:02:26.863345 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application created_value = creator( Aug 02 10:02:26.863345 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 02 10:02:26.863345 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 02 10:02:26.863345 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 02 10:02:26.863345 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 02 10:02:26.863345 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 02 10:02:26.863345 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 02 10:02:26.863345 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance. Aug 02 10:02:26.863345 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application Aug 02 10:02:26.867738 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 292/589] 10.222.0.82 () {56 vars in 1099 bytes} [Tue Aug 2 10:02:26 2022] GET /identity/v3/users/glance => generated 84 bytes in 238 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 02 10:02:26.868680 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4260c81-bba2-4c10-9139-8f4bc363bb40 None admin] Authenticating user token {{(pid=80378) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:02:26.923740 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b51a66a8-ae25-4533-9d94-e7a910d14187 None admin] Validating token access rules against request {{(pid=80377) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:02:26.980969 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1f548e5-4043-4f64-a883-f952adc8e469 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '891005fa5ff74041b75747d16cc66798', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80377) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:02:26.981608 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1f548e5-4043-4f64-a883-f952adc8e469 None admin] REQUEST_METHOD: `GET` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:02:26.981917 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1f548e5-4043-4f64-a883-f952adc8e469 None admin] SCRIPT_NAME: `/identity` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:02:26.982201 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1f548e5-4043-4f64-a883-f952adc8e469 None admin] PATH_INFO: `/v3/domains` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:02:26.982900 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1f548e5-4043-4f64-a883-f952adc8e469 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80377) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 02 10:02:26.983211 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1f548e5-4043-4f64-a883-f952adc8e469 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:02:26.984600 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1f548e5-4043-4f64-a883-f952adc8e469 None admin] RBAC: Authorization granted {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:02:27.024803 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: [pid: 80377|app: 0|req: 298/590] 10.222.0.82 () {56 vars in 1109 bytes} [Tue Aug 2 10:02:26 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 285 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 02 10:02:27.026335 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1f548e5-4043-4f64-a883-f952adc8e469 None admin] Authenticating user token {{(pid=80377) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:02:27.039613 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4260c81-bba2-4c10-9139-8f4bc363bb40 None admin] Validating token access rules against request {{(pid=80378) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:02:27.100011 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf14f211-ba8f-4332-b8f5-4c93eba80b87 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '891005fa5ff74041b75747d16cc66798', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80378) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:02:27.101285 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf14f211-ba8f-4332-b8f5-4c93eba80b87 None admin] REQUEST_METHOD: `GET` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:02:27.102024 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf14f211-ba8f-4332-b8f5-4c93eba80b87 None admin] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:02:27.102600 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf14f211-ba8f-4332-b8f5-4c93eba80b87 None admin] PATH_INFO: `/v3/domains` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:02:27.103516 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf14f211-ba8f-4332-b8f5-4c93eba80b87 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80378) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 02 10:02:27.104044 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf14f211-ba8f-4332-b8f5-4c93eba80b87 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:02:27.106481 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf14f211-ba8f-4332-b8f5-4c93eba80b87 None admin] RBAC: Authorization granted {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:02:27.128818 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 293/591] 10.222.0.82 () {56 vars in 1109 bytes} [Tue Aug 2 10:02:26 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 261 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 02 10:02:27.130979 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf14f211-ba8f-4332-b8f5-4c93eba80b87 None admin] Authenticating user token {{(pid=80378) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:02:27.185110 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1f548e5-4043-4f64-a883-f952adc8e469 None admin] Validating token access rules against request {{(pid=80377) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:02:27.197843 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG dbcounter [-] [80377] Writing DB stats keystone:SELECT=100 {{(pid=80377) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 10:02:27.208710 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG dbcounter [-] [80378] Writing DB stats keystone:SELECT=100 {{(pid=80378) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 10:02:27.247524 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b9e87ca-7bad-4630-8a96-db2ae3a8b58b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '891005fa5ff74041b75747d16cc66798', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80377) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:02:27.248327 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b9e87ca-7bad-4630-8a96-db2ae3a8b58b None admin] REQUEST_METHOD: `GET` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:02:27.248613 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b9e87ca-7bad-4630-8a96-db2ae3a8b58b None admin] SCRIPT_NAME: `/identity` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:02:27.248897 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b9e87ca-7bad-4630-8a96-db2ae3a8b58b None admin] PATH_INFO: `/v3/users` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:02:27.249708 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b9e87ca-7bad-4630-8a96-db2ae3a8b58b None admin] RBAC: Adding query filter params (domain_id=default, name=glance) {{(pid=80377) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 02 10:02:27.250054 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b9e87ca-7bad-4630-8a96-db2ae3a8b58b None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:02:27.253023 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b9e87ca-7bad-4630-8a96-db2ae3a8b58b None admin] RBAC: Authorization granted {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:02:27.285315 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: [pid: 80377|app: 0|req: 299/592] 10.222.0.82 () {56 vars in 1137 bytes} [Tue Aug 2 10:02:27 2022] GET /identity/v3/users?domain_id=default&name=glance => generated 372 bytes in 260 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 02 10:02:27.286939 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b9e87ca-7bad-4630-8a96-db2ae3a8b58b None admin] Authenticating user token {{(pid=80377) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:02:27.302304 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf14f211-ba8f-4332-b8f5-4c93eba80b87 None admin] Validating token access rules against request {{(pid=80378) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:02:27.366098 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-100e0daa-0510-46a7-b292-bf32afa8d2c9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '891005fa5ff74041b75747d16cc66798', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80378) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:02:27.367011 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-100e0daa-0510-46a7-b292-bf32afa8d2c9 None admin] REQUEST_METHOD: `GET` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:02:27.367300 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-100e0daa-0510-46a7-b292-bf32afa8d2c9 None admin] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:02:27.367579 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-100e0daa-0510-46a7-b292-bf32afa8d2c9 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:02:27.376075 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-100e0daa-0510-46a7-b292-bf32afa8d2c9 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:02:27.378506 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-100e0daa-0510-46a7-b292-bf32afa8d2c9 None admin] RBAC: Authorization granted {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:02:27.392495 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application [None req-100e0daa-0510-46a7-b292-bf32afa8d2c9 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 02 10:02:27.392495 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 02 10:02:27.392495 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 02 10:02:27.392495 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 02 10:02:27.392495 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 02 10:02:27.392495 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 02 10:02:27.392495 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 02 10:02:27.392495 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 02 10:02:27.392495 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 02 10:02:27.392495 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 02 10:02:27.392495 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 02 10:02:27.392495 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 02 10:02:27.392495 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 02 10:02:27.392495 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 02 10:02:27.392495 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 02 10:02:27.394292 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 02 10:02:27.394292 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 02 10:02:27.394292 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 02 10:02:27.394292 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 02 10:02:27.394292 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 02 10:02:27.394292 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 02 10:02:27.394292 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application return self.get_or_create( Aug 02 10:02:27.394292 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 02 10:02:27.394292 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application with Lock( Aug 02 10:02:27.394292 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 02 10:02:27.394292 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application return self._enter() Aug 02 10:02:27.394292 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 02 10:02:27.394292 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 02 10:02:27.394292 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 02 10:02:27.394292 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application return self.creator() Aug 02 10:02:27.394292 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 02 10:02:27.394292 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application created_value = creator( Aug 02 10:02:27.395880 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 02 10:02:27.395880 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 02 10:02:27.395880 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 02 10:02:27.395880 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 02 10:02:27.395880 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 02 10:02:27.395880 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 02 10:02:27.395880 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 02 10:02:27.395880 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application Aug 02 10:02:27.398077 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 294/593] 10.222.0.82 () {56 vars in 1111 bytes} [Tue Aug 2 10:02:27 2022] GET /identity/v3/projects/service => generated 88 bytes in 264 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 02 10:02:27.398997 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-100e0daa-0510-46a7-b292-bf32afa8d2c9 None admin] Authenticating user token {{(pid=80378) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:02:27.446391 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b9e87ca-7bad-4630-8a96-db2ae3a8b58b None admin] Validating token access rules against request {{(pid=80377) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:02:27.507281 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e348fb9e-dc9b-432a-9e53-a50a77520dda None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '891005fa5ff74041b75747d16cc66798', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80377) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:02:27.509182 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e348fb9e-dc9b-432a-9e53-a50a77520dda None admin] REQUEST_METHOD: `GET` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:02:27.509182 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e348fb9e-dc9b-432a-9e53-a50a77520dda None admin] SCRIPT_NAME: `/identity` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:02:27.509451 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e348fb9e-dc9b-432a-9e53-a50a77520dda None admin] PATH_INFO: `/v3/users/neutron` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:02:27.519136 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e348fb9e-dc9b-432a-9e53-a50a77520dda None admin] RBAC: Authorizing `identity:get_user(user_id=neutron)` {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:02:27.520611 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e348fb9e-dc9b-432a-9e53-a50a77520dda None admin] RBAC: Authorization granted {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:02:27.536181 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application [None req-e348fb9e-dc9b-432a-9e53-a50a77520dda None admin] Could not find user: neutron.: keystone.exception.UserNotFound: Could not find user: neutron. Aug 02 10:02:27.536181 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 02 10:02:27.536181 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 02 10:02:27.536181 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 02 10:02:27.536181 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 02 10:02:27.536181 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 02 10:02:27.536181 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 02 10:02:27.536181 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 02 10:02:27.536181 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 02 10:02:27.536181 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 02 10:02:27.536181 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 02 10:02:27.536181 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 02 10:02:27.536181 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 02 10:02:27.536181 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 02 10:02:27.536181 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 02 10:02:27.536181 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 02 10:02:27.537347 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 02 10:02:27.537347 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 02 10:02:27.537347 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 02 10:02:27.537347 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 02 10:02:27.537347 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 02 10:02:27.537347 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 02 10:02:27.537347 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 02 10:02:27.537347 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 02 10:02:27.537347 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 02 10:02:27.537347 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application return self.get_or_create( Aug 02 10:02:27.537347 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 02 10:02:27.537347 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application with Lock( Aug 02 10:02:27.537347 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 02 10:02:27.537347 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application return self._enter() Aug 02 10:02:27.537347 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 02 10:02:27.537347 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 02 10:02:27.537347 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 02 10:02:27.537347 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application return self.creator() Aug 02 10:02:27.538811 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 02 10:02:27.538811 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application created_value = creator( Aug 02 10:02:27.538811 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 02 10:02:27.538811 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 02 10:02:27.538811 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 02 10:02:27.538811 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 02 10:02:27.538811 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 02 10:02:27.538811 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 02 10:02:27.538811 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: neutron. Aug 02 10:02:27.538811 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application Aug 02 10:02:27.538811 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: [pid: 80377|app: 0|req: 300/594] 10.222.0.82 () {56 vars in 1102 bytes} [Tue Aug 2 10:02:27 2022] GET /identity/v3/users/neutron => generated 85 bytes in 251 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 02 10:02:27.538811 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e348fb9e-dc9b-432a-9e53-a50a77520dda None admin] Authenticating user token {{(pid=80377) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:02:27.562978 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-100e0daa-0510-46a7-b292-bf32afa8d2c9 None admin] Validating token access rules against request {{(pid=80378) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:02:27.567452 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG dbcounter [-] [80378] Writing DB stats keystone:SELECT=100 {{(pid=80378) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 10:02:27.591341 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG dbcounter [-] [80377] Writing DB stats keystone:SELECT=100 {{(pid=80377) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 10:02:27.616415 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fdf6ce5c-982d-4a35-bff5-489dc50fc0e8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '891005fa5ff74041b75747d16cc66798', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80378) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:02:27.617070 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdf6ce5c-982d-4a35-bff5-489dc50fc0e8 None admin] REQUEST_METHOD: `GET` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:02:27.617617 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdf6ce5c-982d-4a35-bff5-489dc50fc0e8 None admin] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:02:27.617617 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdf6ce5c-982d-4a35-bff5-489dc50fc0e8 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:02:27.631508 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fdf6ce5c-982d-4a35-bff5-489dc50fc0e8 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:02:27.632680 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fdf6ce5c-982d-4a35-bff5-489dc50fc0e8 None admin] RBAC: Authorization granted {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:02:27.654138 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application [None req-fdf6ce5c-982d-4a35-bff5-489dc50fc0e8 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 02 10:02:27.654138 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 02 10:02:27.654138 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 02 10:02:27.654138 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 02 10:02:27.654138 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 02 10:02:27.654138 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 02 10:02:27.654138 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 02 10:02:27.654138 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 02 10:02:27.654138 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 02 10:02:27.654138 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 02 10:02:27.654138 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 02 10:02:27.654138 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 02 10:02:27.654138 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 02 10:02:27.654138 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 02 10:02:27.654138 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 02 10:02:27.655114 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 02 10:02:27.655114 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 02 10:02:27.655114 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 02 10:02:27.655114 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 02 10:02:27.655114 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 02 10:02:27.655114 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 02 10:02:27.655114 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application return self.get_or_create( Aug 02 10:02:27.655114 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 02 10:02:27.655114 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application with Lock( Aug 02 10:02:27.655114 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 02 10:02:27.655114 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application return self._enter() Aug 02 10:02:27.655114 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 02 10:02:27.655114 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 02 10:02:27.655114 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 02 10:02:27.655114 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application return self.creator() Aug 02 10:02:27.655114 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 02 10:02:27.655114 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application created_value = creator( Aug 02 10:02:27.655998 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 02 10:02:27.655998 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 02 10:02:27.655998 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 02 10:02:27.655998 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application Aug 02 10:02:27.655998 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 295/595] 10.222.0.82 () {56 vars in 1108 bytes} [Tue Aug 2 10:02:27 2022] GET /identity/v3/domains/Default => generated 87 bytes in 257 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 02 10:02:27.656839 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c46367b5-02b5-420d-a7bb-0feb73a7bb6a None None] REQUEST_METHOD: `GET` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:02:27.656839 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c46367b5-02b5-420d-a7bb-0feb73a7bb6a None None] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:02:27.657078 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c46367b5-02b5-420d-a7bb-0feb73a7bb6a None None] PATH_INFO: `/` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:02:27.659583 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 296/596] 10.222.0.82 () {54 vars in 926 bytes} [Tue Aug 2 10:02:27 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 02 10:02:27.660356 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c46367b5-02b5-420d-a7bb-0feb73a7bb6a None None] Authenticating user token {{(pid=80378) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:02:27.711475 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e348fb9e-dc9b-432a-9e53-a50a77520dda None admin] Validating token access rules against request {{(pid=80377) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:02:27.774068 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3cc3f536-20ff-4a55-a8a0-7232f400b759 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '891005fa5ff74041b75747d16cc66798', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80377) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:02:27.775010 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cc3f536-20ff-4a55-a8a0-7232f400b759 None admin] REQUEST_METHOD: `GET` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:02:27.775303 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cc3f536-20ff-4a55-a8a0-7232f400b759 None admin] SCRIPT_NAME: `/identity` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:02:27.775578 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cc3f536-20ff-4a55-a8a0-7232f400b759 None admin] PATH_INFO: `/v3/projects` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:02:27.776836 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3cc3f536-20ff-4a55-a8a0-7232f400b759 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80377) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 02 10:02:27.777297 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3cc3f536-20ff-4a55-a8a0-7232f400b759 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:02:27.779187 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3cc3f536-20ff-4a55-a8a0-7232f400b759 None admin] RBAC: Authorization granted {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:02:27.801492 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: [pid: 80377|app: 0|req: 301/597] 10.222.0.82 () {56 vars in 1148 bytes} [Tue Aug 2 10:02:27 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 263 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 02 10:02:27.802753 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3cc3f536-20ff-4a55-a8a0-7232f400b759 None admin] Authenticating user token {{(pid=80377) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:02:27.821107 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c46367b5-02b5-420d-a7bb-0feb73a7bb6a None None] Validating token access rules against request {{(pid=80378) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:02:27.867349 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-478948c0-9949-4aba-82b7-5349d9292402 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '891005fa5ff74041b75747d16cc66798', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80378) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:02:27.868401 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-478948c0-9949-4aba-82b7-5349d9292402 None admin] REQUEST_METHOD: `GET` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:02:27.868890 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-478948c0-9949-4aba-82b7-5349d9292402 None admin] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:02:27.869374 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-478948c0-9949-4aba-82b7-5349d9292402 None admin] PATH_INFO: `/v3/users` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:02:27.870869 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-478948c0-9949-4aba-82b7-5349d9292402 None admin] RBAC: Adding query filter params (domain_id=default, name=neutron) {{(pid=80378) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 02 10:02:27.871413 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-478948c0-9949-4aba-82b7-5349d9292402 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:02:27.873820 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-478948c0-9949-4aba-82b7-5349d9292402 None admin] RBAC: Authorization granted {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:02:27.906446 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 297/598] 10.222.0.82 () {56 vars in 1139 bytes} [Tue Aug 2 10:02:27 2022] GET /identity/v3/users?domain_id=default&name=neutron => generated 374 bytes in 240 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 02 10:02:27.906446 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f5ecf5f-a42c-49a2-b1cd-9ad80e4acaea None None] REQUEST_METHOD: `POST` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:02:27.906446 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f5ecf5f-a42c-49a2-b1cd-9ad80e4acaea None None] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:02:27.906446 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f5ecf5f-a42c-49a2-b1cd-9ad80e4acaea None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:02:27.932729 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.auth.core [None req-2f5ecf5f-a42c-49a2-b1cd-9ad80e4acaea None None] MFA Rules not processed for user `891005fa5ff74041b75747d16cc66798`. Rule list: `[]` (Enabled: `True`). {{(pid=80378) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 02 10:02:27.933987 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.fernet_utils [None req-2f5ecf5f-a42c-49a2-b1cd-9ad80e4acaea None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80378) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 10:02:27.946817 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG dbcounter [-] [80378] Writing DB stats keystone:SELECT=100 {{(pid=80378) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 10:02:27.949715 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG dbcounter [-] [80377] Writing DB stats keystone:SELECT=100 {{(pid=80377) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 10:02:27.965383 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3cc3f536-20ff-4a55-a8a0-7232f400b759 None admin] Validating token access rules against request {{(pid=80377) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:02:28.022611 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6881bea-9a69-44e2-a0e2-45287bd75e29 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '891005fa5ff74041b75747d16cc66798', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80377) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:02:28.023601 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6881bea-9a69-44e2-a0e2-45287bd75e29 None admin] REQUEST_METHOD: `GET` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:02:28.023991 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6881bea-9a69-44e2-a0e2-45287bd75e29 None admin] SCRIPT_NAME: `/identity` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:02:28.024337 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6881bea-9a69-44e2-a0e2-45287bd75e29 None admin] PATH_INFO: `/v3/domains` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:02:28.025213 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6881bea-9a69-44e2-a0e2-45287bd75e29 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80377) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 02 10:02:28.025715 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6881bea-9a69-44e2-a0e2-45287bd75e29 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:02:28.028253 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6881bea-9a69-44e2-a0e2-45287bd75e29 None admin] RBAC: Authorization granted {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:02:28.057139 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: [pid: 80377|app: 0|req: 302/599] 10.222.0.82 () {56 vars in 1109 bytes} [Tue Aug 2 10:02:27 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 248 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 02 10:02:28.057139 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6881bea-9a69-44e2-a0e2-45287bd75e29 None admin] Authenticating user token {{(pid=80377) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:02:28.059937 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 298/600] 10.222.0.82 () {58 vars in 1023 bytes} [Tue Aug 2 10:02:27 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 159 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 02 10:02:28.061323 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f5ecf5f-a42c-49a2-b1cd-9ad80e4acaea None None] Authenticating user token {{(pid=80378) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:02:28.216079 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6881bea-9a69-44e2-a0e2-45287bd75e29 None admin] Validating token access rules against request {{(pid=80377) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:02:28.254399 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f5ecf5f-a42c-49a2-b1cd-9ad80e4acaea None None] Validating token access rules against request {{(pid=80378) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:02:28.267903 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3f8389c-c666-4866-9fbc-72d451269c08 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '891005fa5ff74041b75747d16cc66798', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80377) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:02:28.268930 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3f8389c-c666-4866-9fbc-72d451269c08 None admin] REQUEST_METHOD: `PUT` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:02:28.269222 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3f8389c-c666-4866-9fbc-72d451269c08 None admin] SCRIPT_NAME: `/identity` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:02:28.269503 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3f8389c-c666-4866-9fbc-72d451269c08 None admin] PATH_INFO: `/v3/projects/f1b38ebb804c41c69772c998aae72c41/users/1023de281ae74e9d8a6d6c312c1c97a7/roles/02003a0be60f47f2a8dfea17053af52c` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:02:28.289104 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3f8389c-c666-4866-9fbc-72d451269c08 None admin] RBAC: Authorizing `identity:create_grant(project_id=f1b38ebb804c41c69772c998aae72c41, user_id=1023de281ae74e9d8a6d6c312c1c97a7, role_id=02003a0be60f47f2a8dfea17053af52c)` {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:02:28.290532 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3f8389c-c666-4866-9fbc-72d451269c08 None admin] RBAC: Authorization granted {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:02:28.293878 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG dbcounter [-] [80377] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80377) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 10:02:28.299096 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b361069-b047-4572-8016-2817b51ecda0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '891005fa5ff74041b75747d16cc66798', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80378) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:02:28.299978 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b361069-b047-4572-8016-2817b51ecda0 None admin] REQUEST_METHOD: `GET` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:02:28.300363 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b361069-b047-4572-8016-2817b51ecda0 None admin] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:02:28.300664 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b361069-b047-4572-8016-2817b51ecda0 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:02:28.310866 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: [pid: 80377|app: 0|req: 303/601] 10.222.0.82 () {58 vars in 1439 bytes} [Tue Aug 2 10:02:28 2022] PUT /identity/v3/projects/f1b38ebb804c41c69772c998aae72c41/users/1023de281ae74e9d8a6d6c312c1c97a7/roles/02003a0be60f47f2a8dfea17053af52c => generated 0 bytes in 258 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 02 10:02:28.310866 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3f8389c-c666-4866-9fbc-72d451269c08 None admin] Authenticating user token {{(pid=80377) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:02:28.316791 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b361069-b047-4572-8016-2817b51ecda0 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:02:28.318446 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b361069-b047-4572-8016-2817b51ecda0 None admin] RBAC: Authorization granted {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:02:28.322125 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG dbcounter [-] [80378] Writing DB stats keystone:SELECT=100 {{(pid=80378) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 10:02:28.349263 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application [None req-4b361069-b047-4572-8016-2817b51ecda0 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 02 10:02:28.349263 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 02 10:02:28.349263 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 02 10:02:28.349263 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 02 10:02:28.349263 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 02 10:02:28.349263 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 02 10:02:28.349263 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 02 10:02:28.349263 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 02 10:02:28.349263 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 02 10:02:28.349263 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 02 10:02:28.349263 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 02 10:02:28.349263 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 02 10:02:28.349263 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 02 10:02:28.349263 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 02 10:02:28.349263 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 02 10:02:28.351325 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 02 10:02:28.351325 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 02 10:02:28.351325 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 02 10:02:28.351325 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 02 10:02:28.351325 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 02 10:02:28.351325 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 02 10:02:28.351325 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application return self.get_or_create( Aug 02 10:02:28.351325 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 02 10:02:28.351325 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application with Lock( Aug 02 10:02:28.351325 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 02 10:02:28.351325 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application return self._enter() Aug 02 10:02:28.351325 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 02 10:02:28.351325 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 02 10:02:28.351325 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 02 10:02:28.351325 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application return self.creator() Aug 02 10:02:28.351325 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 02 10:02:28.351325 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application created_value = creator( Aug 02 10:02:28.353461 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 02 10:02:28.353461 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 02 10:02:28.353461 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 02 10:02:28.353461 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application Aug 02 10:02:28.355450 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 299/602] 10.222.0.82 () {56 vars in 1108 bytes} [Tue Aug 2 10:02:28 2022] GET /identity/v3/domains/Default => generated 87 bytes in 295 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 02 10:02:28.357289 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b361069-b047-4572-8016-2817b51ecda0 None admin] Authenticating user token {{(pid=80378) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:02:28.530583 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3f8389c-c666-4866-9fbc-72d451269c08 None admin] Validating token access rules against request {{(pid=80377) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:02:28.549774 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b361069-b047-4572-8016-2817b51ecda0 None admin] Validating token access rules against request {{(pid=80378) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:02:28.586018 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31628ffb-bb34-44ad-80a3-ac29cf3f98b9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '891005fa5ff74041b75747d16cc66798', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80377) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:02:28.586997 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31628ffb-bb34-44ad-80a3-ac29cf3f98b9 None admin] REQUEST_METHOD: `GET` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:02:28.587480 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31628ffb-bb34-44ad-80a3-ac29cf3f98b9 None admin] SCRIPT_NAME: `/identity` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:02:28.587959 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31628ffb-bb34-44ad-80a3-ac29cf3f98b9 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:02:28.598965 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31628ffb-bb34-44ad-80a3-ac29cf3f98b9 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:02:28.600902 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31628ffb-bb34-44ad-80a3-ac29cf3f98b9 None admin] RBAC: Authorization granted {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:02:28.619673 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application [None req-31628ffb-bb34-44ad-80a3-ac29cf3f98b9 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 02 10:02:28.619673 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 02 10:02:28.619673 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 02 10:02:28.619673 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 02 10:02:28.619673 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 02 10:02:28.619673 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 02 10:02:28.619673 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 02 10:02:28.619673 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 02 10:02:28.619673 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 02 10:02:28.619673 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 02 10:02:28.619673 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 02 10:02:28.619673 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 02 10:02:28.619673 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 02 10:02:28.619673 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 02 10:02:28.619673 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 02 10:02:28.621487 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 02 10:02:28.621487 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 02 10:02:28.621487 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 02 10:02:28.621487 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 02 10:02:28.621487 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 02 10:02:28.621487 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 02 10:02:28.621487 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application return self.get_or_create( Aug 02 10:02:28.621487 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 02 10:02:28.621487 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application with Lock( Aug 02 10:02:28.621487 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 02 10:02:28.621487 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application return self._enter() Aug 02 10:02:28.621487 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 02 10:02:28.621487 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 02 10:02:28.621487 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 02 10:02:28.621487 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application return self.creator() Aug 02 10:02:28.621487 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 02 10:02:28.621487 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application created_value = creator( Aug 02 10:02:28.623358 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 02 10:02:28.623358 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 02 10:02:28.623358 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 02 10:02:28.623358 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 02 10:02:28.623358 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 02 10:02:28.623358 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 02 10:02:28.623358 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 02 10:02:28.623358 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application Aug 02 10:02:28.623358 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: [pid: 80377|app: 0|req: 304/603] 10.222.0.82 () {56 vars in 1111 bytes} [Tue Aug 2 10:02:28 2022] GET /identity/v3/projects/service => generated 88 bytes in 311 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 02 10:02:28.623358 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31628ffb-bb34-44ad-80a3-ac29cf3f98b9 None admin] Authenticating user token {{(pid=80377) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:02:28.638188 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6aac61f-e91c-47a2-9d97-be0cd5510191 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '891005fa5ff74041b75747d16cc66798', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80378) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:02:28.639169 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6aac61f-e91c-47a2-9d97-be0cd5510191 None admin] REQUEST_METHOD: `GET` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:02:28.639621 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6aac61f-e91c-47a2-9d97-be0cd5510191 None admin] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:02:28.640063 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6aac61f-e91c-47a2-9d97-be0cd5510191 None admin] PATH_INFO: `/v3/roles/service` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:02:28.663951 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6aac61f-e91c-47a2-9d97-be0cd5510191 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 02 10:02:28.677273 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application [None req-e6aac61f-e91c-47a2-9d97-be0cd5510191 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Aug 02 10:02:28.677273 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 02 10:02:28.677273 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 02 10:02:28.677273 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 02 10:02:28.677273 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 02 10:02:28.677273 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 02 10:02:28.677273 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 02 10:02:28.677273 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 02 10:02:28.677273 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 02 10:02:28.677273 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 02 10:02:28.677273 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 02 10:02:28.677273 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 02 10:02:28.677273 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 02 10:02:28.677273 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 02 10:02:28.677273 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 02 10:02:28.677273 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 02 10:02:28.678344 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 02 10:02:28.678344 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 02 10:02:28.678344 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 02 10:02:28.678344 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 02 10:02:28.678344 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 02 10:02:28.678344 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 02 10:02:28.678344 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 02 10:02:28.678344 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 02 10:02:28.678344 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 02 10:02:28.678344 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application return self.get_or_create( Aug 02 10:02:28.678344 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 02 10:02:28.678344 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application with Lock( Aug 02 10:02:28.678344 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 02 10:02:28.678344 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application return self._enter() Aug 02 10:02:28.678344 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 02 10:02:28.678344 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 02 10:02:28.679245 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 02 10:02:28.679245 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application return self.creator() Aug 02 10:02:28.679245 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 02 10:02:28.679245 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application created_value = creator( Aug 02 10:02:28.679245 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 02 10:02:28.679245 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 02 10:02:28.679245 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 02 10:02:28.679245 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 02 10:02:28.679245 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 02 10:02:28.679245 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 02 10:02:28.679245 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Aug 02 10:02:28.679245 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application Aug 02 10:02:28.682881 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 300/604] 10.222.0.82 () {56 vars in 1102 bytes} [Tue Aug 2 10:02:28 2022] GET /identity/v3/roles/service => generated 85 bytes in 327 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 02 10:02:28.684260 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6aac61f-e91c-47a2-9d97-be0cd5510191 None admin] Authenticating user token {{(pid=80378) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:02:28.723681 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG dbcounter [-] [80377] Writing DB stats keystone:SELECT=100 {{(pid=80377) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 10:02:28.796770 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG dbcounter [-] [80378] Writing DB stats keystone:SELECT=100 {{(pid=80378) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 10:02:28.811335 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31628ffb-bb34-44ad-80a3-ac29cf3f98b9 None admin] Validating token access rules against request {{(pid=80377) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:02:28.859870 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6aac61f-e91c-47a2-9d97-be0cd5510191 None admin] Validating token access rules against request {{(pid=80378) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:02:28.877465 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-991f9a67-4e2b-401d-ac56-cd6ee092e16c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '891005fa5ff74041b75747d16cc66798', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80377) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:02:28.877465 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-991f9a67-4e2b-401d-ac56-cd6ee092e16c None admin] REQUEST_METHOD: `GET` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:02:28.877465 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-991f9a67-4e2b-401d-ac56-cd6ee092e16c None admin] SCRIPT_NAME: `/identity` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:02:28.877465 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-991f9a67-4e2b-401d-ac56-cd6ee092e16c None admin] PATH_INFO: `/v3/domains` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:02:28.878110 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-991f9a67-4e2b-401d-ac56-cd6ee092e16c None admin] RBAC: Adding query filter params (name=Default) {{(pid=80377) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 02 10:02:28.878110 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-991f9a67-4e2b-401d-ac56-cd6ee092e16c None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:02:28.879500 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-991f9a67-4e2b-401d-ac56-cd6ee092e16c None admin] RBAC: Authorization granted {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:02:28.904314 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: [pid: 80377|app: 0|req: 305/605] 10.222.0.82 () {56 vars in 1109 bytes} [Tue Aug 2 10:02:28 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 279 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 02 10:02:28.904314 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-991f9a67-4e2b-401d-ac56-cd6ee092e16c None admin] Authenticating user token {{(pid=80377) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:02:28.917100 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b130770-8d0e-4220-be5e-0d11730fcd92 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '891005fa5ff74041b75747d16cc66798', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80378) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:02:28.917758 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b130770-8d0e-4220-be5e-0d11730fcd92 None admin] REQUEST_METHOD: `GET` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:02:28.918270 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b130770-8d0e-4220-be5e-0d11730fcd92 None admin] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:02:28.918745 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b130770-8d0e-4220-be5e-0d11730fcd92 None admin] PATH_INFO: `/v3/projects` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:02:28.919536 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b130770-8d0e-4220-be5e-0d11730fcd92 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80378) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 02 10:02:28.919841 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b130770-8d0e-4220-be5e-0d11730fcd92 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:02:28.921462 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b130770-8d0e-4220-be5e-0d11730fcd92 None admin] RBAC: Authorization granted {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:02:28.949049 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 301/606] 10.222.0.82 () {56 vars in 1148 bytes} [Tue Aug 2 10:02:28 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 260 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 02 10:02:28.949049 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c9abed6-537f-497e-aaa8-e1f7d5c06b99 None None] REQUEST_METHOD: `GET` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:02:28.949049 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c9abed6-537f-497e-aaa8-e1f7d5c06b99 None None] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:02:28.949049 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c9abed6-537f-497e-aaa8-e1f7d5c06b99 None None] PATH_INFO: `/` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:02:28.949049 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 302/607] 10.222.0.82 () {54 vars in 926 bytes} [Tue Aug 2 10:02:28 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 02 10:02:28.949049 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c9abed6-537f-497e-aaa8-e1f7d5c06b99 None None] Authenticating user token {{(pid=80378) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:02:29.097525 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-991f9a67-4e2b-401d-ac56-cd6ee092e16c None admin] Validating token access rules against request {{(pid=80377) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:02:29.120490 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG dbcounter [-] [80377] Writing DB stats keystone:SELECT=100 {{(pid=80377) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 10:02:29.128190 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c9abed6-537f-497e-aaa8-e1f7d5c06b99 None None] Validating token access rules against request {{(pid=80378) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:02:29.145258 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b290a9fd-7290-4a19-ab26-8f35e7082d65 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '891005fa5ff74041b75747d16cc66798', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80377) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:02:29.145930 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b290a9fd-7290-4a19-ab26-8f35e7082d65 None admin] REQUEST_METHOD: `GET` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:02:29.146213 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b290a9fd-7290-4a19-ab26-8f35e7082d65 None admin] SCRIPT_NAME: `/identity` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:02:29.146487 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b290a9fd-7290-4a19-ab26-8f35e7082d65 None admin] PATH_INFO: `/v3/roles` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:02:29.147217 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b290a9fd-7290-4a19-ab26-8f35e7082d65 None admin] RBAC: Adding query filter params (name=service) {{(pid=80377) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 02 10:02:29.147569 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b290a9fd-7290-4a19-ab26-8f35e7082d65 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:02:29.149089 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b290a9fd-7290-4a19-ab26-8f35e7082d65 None admin] RBAC: Authorization granted {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:02:29.168900 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: [pid: 80377|app: 0|req: 306/608] 10.222.0.82 () {56 vars in 1103 bytes} [Tue Aug 2 10:02:28 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 261 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 02 10:02:29.168900 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b290a9fd-7290-4a19-ab26-8f35e7082d65 None admin] Authenticating user token {{(pid=80377) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:02:29.170549 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG dbcounter [-] [80378] Writing DB stats keystone:SELECT=100 {{(pid=80378) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 10:02:29.204228 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c573ccdf-4726-454e-aa81-e8269f443b2f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '891005fa5ff74041b75747d16cc66798', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80378) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:02:29.204995 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c573ccdf-4726-454e-aa81-e8269f443b2f None admin] REQUEST_METHOD: `GET` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:02:29.205351 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c573ccdf-4726-454e-aa81-e8269f443b2f None admin] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:02:29.205702 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c573ccdf-4726-454e-aa81-e8269f443b2f None admin] PATH_INFO: `/v3/projects/service` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:02:29.213524 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c573ccdf-4726-454e-aa81-e8269f443b2f None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:02:29.215069 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c573ccdf-4726-454e-aa81-e8269f443b2f None admin] RBAC: Authorization granted {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:02:29.228338 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application [None req-c573ccdf-4726-454e-aa81-e8269f443b2f None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 02 10:02:29.228338 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 02 10:02:29.228338 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 02 10:02:29.228338 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 02 10:02:29.228338 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 02 10:02:29.228338 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 02 10:02:29.228338 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 02 10:02:29.228338 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 02 10:02:29.228338 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 02 10:02:29.228338 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 02 10:02:29.228338 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 02 10:02:29.228338 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 02 10:02:29.228338 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 02 10:02:29.228338 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 02 10:02:29.228338 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 02 10:02:29.229481 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 02 10:02:29.229481 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 02 10:02:29.229481 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 02 10:02:29.229481 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 02 10:02:29.229481 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 02 10:02:29.229481 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 02 10:02:29.229481 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application return self.get_or_create( Aug 02 10:02:29.229481 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 02 10:02:29.229481 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application with Lock( Aug 02 10:02:29.229481 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 02 10:02:29.229481 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application return self._enter() Aug 02 10:02:29.229481 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 02 10:02:29.229481 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 02 10:02:29.229481 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 02 10:02:29.229481 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application return self.creator() Aug 02 10:02:29.229481 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 02 10:02:29.229481 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application created_value = creator( Aug 02 10:02:29.230779 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 02 10:02:29.230779 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 02 10:02:29.230779 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 02 10:02:29.230779 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 02 10:02:29.230779 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 02 10:02:29.230779 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 02 10:02:29.230779 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 02 10:02:29.230779 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application Aug 02 10:02:29.230779 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 303/609] 10.222.0.82 () {56 vars in 1111 bytes} [Tue Aug 2 10:02:28 2022] GET /identity/v3/projects/service => generated 88 bytes in 281 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 02 10:02:29.230779 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d925b1d5-d8cd-4542-854d-5a384df5231f None None] REQUEST_METHOD: `POST` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:02:29.231584 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d925b1d5-d8cd-4542-854d-5a384df5231f None None] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:02:29.231584 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d925b1d5-d8cd-4542-854d-5a384df5231f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:02:29.255884 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.auth.core [None req-d925b1d5-d8cd-4542-854d-5a384df5231f None None] MFA Rules not processed for user `891005fa5ff74041b75747d16cc66798`. Rule list: `[]` (Enabled: `True`). {{(pid=80378) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 02 10:02:29.257086 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.fernet_utils [None req-d925b1d5-d8cd-4542-854d-5a384df5231f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80378) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 10:02:29.320879 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b290a9fd-7290-4a19-ab26-8f35e7082d65 None admin] Validating token access rules against request {{(pid=80377) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:02:29.362485 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 304/610] 10.222.0.82 () {58 vars in 1023 bytes} [Tue Aug 2 10:02:29 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 132 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 02 10:02:29.370835 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d925b1d5-d8cd-4542-854d-5a384df5231f None None] Authenticating user token {{(pid=80378) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:02:29.372250 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2135e7c4-32ca-492a-a12b-9084b498431b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '891005fa5ff74041b75747d16cc66798', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80377) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:02:29.373531 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2135e7c4-32ca-492a-a12b-9084b498431b None admin] REQUEST_METHOD: `PUT` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:02:29.377541 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2135e7c4-32ca-492a-a12b-9084b498431b None admin] SCRIPT_NAME: `/identity` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:02:29.378227 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2135e7c4-32ca-492a-a12b-9084b498431b None admin] PATH_INFO: `/v3/projects/f1b38ebb804c41c69772c998aae72c41/users/fa964af53a2b4fb9ace6afb544d54d40/roles/02003a0be60f47f2a8dfea17053af52c` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:02:29.406241 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2135e7c4-32ca-492a-a12b-9084b498431b None admin] RBAC: Authorizing `identity:create_grant(project_id=f1b38ebb804c41c69772c998aae72c41, user_id=fa964af53a2b4fb9ace6afb544d54d40, role_id=02003a0be60f47f2a8dfea17053af52c)` {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:02:29.409584 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2135e7c4-32ca-492a-a12b-9084b498431b None admin] RBAC: Authorization granted {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:02:29.445823 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: [pid: 80377|app: 0|req: 307/611] 10.222.0.82 () {58 vars in 1439 bytes} [Tue Aug 2 10:02:29 2022] PUT /identity/v3/projects/f1b38ebb804c41c69772c998aae72c41/users/fa964af53a2b4fb9ace6afb544d54d40/roles/02003a0be60f47f2a8dfea17053af52c => generated 0 bytes in 272 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 02 10:02:29.445823 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2135e7c4-32ca-492a-a12b-9084b498431b None admin] Authenticating user token {{(pid=80377) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:02:29.501926 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG dbcounter [-] [80377] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80377) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 10:02:29.588891 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d925b1d5-d8cd-4542-854d-5a384df5231f None None] Validating token access rules against request {{(pid=80378) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:02:29.602109 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG dbcounter [-] [80378] Writing DB stats keystone:SELECT=100 {{(pid=80378) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 10:02:29.640998 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2135e7c4-32ca-492a-a12b-9084b498431b None admin] Validating token access rules against request {{(pid=80377) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:02:29.656382 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d277fa2-d576-4bb2-983a-d4f9f32e7649 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '891005fa5ff74041b75747d16cc66798', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80378) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:02:29.657047 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d277fa2-d576-4bb2-983a-d4f9f32e7649 None admin] REQUEST_METHOD: `GET` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:02:29.657402 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d277fa2-d576-4bb2-983a-d4f9f32e7649 None admin] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:02:29.658229 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d277fa2-d576-4bb2-983a-d4f9f32e7649 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:02:29.675842 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d277fa2-d576-4bb2-983a-d4f9f32e7649 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:02:29.677557 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d277fa2-d576-4bb2-983a-d4f9f32e7649 None admin] RBAC: Authorization granted {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:02:29.693166 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b962e8f9-7136-44d0-abcd-999ff0095208 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '891005fa5ff74041b75747d16cc66798', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80377) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:02:29.693887 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b962e8f9-7136-44d0-abcd-999ff0095208 None admin] REQUEST_METHOD: `GET` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:02:29.694244 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b962e8f9-7136-44d0-abcd-999ff0095208 None admin] SCRIPT_NAME: `/identity` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:02:29.694529 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b962e8f9-7136-44d0-abcd-999ff0095208 None admin] PATH_INFO: `/v3/projects` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:02:29.695407 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b962e8f9-7136-44d0-abcd-999ff0095208 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80377) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 02 10:02:29.695731 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b962e8f9-7136-44d0-abcd-999ff0095208 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:02:29.697644 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b962e8f9-7136-44d0-abcd-999ff0095208 None admin] RBAC: Authorization granted {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:02:29.698629 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application [None req-3d277fa2-d576-4bb2-983a-d4f9f32e7649 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 02 10:02:29.698629 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 02 10:02:29.698629 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 02 10:02:29.698629 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 02 10:02:29.698629 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 02 10:02:29.698629 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 02 10:02:29.698629 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 02 10:02:29.698629 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 02 10:02:29.698629 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 02 10:02:29.698629 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 02 10:02:29.698629 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 02 10:02:29.698629 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 02 10:02:29.698629 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 02 10:02:29.698629 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 02 10:02:29.698629 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 02 10:02:29.699506 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 02 10:02:29.699506 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 02 10:02:29.699506 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 02 10:02:29.699506 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 02 10:02:29.699506 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 02 10:02:29.699506 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 02 10:02:29.699506 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application return self.get_or_create( Aug 02 10:02:29.699506 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 02 10:02:29.699506 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application with Lock( Aug 02 10:02:29.699506 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 02 10:02:29.699506 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application return self._enter() Aug 02 10:02:29.699506 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 02 10:02:29.699506 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 02 10:02:29.699506 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 02 10:02:29.699506 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application return self.creator() Aug 02 10:02:29.699506 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 02 10:02:29.699506 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application created_value = creator( Aug 02 10:02:29.700433 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 02 10:02:29.700433 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 02 10:02:29.700433 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 02 10:02:29.700433 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application Aug 02 10:02:29.700433 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 305/612] 10.222.0.82 () {56 vars in 1108 bytes} [Tue Aug 2 10:02:29 2022] GET /identity/v3/domains/Default => generated 87 bytes in 337 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 02 10:02:29.701334 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d277fa2-d576-4bb2-983a-d4f9f32e7649 None admin] Authenticating user token {{(pid=80378) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:02:29.722992 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: [pid: 80377|app: 0|req: 308/613] 10.222.0.82 () {56 vars in 1148 bytes} [Tue Aug 2 10:02:29 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 286 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 02 10:02:29.724581 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b962e8f9-7136-44d0-abcd-999ff0095208 None admin] Authenticating user token {{(pid=80377) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:02:29.862102 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d277fa2-d576-4bb2-983a-d4f9f32e7649 None admin] Validating token access rules against request {{(pid=80378) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:02:29.866597 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG dbcounter [-] [80377] Writing DB stats keystone:SELECT=100 {{(pid=80377) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 10:02:29.883566 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b962e8f9-7136-44d0-abcd-999ff0095208 None admin] Validating token access rules against request {{(pid=80377) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:02:29.913639 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bc6dcad-87b0-4d90-9788-757ac86fbb84 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '891005fa5ff74041b75747d16cc66798', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80378) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:02:29.914578 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bc6dcad-87b0-4d90-9788-757ac86fbb84 None admin] REQUEST_METHOD: `GET` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:02:29.915006 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bc6dcad-87b0-4d90-9788-757ac86fbb84 None admin] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:02:29.915508 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bc6dcad-87b0-4d90-9788-757ac86fbb84 None admin] PATH_INFO: `/v3/roles/service` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:02:29.932158 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7bc6dcad-87b0-4d90-9788-757ac86fbb84 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 02 10:02:29.939769 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application [None req-7bc6dcad-87b0-4d90-9788-757ac86fbb84 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Aug 02 10:02:29.939769 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 02 10:02:29.939769 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 02 10:02:29.939769 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 02 10:02:29.939769 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 02 10:02:29.939769 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 02 10:02:29.939769 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 02 10:02:29.939769 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 02 10:02:29.939769 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 02 10:02:29.939769 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 02 10:02:29.939769 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 02 10:02:29.939769 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 02 10:02:29.939769 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 02 10:02:29.939769 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 02 10:02:29.939769 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 02 10:02:29.939769 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 02 10:02:29.940925 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 02 10:02:29.940925 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 02 10:02:29.940925 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 02 10:02:29.940925 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 02 10:02:29.940925 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 02 10:02:29.940925 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 02 10:02:29.940925 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 02 10:02:29.940925 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 02 10:02:29.940925 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 02 10:02:29.940925 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application return self.get_or_create( Aug 02 10:02:29.940925 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 02 10:02:29.940925 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application with Lock( Aug 02 10:02:29.940925 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 02 10:02:29.940925 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application return self._enter() Aug 02 10:02:29.940925 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 02 10:02:29.940925 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 02 10:02:29.941948 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 02 10:02:29.941948 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application return self.creator() Aug 02 10:02:29.941948 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 02 10:02:29.941948 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application created_value = creator( Aug 02 10:02:29.941948 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 02 10:02:29.941948 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 02 10:02:29.941948 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 02 10:02:29.941948 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 02 10:02:29.941948 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 02 10:02:29.941948 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 02 10:02:29.941948 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Aug 02 10:02:29.941948 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application Aug 02 10:02:29.948377 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 306/614] 10.222.0.82 () {56 vars in 1102 bytes} [Tue Aug 2 10:02:29 2022] GET /identity/v3/roles/service => generated 85 bytes in 242 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 02 10:02:29.948377 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bc6dcad-87b0-4d90-9788-757ac86fbb84 None admin] Authenticating user token {{(pid=80378) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:02:29.952749 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8d7b222-3582-4947-b1b6-31ba4c146ffc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '891005fa5ff74041b75747d16cc66798', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80377) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:02:29.954598 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8d7b222-3582-4947-b1b6-31ba4c146ffc None admin] REQUEST_METHOD: `GET` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:02:29.954598 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8d7b222-3582-4947-b1b6-31ba4c146ffc None admin] SCRIPT_NAME: `/identity` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:02:29.954598 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8d7b222-3582-4947-b1b6-31ba4c146ffc None admin] PATH_INFO: `/v3/domains` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:02:29.955571 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8d7b222-3582-4947-b1b6-31ba4c146ffc None admin] RBAC: Adding query filter params (name=Default) {{(pid=80377) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 02 10:02:29.956000 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8d7b222-3582-4947-b1b6-31ba4c146ffc None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:02:29.957942 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8d7b222-3582-4947-b1b6-31ba4c146ffc None admin] RBAC: Authorization granted {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:02:29.980389 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: [pid: 80377|app: 0|req: 309/615] 10.222.0.82 () {56 vars in 1109 bytes} [Tue Aug 2 10:02:29 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 252 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 02 10:02:29.981259 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8d7b222-3582-4947-b1b6-31ba4c146ffc None admin] Authenticating user token {{(pid=80377) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:02:29.986839 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG dbcounter [-] [80378] Writing DB stats keystone:SELECT=100 {{(pid=80378) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 10:02:30.125893 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bc6dcad-87b0-4d90-9788-757ac86fbb84 None admin] Validating token access rules against request {{(pid=80378) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:02:30.136577 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8d7b222-3582-4947-b1b6-31ba4c146ffc None admin] Validating token access rules against request {{(pid=80377) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:02:30.186357 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb8246a3-142e-44bb-8d42-acebc363c810 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '891005fa5ff74041b75747d16cc66798', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80378) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:02:30.187838 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb8246a3-142e-44bb-8d42-acebc363c810 None admin] REQUEST_METHOD: `PUT` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:02:30.188917 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb8246a3-142e-44bb-8d42-acebc363c810 None admin] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:02:30.189358 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb8246a3-142e-44bb-8d42-acebc363c810 None admin] PATH_INFO: `/v3/projects/f1b38ebb804c41c69772c998aae72c41/users/33b87170f0634b67bce60750847b5cbe/roles/02003a0be60f47f2a8dfea17053af52c` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:02:30.203721 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39a33eb8-b86b-4ab9-9e3c-0f81033d7da9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '891005fa5ff74041b75747d16cc66798', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80377) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:02:30.204412 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39a33eb8-b86b-4ab9-9e3c-0f81033d7da9 None admin] REQUEST_METHOD: `GET` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:02:30.204702 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39a33eb8-b86b-4ab9-9e3c-0f81033d7da9 None admin] SCRIPT_NAME: `/identity` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:02:30.205004 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39a33eb8-b86b-4ab9-9e3c-0f81033d7da9 None admin] PATH_INFO: `/v3/roles` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:02:30.205750 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39a33eb8-b86b-4ab9-9e3c-0f81033d7da9 None admin] RBAC: Adding query filter params (name=service) {{(pid=80377) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 02 10:02:30.206172 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39a33eb8-b86b-4ab9-9e3c-0f81033d7da9 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:02:30.207577 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39a33eb8-b86b-4ab9-9e3c-0f81033d7da9 None admin] RBAC: Authorization granted {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:02:30.216100 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb8246a3-142e-44bb-8d42-acebc363c810 None admin] RBAC: Authorizing `identity:create_grant(project_id=f1b38ebb804c41c69772c998aae72c41, user_id=33b87170f0634b67bce60750847b5cbe, role_id=02003a0be60f47f2a8dfea17053af52c)` {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:02:30.218382 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb8246a3-142e-44bb-8d42-acebc363c810 None admin] RBAC: Authorization granted {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:02:30.225371 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: [pid: 80377|app: 0|req: 310/616] 10.222.0.82 () {56 vars in 1103 bytes} [Tue Aug 2 10:02:29 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 239 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 02 10:02:30.225371 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39a33eb8-b86b-4ab9-9e3c-0f81033d7da9 None admin] Authenticating user token {{(pid=80377) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:02:30.241110 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG dbcounter [-] [80377] Writing DB stats keystone:SELECT=100 {{(pid=80377) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 10:02:30.251006 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 307/617] 10.222.0.82 () {58 vars in 1439 bytes} [Tue Aug 2 10:02:29 2022] PUT /identity/v3/projects/f1b38ebb804c41c69772c998aae72c41/users/33b87170f0634b67bce60750847b5cbe/roles/02003a0be60f47f2a8dfea17053af52c => generated 0 bytes in 307 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 02 10:02:30.253275 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb8246a3-142e-44bb-8d42-acebc363c810 None admin] Authenticating user token {{(pid=80378) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:02:30.371837 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39a33eb8-b86b-4ab9-9e3c-0f81033d7da9 None admin] Validating token access rules against request {{(pid=80377) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:02:30.385631 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG dbcounter [-] [80378] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80378) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 10:02:30.425021 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a21268eb-37d2-4a7f-b3d3-c9999d327eeb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '891005fa5ff74041b75747d16cc66798', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80377) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:02:30.425778 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a21268eb-37d2-4a7f-b3d3-c9999d327eeb None admin] REQUEST_METHOD: `GET` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:02:30.426104 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a21268eb-37d2-4a7f-b3d3-c9999d327eeb None admin] SCRIPT_NAME: `/identity` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:02:30.426374 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a21268eb-37d2-4a7f-b3d3-c9999d327eeb None admin] PATH_INFO: `/v3/users/swift` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:02:30.437932 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a21268eb-37d2-4a7f-b3d3-c9999d327eeb None admin] RBAC: Authorizing `identity:get_user(user_id=swift)` {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:02:30.439537 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a21268eb-37d2-4a7f-b3d3-c9999d327eeb None admin] RBAC: Authorization granted {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:02:30.456369 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application [None req-a21268eb-37d2-4a7f-b3d3-c9999d327eeb None admin] Could not find user: swift.: keystone.exception.UserNotFound: Could not find user: swift. Aug 02 10:02:30.456369 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 02 10:02:30.456369 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 02 10:02:30.456369 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 02 10:02:30.456369 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 02 10:02:30.456369 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 02 10:02:30.456369 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 02 10:02:30.456369 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 02 10:02:30.456369 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 02 10:02:30.456369 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 02 10:02:30.456369 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 02 10:02:30.456369 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 02 10:02:30.456369 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 02 10:02:30.456369 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 02 10:02:30.456369 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 02 10:02:30.456369 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 02 10:02:30.458953 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 02 10:02:30.458953 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 02 10:02:30.458953 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 02 10:02:30.458953 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 02 10:02:30.458953 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 02 10:02:30.458953 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 02 10:02:30.458953 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 02 10:02:30.458953 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 02 10:02:30.458953 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 02 10:02:30.458953 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application return self.get_or_create( Aug 02 10:02:30.458953 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 02 10:02:30.458953 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application with Lock( Aug 02 10:02:30.458953 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 02 10:02:30.458953 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application return self._enter() Aug 02 10:02:30.458953 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 02 10:02:30.458953 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 02 10:02:30.458953 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 02 10:02:30.458953 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application return self.creator() Aug 02 10:02:30.461008 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 02 10:02:30.461008 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application created_value = creator( Aug 02 10:02:30.461008 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 02 10:02:30.461008 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 02 10:02:30.461008 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 02 10:02:30.461008 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 02 10:02:30.461008 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 02 10:02:30.461008 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 02 10:02:30.461008 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: swift. Aug 02 10:02:30.461008 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application Aug 02 10:02:30.463134 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: [pid: 80377|app: 0|req: 311/618] 10.222.0.82 () {56 vars in 1096 bytes} [Tue Aug 2 10:02:30 2022] GET /identity/v3/users/swift => generated 83 bytes in 235 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 02 10:02:30.465034 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a21268eb-37d2-4a7f-b3d3-c9999d327eeb None admin] Authenticating user token {{(pid=80377) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:02:30.475525 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb8246a3-142e-44bb-8d42-acebc363c810 None admin] Validating token access rules against request {{(pid=80378) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:02:30.550719 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99da246b-4b8e-4244-a09e-b2721be957fc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '891005fa5ff74041b75747d16cc66798', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80378) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:02:30.552191 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99da246b-4b8e-4244-a09e-b2721be957fc None admin] REQUEST_METHOD: `GET` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:02:30.552935 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99da246b-4b8e-4244-a09e-b2721be957fc None admin] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:02:30.553600 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99da246b-4b8e-4244-a09e-b2721be957fc None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:02:30.571175 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG dbcounter [-] [80377] Writing DB stats keystone:SELECT=100 {{(pid=80377) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 10:02:30.575270 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99da246b-4b8e-4244-a09e-b2721be957fc None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:02:30.576757 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99da246b-4b8e-4244-a09e-b2721be957fc None admin] RBAC: Authorization granted {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:02:30.614375 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application [None req-99da246b-4b8e-4244-a09e-b2721be957fc None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 02 10:02:30.614375 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 02 10:02:30.614375 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 02 10:02:30.614375 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 02 10:02:30.614375 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 02 10:02:30.614375 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 02 10:02:30.614375 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 02 10:02:30.614375 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 02 10:02:30.614375 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 02 10:02:30.614375 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 02 10:02:30.614375 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 02 10:02:30.614375 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 02 10:02:30.614375 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 02 10:02:30.614375 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 02 10:02:30.614375 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 02 10:02:30.616177 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 02 10:02:30.616177 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 02 10:02:30.616177 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 02 10:02:30.616177 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 02 10:02:30.616177 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 02 10:02:30.616177 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 02 10:02:30.616177 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application return self.get_or_create( Aug 02 10:02:30.616177 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 02 10:02:30.616177 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application with Lock( Aug 02 10:02:30.616177 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 02 10:02:30.616177 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application return self._enter() Aug 02 10:02:30.616177 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 02 10:02:30.616177 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 02 10:02:30.616177 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 02 10:02:30.616177 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application return self.creator() Aug 02 10:02:30.616177 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 02 10:02:30.616177 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application created_value = creator( Aug 02 10:02:30.617269 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 02 10:02:30.617269 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 02 10:02:30.617269 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 02 10:02:30.617269 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application Aug 02 10:02:30.617269 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 308/619] 10.222.0.82 () {56 vars in 1108 bytes} [Tue Aug 2 10:02:30 2022] GET /identity/v3/domains/Default => generated 87 bytes in 363 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 02 10:02:30.624255 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99da246b-4b8e-4244-a09e-b2721be957fc None admin] Authenticating user token {{(pid=80378) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:02:30.641186 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a21268eb-37d2-4a7f-b3d3-c9999d327eeb None admin] Validating token access rules against request {{(pid=80377) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:02:30.747648 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6581676b-5c78-4a0d-b1a6-a36f91f7e6b5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '891005fa5ff74041b75747d16cc66798', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80377) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:02:30.748291 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6581676b-5c78-4a0d-b1a6-a36f91f7e6b5 None admin] REQUEST_METHOD: `GET` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:02:30.748576 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6581676b-5c78-4a0d-b1a6-a36f91f7e6b5 None admin] SCRIPT_NAME: `/identity` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:02:30.748851 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6581676b-5c78-4a0d-b1a6-a36f91f7e6b5 None admin] PATH_INFO: `/v3/users` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:02:30.749669 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6581676b-5c78-4a0d-b1a6-a36f91f7e6b5 None admin] RBAC: Adding query filter params (domain_id=default, name=swift) {{(pid=80377) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 02 10:02:30.749997 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6581676b-5c78-4a0d-b1a6-a36f91f7e6b5 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:02:30.751522 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6581676b-5c78-4a0d-b1a6-a36f91f7e6b5 None admin] RBAC: Authorization granted {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:02:30.799706 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: [pid: 80377|app: 0|req: 312/620] 10.222.0.82 () {56 vars in 1135 bytes} [Tue Aug 2 10:02:30 2022] GET /identity/v3/users?domain_id=default&name=swift => generated 370 bytes in 337 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 02 10:02:30.811925 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6581676b-5c78-4a0d-b1a6-a36f91f7e6b5 None admin] Authenticating user token {{(pid=80377) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:02:30.883415 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99da246b-4b8e-4244-a09e-b2721be957fc None admin] Validating token access rules against request {{(pid=80378) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:02:30.936132 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG dbcounter [-] [80378] Writing DB stats keystone:SELECT=100 {{(pid=80378) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 10:02:30.987803 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25c45ea1-c1f4-4509-a777-7ce01b5301f6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '891005fa5ff74041b75747d16cc66798', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80378) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:02:30.988896 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25c45ea1-c1f4-4509-a777-7ce01b5301f6 None admin] REQUEST_METHOD: `GET` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:02:30.989538 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25c45ea1-c1f4-4509-a777-7ce01b5301f6 None admin] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:02:30.990066 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25c45ea1-c1f4-4509-a777-7ce01b5301f6 None admin] PATH_INFO: `/v3/domains` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:02:30.991078 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25c45ea1-c1f4-4509-a777-7ce01b5301f6 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80378) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 02 10:02:30.991884 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25c45ea1-c1f4-4509-a777-7ce01b5301f6 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:02:30.993771 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25c45ea1-c1f4-4509-a777-7ce01b5301f6 None admin] RBAC: Authorization granted {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:02:31.011720 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6581676b-5c78-4a0d-b1a6-a36f91f7e6b5 None admin] Validating token access rules against request {{(pid=80377) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:02:31.028691 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 309/621] 10.222.0.82 () {56 vars in 1109 bytes} [Tue Aug 2 10:02:30 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 405 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 02 10:02:31.035484 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25c45ea1-c1f4-4509-a777-7ce01b5301f6 None admin] Authenticating user token {{(pid=80378) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:02:31.050552 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG dbcounter [-] [80377] Writing DB stats keystone:SELECT=100 {{(pid=80377) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 10:02:31.060024 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc13325f-6607-4d8a-9cb0-536c042c12c7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '891005fa5ff74041b75747d16cc66798', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80377) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:02:31.061287 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc13325f-6607-4d8a-9cb0-536c042c12c7 None admin] REQUEST_METHOD: `GET` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:02:31.062047 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc13325f-6607-4d8a-9cb0-536c042c12c7 None admin] SCRIPT_NAME: `/identity` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:02:31.063214 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc13325f-6607-4d8a-9cb0-536c042c12c7 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:02:31.082991 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc13325f-6607-4d8a-9cb0-536c042c12c7 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:02:31.085812 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc13325f-6607-4d8a-9cb0-536c042c12c7 None admin] RBAC: Authorization granted {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:02:31.107004 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application [None req-fc13325f-6607-4d8a-9cb0-536c042c12c7 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 02 10:02:31.107004 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 02 10:02:31.107004 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 02 10:02:31.107004 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 02 10:02:31.107004 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 02 10:02:31.107004 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 02 10:02:31.107004 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 02 10:02:31.107004 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 02 10:02:31.107004 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 02 10:02:31.107004 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 02 10:02:31.107004 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 02 10:02:31.107004 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 02 10:02:31.107004 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 02 10:02:31.107004 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 02 10:02:31.107004 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 02 10:02:31.108357 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 02 10:02:31.108357 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 02 10:02:31.108357 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 02 10:02:31.108357 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 02 10:02:31.108357 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 02 10:02:31.108357 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 02 10:02:31.108357 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application return self.get_or_create( Aug 02 10:02:31.108357 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 02 10:02:31.108357 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application with Lock( Aug 02 10:02:31.108357 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 02 10:02:31.108357 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application return self._enter() Aug 02 10:02:31.108357 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 02 10:02:31.108357 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 02 10:02:31.108357 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 02 10:02:31.108357 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application return self.creator() Aug 02 10:02:31.108357 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 02 10:02:31.108357 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application created_value = creator( Aug 02 10:02:31.109440 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 02 10:02:31.109440 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 02 10:02:31.109440 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 02 10:02:31.109440 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application Aug 02 10:02:31.112000 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: [pid: 80377|app: 0|req: 313/622] 10.222.0.82 () {56 vars in 1108 bytes} [Tue Aug 2 10:02:30 2022] GET /identity/v3/domains/Default => generated 87 bytes in 301 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 02 10:02:31.117814 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-594aea17-bfe9-4571-8bfa-5f8169c37ec7 None None] REQUEST_METHOD: `GET` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:02:31.117814 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-594aea17-bfe9-4571-8bfa-5f8169c37ec7 None None] SCRIPT_NAME: `/identity` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:02:31.117814 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-594aea17-bfe9-4571-8bfa-5f8169c37ec7 None None] PATH_INFO: `/` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:02:31.117814 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: [pid: 80377|app: 0|req: 314/623] 10.222.0.82 () {54 vars in 926 bytes} [Tue Aug 2 10:02:31 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 02 10:02:31.120073 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-594aea17-bfe9-4571-8bfa-5f8169c37ec7 None None] Authenticating user token {{(pid=80377) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:02:31.221498 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25c45ea1-c1f4-4509-a777-7ce01b5301f6 None admin] Validating token access rules against request {{(pid=80378) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:02:31.264130 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22d53edf-faca-4058-bab3-444e2c65f1d7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '891005fa5ff74041b75747d16cc66798', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80378) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:02:31.264700 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22d53edf-faca-4058-bab3-444e2c65f1d7 None admin] REQUEST_METHOD: `GET` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:02:31.264977 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22d53edf-faca-4058-bab3-444e2c65f1d7 None admin] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:02:31.265243 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22d53edf-faca-4058-bab3-444e2c65f1d7 None admin] PATH_INFO: `/v3/users/nova` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:02:31.279016 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22d53edf-faca-4058-bab3-444e2c65f1d7 None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:02:31.281224 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22d53edf-faca-4058-bab3-444e2c65f1d7 None admin] RBAC: Authorization granted {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:02:31.296464 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application [None req-22d53edf-faca-4058-bab3-444e2c65f1d7 None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Aug 02 10:02:31.296464 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 02 10:02:31.296464 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 02 10:02:31.296464 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 02 10:02:31.296464 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 02 10:02:31.296464 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 02 10:02:31.296464 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 02 10:02:31.296464 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 02 10:02:31.296464 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 02 10:02:31.296464 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 02 10:02:31.296464 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 02 10:02:31.296464 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 02 10:02:31.296464 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 02 10:02:31.296464 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 02 10:02:31.296464 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 02 10:02:31.296464 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 02 10:02:31.297510 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 02 10:02:31.297510 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 02 10:02:31.297510 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 02 10:02:31.297510 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 02 10:02:31.297510 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 02 10:02:31.297510 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 02 10:02:31.297510 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 02 10:02:31.297510 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 02 10:02:31.297510 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 02 10:02:31.297510 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application return self.get_or_create( Aug 02 10:02:31.297510 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 02 10:02:31.297510 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application with Lock( Aug 02 10:02:31.297510 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 02 10:02:31.297510 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application return self._enter() Aug 02 10:02:31.297510 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 02 10:02:31.297510 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 02 10:02:31.297510 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 02 10:02:31.297510 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application return self.creator() Aug 02 10:02:31.298793 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 02 10:02:31.298793 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application created_value = creator( Aug 02 10:02:31.298793 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 02 10:02:31.298793 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 02 10:02:31.298793 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 02 10:02:31.298793 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 02 10:02:31.298793 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 02 10:02:31.298793 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 02 10:02:31.298793 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Aug 02 10:02:31.298793 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application Aug 02 10:02:31.307052 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 310/624] 10.222.0.82 () {56 vars in 1093 bytes} [Tue Aug 2 10:02:31 2022] GET /identity/v3/users/nova => generated 82 bytes in 268 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 02 10:02:31.308918 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1828039f-602a-44ae-b2b9-5c68fdf44e0a None None] REQUEST_METHOD: `POST` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:02:31.309438 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1828039f-602a-44ae-b2b9-5c68fdf44e0a None None] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:02:31.309986 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1828039f-602a-44ae-b2b9-5c68fdf44e0a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:02:31.322387 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-594aea17-bfe9-4571-8bfa-5f8169c37ec7 None None] Validating token access rules against request {{(pid=80377) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:02:31.335061 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.auth.core [None req-1828039f-602a-44ae-b2b9-5c68fdf44e0a None None] MFA Rules not processed for user `891005fa5ff74041b75747d16cc66798`. Rule list: `[]` (Enabled: `True`). {{(pid=80378) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 02 10:02:31.336662 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.fernet_utils [None req-1828039f-602a-44ae-b2b9-5c68fdf44e0a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80378) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 10:02:31.370150 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa5c50bf-7e52-4067-ad32-453b5b02e532 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '891005fa5ff74041b75747d16cc66798', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80377) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:02:31.371228 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa5c50bf-7e52-4067-ad32-453b5b02e532 None admin] REQUEST_METHOD: `GET` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:02:31.371529 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa5c50bf-7e52-4067-ad32-453b5b02e532 None admin] SCRIPT_NAME: `/identity` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:02:31.371810 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa5c50bf-7e52-4067-ad32-453b5b02e532 None admin] PATH_INFO: `/v3/domains` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:02:31.372662 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa5c50bf-7e52-4067-ad32-453b5b02e532 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80377) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 02 10:02:31.373102 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa5c50bf-7e52-4067-ad32-453b5b02e532 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:02:31.374808 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa5c50bf-7e52-4067-ad32-453b5b02e532 None admin] RBAC: Authorization granted {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:02:31.382925 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG dbcounter [-] [80378] Writing DB stats keystone:SELECT=100 {{(pid=80378) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 10:02:31.409550 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: [pid: 80377|app: 0|req: 315/625] 10.222.0.82 () {56 vars in 1109 bytes} [Tue Aug 2 10:02:31 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 286 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 02 10:02:31.409550 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa5c50bf-7e52-4067-ad32-453b5b02e532 None admin] Authenticating user token {{(pid=80377) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:02:31.444376 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 311/626] 10.222.0.82 () {58 vars in 1023 bytes} [Tue Aug 2 10:02:31 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 136 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 02 10:02:31.445880 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1828039f-602a-44ae-b2b9-5c68fdf44e0a None None] Authenticating user token {{(pid=80378) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:02:31.462701 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG dbcounter [-] [80377] Writing DB stats keystone:SELECT=100 {{(pid=80377) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 10:02:31.536043 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa5c50bf-7e52-4067-ad32-453b5b02e532 None admin] Validating token access rules against request {{(pid=80377) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:02:31.570922 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1828039f-602a-44ae-b2b9-5c68fdf44e0a None None] Validating token access rules against request {{(pid=80378) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:02:31.577233 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0406ba9c-0c9a-483d-ba1e-943364abafb9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '891005fa5ff74041b75747d16cc66798', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80377) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:02:31.578284 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0406ba9c-0c9a-483d-ba1e-943364abafb9 None admin] REQUEST_METHOD: `GET` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:02:31.578831 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0406ba9c-0c9a-483d-ba1e-943364abafb9 None admin] SCRIPT_NAME: `/identity` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:02:31.579973 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0406ba9c-0c9a-483d-ba1e-943364abafb9 None admin] PATH_INFO: `/v3/users` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:02:31.581005 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0406ba9c-0c9a-483d-ba1e-943364abafb9 None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=80377) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 02 10:02:31.581531 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0406ba9c-0c9a-483d-ba1e-943364abafb9 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:02:31.583249 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0406ba9c-0c9a-483d-ba1e-943364abafb9 None admin] RBAC: Authorization granted {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:02:31.609187 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: [pid: 80377|app: 0|req: 316/627] 10.222.0.82 () {56 vars in 1133 bytes} [Tue Aug 2 10:02:31 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 198 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 02 10:02:31.610778 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0406ba9c-0c9a-483d-ba1e-943364abafb9 None admin] Authenticating user token {{(pid=80377) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:02:31.614235 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11474f4c-683f-4324-8f05-9b976827d129 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '891005fa5ff74041b75747d16cc66798', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80378) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:02:31.615132 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11474f4c-683f-4324-8f05-9b976827d129 None admin] REQUEST_METHOD: `GET` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:02:31.615250 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11474f4c-683f-4324-8f05-9b976827d129 None admin] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:02:31.615546 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11474f4c-683f-4324-8f05-9b976827d129 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:02:31.623850 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11474f4c-683f-4324-8f05-9b976827d129 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:02:31.625366 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11474f4c-683f-4324-8f05-9b976827d129 None admin] RBAC: Authorization granted {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:02:31.638967 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application [None req-11474f4c-683f-4324-8f05-9b976827d129 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 02 10:02:31.638967 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 02 10:02:31.638967 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 02 10:02:31.638967 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 02 10:02:31.638967 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 02 10:02:31.638967 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 02 10:02:31.638967 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 02 10:02:31.638967 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 02 10:02:31.638967 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 02 10:02:31.638967 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 02 10:02:31.638967 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 02 10:02:31.638967 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 02 10:02:31.638967 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 02 10:02:31.638967 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 02 10:02:31.638967 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 02 10:02:31.639954 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 02 10:02:31.639954 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 02 10:02:31.639954 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 02 10:02:31.639954 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 02 10:02:31.639954 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 02 10:02:31.639954 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 02 10:02:31.639954 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application return self.get_or_create( Aug 02 10:02:31.639954 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 02 10:02:31.639954 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application with Lock( Aug 02 10:02:31.639954 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 02 10:02:31.639954 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application return self._enter() Aug 02 10:02:31.639954 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 02 10:02:31.639954 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 02 10:02:31.639954 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 02 10:02:31.639954 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application return self.creator() Aug 02 10:02:31.639954 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 02 10:02:31.639954 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application created_value = creator( Aug 02 10:02:31.640952 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 02 10:02:31.640952 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 02 10:02:31.640952 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 02 10:02:31.640952 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 02 10:02:31.640952 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 02 10:02:31.640952 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 02 10:02:31.640952 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 02 10:02:31.640952 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application Aug 02 10:02:31.645990 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 312/628] 10.222.0.82 () {56 vars in 1111 bytes} [Tue Aug 2 10:02:31 2022] GET /identity/v3/projects/service => generated 88 bytes in 195 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 02 10:02:31.647502 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11474f4c-683f-4324-8f05-9b976827d129 None admin] Authenticating user token {{(pid=80378) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:02:31.683965 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG dbcounter [-] [80378] Writing DB stats keystone:SELECT=100 {{(pid=80378) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 10:02:31.738760 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG dbcounter [-] [80377] Writing DB stats keystone:SELECT=100 {{(pid=80377) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 10:02:31.745647 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0406ba9c-0c9a-483d-ba1e-943364abafb9 None admin] Validating token access rules against request {{(pid=80377) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:02:31.774752 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11474f4c-683f-4324-8f05-9b976827d129 None admin] Validating token access rules against request {{(pid=80378) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:02:31.788858 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fb0c58e-c917-47f1-a5a5-ff3a45af8286 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '891005fa5ff74041b75747d16cc66798', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80377) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:02:31.789927 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fb0c58e-c917-47f1-a5a5-ff3a45af8286 None admin] REQUEST_METHOD: `GET` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:02:31.790428 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fb0c58e-c917-47f1-a5a5-ff3a45af8286 None admin] SCRIPT_NAME: `/identity` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:02:31.790917 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fb0c58e-c917-47f1-a5a5-ff3a45af8286 None admin] PATH_INFO: `/v3/roles/service` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:02:31.806432 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6fb0c58e-c917-47f1-a5a5-ff3a45af8286 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 02 10:02:31.812221 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application [None req-6fb0c58e-c917-47f1-a5a5-ff3a45af8286 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Aug 02 10:02:31.812221 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 02 10:02:31.812221 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 02 10:02:31.812221 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 02 10:02:31.812221 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 02 10:02:31.812221 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 02 10:02:31.812221 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 02 10:02:31.812221 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 02 10:02:31.812221 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 02 10:02:31.812221 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 02 10:02:31.812221 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 02 10:02:31.812221 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 02 10:02:31.812221 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 02 10:02:31.812221 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 02 10:02:31.812221 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 02 10:02:31.812221 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 02 10:02:31.813262 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 02 10:02:31.813262 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 02 10:02:31.813262 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 02 10:02:31.813262 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 02 10:02:31.813262 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 02 10:02:31.813262 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 02 10:02:31.813262 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 02 10:02:31.813262 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 02 10:02:31.813262 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 02 10:02:31.813262 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application return self.get_or_create( Aug 02 10:02:31.813262 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 02 10:02:31.813262 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application with Lock( Aug 02 10:02:31.813262 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 02 10:02:31.813262 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application return self._enter() Aug 02 10:02:31.813262 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 02 10:02:31.813262 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 02 10:02:31.814503 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 02 10:02:31.814503 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application return self.creator() Aug 02 10:02:31.814503 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 02 10:02:31.814503 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application created_value = creator( Aug 02 10:02:31.814503 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 02 10:02:31.814503 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 02 10:02:31.814503 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 02 10:02:31.814503 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 02 10:02:31.814503 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 02 10:02:31.814503 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 02 10:02:31.814503 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Aug 02 10:02:31.814503 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application Aug 02 10:02:31.818124 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: [pid: 80377|app: 0|req: 317/629] 10.222.0.82 () {56 vars in 1102 bytes} [Tue Aug 2 10:02:31 2022] GET /identity/v3/roles/service => generated 85 bytes in 208 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 02 10:02:31.819403 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fb0c58e-c917-47f1-a5a5-ff3a45af8286 None admin] Authenticating user token {{(pid=80377) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:02:31.838356 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d51ea813-7687-4579-bd8a-996383129b94 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '891005fa5ff74041b75747d16cc66798', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80378) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:02:31.838998 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d51ea813-7687-4579-bd8a-996383129b94 None admin] REQUEST_METHOD: `GET` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:02:31.839284 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d51ea813-7687-4579-bd8a-996383129b94 None admin] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:02:31.839565 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d51ea813-7687-4579-bd8a-996383129b94 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:02:31.939859 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d51ea813-7687-4579-bd8a-996383129b94 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:02:31.941581 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d51ea813-7687-4579-bd8a-996383129b94 None admin] RBAC: Authorization granted {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:02:31.966279 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fb0c58e-c917-47f1-a5a5-ff3a45af8286 None admin] Validating token access rules against request {{(pid=80377) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:02:31.981493 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG dbcounter [-] [80378] Writing DB stats keystone:SELECT=100 {{(pid=80378) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 10:02:32.015744 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05731d6e-7d2f-47ff-bb94-2ebea0e594d9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '891005fa5ff74041b75747d16cc66798', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80377) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:02:32.017621 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05731d6e-7d2f-47ff-bb94-2ebea0e594d9 None admin] REQUEST_METHOD: `GET` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:02:32.018765 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05731d6e-7d2f-47ff-bb94-2ebea0e594d9 None admin] SCRIPT_NAME: `/identity` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:02:32.019441 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05731d6e-7d2f-47ff-bb94-2ebea0e594d9 None admin] PATH_INFO: `/v3/projects` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:02:32.020545 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05731d6e-7d2f-47ff-bb94-2ebea0e594d9 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80377) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 02 10:02:32.021081 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05731d6e-7d2f-47ff-bb94-2ebea0e594d9 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:02:32.022981 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05731d6e-7d2f-47ff-bb94-2ebea0e594d9 None admin] RBAC: Authorization granted {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:02:32.038402 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: [pid: 80377|app: 0|req: 318/630] 10.222.0.82 () {56 vars in 1148 bytes} [Tue Aug 2 10:02:31 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 219 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 02 10:02:32.039835 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05731d6e-7d2f-47ff-bb94-2ebea0e594d9 None admin] Authenticating user token {{(pid=80377) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:02:32.068802 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG dbcounter [-] [80377] Writing DB stats keystone:SELECT=100 {{(pid=80377) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 10:02:32.089549 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 313/631] 10.222.0.82 () {58 vars in 1282 bytes} [Tue Aug 2 10:02:31 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 443 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 02 10:02:32.090946 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32af2d4a-5673-4db8-b066-4962901f507b None None] REQUEST_METHOD: `GET` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:02:32.091235 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32af2d4a-5673-4db8-b066-4962901f507b None None] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:02:32.091510 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32af2d4a-5673-4db8-b066-4962901f507b None None] PATH_INFO: `/` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:02:32.093033 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 314/632] 10.222.0.82 () {54 vars in 926 bytes} [Tue Aug 2 10:02:32 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 02 10:02:32.098339 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32af2d4a-5673-4db8-b066-4962901f507b None None] Authenticating user token {{(pid=80378) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:02:32.195432 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05731d6e-7d2f-47ff-bb94-2ebea0e594d9 None admin] Validating token access rules against request {{(pid=80377) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:02:32.238261 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32af2d4a-5673-4db8-b066-4962901f507b None None] Validating token access rules against request {{(pid=80378) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:02:32.246456 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b93c2841-e8d9-4016-8b15-be3937fc35f4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '891005fa5ff74041b75747d16cc66798', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80377) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:02:32.247526 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b93c2841-e8d9-4016-8b15-be3937fc35f4 None admin] REQUEST_METHOD: `GET` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:02:32.248008 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b93c2841-e8d9-4016-8b15-be3937fc35f4 None admin] SCRIPT_NAME: `/identity` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:02:32.248473 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b93c2841-e8d9-4016-8b15-be3937fc35f4 None admin] PATH_INFO: `/v3/roles` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:02:32.249382 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b93c2841-e8d9-4016-8b15-be3937fc35f4 None admin] RBAC: Adding query filter params (name=service) {{(pid=80377) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 02 10:02:32.249908 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b93c2841-e8d9-4016-8b15-be3937fc35f4 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:02:32.252145 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b93c2841-e8d9-4016-8b15-be3937fc35f4 None admin] RBAC: Authorization granted {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:02:32.270351 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: [pid: 80377|app: 0|req: 319/633] 10.222.0.82 () {56 vars in 1103 bytes} [Tue Aug 2 10:02:32 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 231 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 02 10:02:32.272208 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b93c2841-e8d9-4016-8b15-be3937fc35f4 None admin] Authenticating user token {{(pid=80377) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:02:32.284689 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG dbcounter [-] [80378] Writing DB stats keystone:SELECT=100 {{(pid=80378) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 10:02:32.295840 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5a6327b-9be7-4eca-80b7-598c5420910c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '891005fa5ff74041b75747d16cc66798', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80378) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:02:32.296715 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5a6327b-9be7-4eca-80b7-598c5420910c None admin] REQUEST_METHOD: `PUT` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:02:32.297113 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5a6327b-9be7-4eca-80b7-598c5420910c None admin] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:02:32.297269 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5a6327b-9be7-4eca-80b7-598c5420910c None admin] PATH_INFO: `/v3/projects/f1b38ebb804c41c69772c998aae72c41/users/bef1b2e714cb45c4a51eb6d3e15aad6d/roles/02003a0be60f47f2a8dfea17053af52c` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:02:32.319860 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5a6327b-9be7-4eca-80b7-598c5420910c None admin] RBAC: Authorizing `identity:create_grant(project_id=f1b38ebb804c41c69772c998aae72c41, user_id=bef1b2e714cb45c4a51eb6d3e15aad6d, role_id=02003a0be60f47f2a8dfea17053af52c)` {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:02:32.322394 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5a6327b-9be7-4eca-80b7-598c5420910c None admin] RBAC: Authorization granted {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:02:32.342837 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 315/634] 10.222.0.82 () {58 vars in 1439 bytes} [Tue Aug 2 10:02:32 2022] PUT /identity/v3/projects/f1b38ebb804c41c69772c998aae72c41/users/bef1b2e714cb45c4a51eb6d3e15aad6d/roles/02003a0be60f47f2a8dfea17053af52c => generated 0 bytes in 249 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 02 10:02:32.346104 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e4db814-f6ac-4cbc-8c75-456916e2b412 None None] REQUEST_METHOD: `POST` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:02:32.347069 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e4db814-f6ac-4cbc-8c75-456916e2b412 None None] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:02:32.347793 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e4db814-f6ac-4cbc-8c75-456916e2b412 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:02:32.381684 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.auth.core [None req-8e4db814-f6ac-4cbc-8c75-456916e2b412 None None] MFA Rules not processed for user `891005fa5ff74041b75747d16cc66798`. Rule list: `[]` (Enabled: `True`). {{(pid=80378) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 02 10:02:32.384009 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.fernet_utils [None req-8e4db814-f6ac-4cbc-8c75-456916e2b412 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80378) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 10:02:32.433238 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG dbcounter [-] [80377] Writing DB stats keystone:SELECT=100 {{(pid=80377) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 10:02:32.480179 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b93c2841-e8d9-4016-8b15-be3937fc35f4 None admin] Validating token access rules against request {{(pid=80377) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:02:32.492918 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 316/635] 10.222.0.82 () {58 vars in 1023 bytes} [Tue Aug 2 10:02:32 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 149 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 02 10:02:32.494288 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e4db814-f6ac-4cbc-8c75-456916e2b412 None None] Authenticating user token {{(pid=80378) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:02:32.560326 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9fc41d6-db50-40ec-aebd-ffa5614fc304 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '891005fa5ff74041b75747d16cc66798', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80377) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:02:32.561411 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9fc41d6-db50-40ec-aebd-ffa5614fc304 None admin] REQUEST_METHOD: `GET` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:02:32.561991 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9fc41d6-db50-40ec-aebd-ffa5614fc304 None admin] SCRIPT_NAME: `/identity` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:02:32.562770 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9fc41d6-db50-40ec-aebd-ffa5614fc304 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:02:32.585489 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9fc41d6-db50-40ec-aebd-ffa5614fc304 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:02:32.588119 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9fc41d6-db50-40ec-aebd-ffa5614fc304 None admin] RBAC: Authorization granted {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:02:32.617709 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application [None req-b9fc41d6-db50-40ec-aebd-ffa5614fc304 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 02 10:02:32.617709 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 02 10:02:32.617709 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 02 10:02:32.617709 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 02 10:02:32.617709 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 02 10:02:32.617709 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 02 10:02:32.617709 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 02 10:02:32.617709 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 02 10:02:32.617709 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 02 10:02:32.617709 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 02 10:02:32.617709 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 02 10:02:32.617709 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 02 10:02:32.617709 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 02 10:02:32.617709 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 02 10:02:32.617709 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 02 10:02:32.619331 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 02 10:02:32.619331 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 02 10:02:32.619331 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 02 10:02:32.619331 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 02 10:02:32.619331 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 02 10:02:32.619331 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 02 10:02:32.619331 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application return self.get_or_create( Aug 02 10:02:32.619331 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 02 10:02:32.619331 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application with Lock( Aug 02 10:02:32.619331 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 02 10:02:32.619331 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application return self._enter() Aug 02 10:02:32.619331 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 02 10:02:32.619331 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 02 10:02:32.619331 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 02 10:02:32.619331 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application return self.creator() Aug 02 10:02:32.619331 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 02 10:02:32.619331 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application created_value = creator( Aug 02 10:02:32.620586 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 02 10:02:32.620586 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 02 10:02:32.620586 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 02 10:02:32.620586 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application Aug 02 10:02:32.620586 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: [pid: 80377|app: 0|req: 320/636] 10.222.0.82 () {56 vars in 1108 bytes} [Tue Aug 2 10:02:32 2022] GET /identity/v3/domains/Default => generated 87 bytes in 348 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 02 10:02:32.620586 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9fc41d6-db50-40ec-aebd-ffa5614fc304 None admin] Authenticating user token {{(pid=80377) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:02:32.668429 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e4db814-f6ac-4cbc-8c75-456916e2b412 None None] Validating token access rules against request {{(pid=80378) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:02:32.695095 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG dbcounter [-] [80378] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80378) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 10:02:32.742045 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07fda3a5-e2da-4f5a-b5e3-2570ae7d36e2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '891005fa5ff74041b75747d16cc66798', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80378) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:02:32.742933 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07fda3a5-e2da-4f5a-b5e3-2570ae7d36e2 None admin] REQUEST_METHOD: `GET` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:02:32.743317 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07fda3a5-e2da-4f5a-b5e3-2570ae7d36e2 None admin] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:02:32.743703 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07fda3a5-e2da-4f5a-b5e3-2570ae7d36e2 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:02:32.761464 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07fda3a5-e2da-4f5a-b5e3-2570ae7d36e2 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:02:32.763913 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07fda3a5-e2da-4f5a-b5e3-2570ae7d36e2 None admin] RBAC: Authorization granted {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:02:32.771291 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9fc41d6-db50-40ec-aebd-ffa5614fc304 None admin] Validating token access rules against request {{(pid=80377) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:02:32.788561 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application [None req-07fda3a5-e2da-4f5a-b5e3-2570ae7d36e2 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 02 10:02:32.788561 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 02 10:02:32.788561 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 02 10:02:32.788561 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 02 10:02:32.788561 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 02 10:02:32.788561 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 02 10:02:32.788561 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 02 10:02:32.788561 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 02 10:02:32.788561 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 02 10:02:32.788561 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 02 10:02:32.788561 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 02 10:02:32.788561 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 02 10:02:32.788561 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 02 10:02:32.788561 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 02 10:02:32.788561 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 02 10:02:32.790549 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 02 10:02:32.790549 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 02 10:02:32.790549 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 02 10:02:32.790549 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 02 10:02:32.790549 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 02 10:02:32.790549 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 02 10:02:32.790549 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application return self.get_or_create( Aug 02 10:02:32.790549 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 02 10:02:32.790549 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application with Lock( Aug 02 10:02:32.790549 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 02 10:02:32.790549 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application return self._enter() Aug 02 10:02:32.790549 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 02 10:02:32.790549 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 02 10:02:32.790549 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 02 10:02:32.790549 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application return self.creator() Aug 02 10:02:32.790549 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 02 10:02:32.790549 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application created_value = creator( Aug 02 10:02:32.792752 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 02 10:02:32.792752 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 02 10:02:32.792752 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 02 10:02:32.792752 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application Aug 02 10:02:32.797114 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 317/637] 10.222.0.82 () {56 vars in 1108 bytes} [Tue Aug 2 10:02:32 2022] GET /identity/v3/domains/Default => generated 87 bytes in 297 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 02 10:02:32.797114 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07fda3a5-e2da-4f5a-b5e3-2570ae7d36e2 None admin] Authenticating user token {{(pid=80378) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:02:32.815300 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG dbcounter [-] [80377] Writing DB stats keystone:SELECT=100 {{(pid=80377) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 10:02:32.821885 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7cfba23-2758-4f1b-b250-20cdf7c2e6a5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '891005fa5ff74041b75747d16cc66798', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80377) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:02:32.822690 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7cfba23-2758-4f1b-b250-20cdf7c2e6a5 None admin] REQUEST_METHOD: `GET` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:02:32.823044 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7cfba23-2758-4f1b-b250-20cdf7c2e6a5 None admin] SCRIPT_NAME: `/identity` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:02:32.823498 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7cfba23-2758-4f1b-b250-20cdf7c2e6a5 None admin] PATH_INFO: `/v3/roles/service` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:02:32.836410 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7cfba23-2758-4f1b-b250-20cdf7c2e6a5 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 02 10:02:32.842472 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application [None req-c7cfba23-2758-4f1b-b250-20cdf7c2e6a5 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Aug 02 10:02:32.842472 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 02 10:02:32.842472 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 02 10:02:32.842472 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 02 10:02:32.842472 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 02 10:02:32.842472 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 02 10:02:32.842472 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 02 10:02:32.842472 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 02 10:02:32.842472 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 02 10:02:32.842472 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 02 10:02:32.842472 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 02 10:02:32.842472 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 02 10:02:32.842472 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 02 10:02:32.842472 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 02 10:02:32.842472 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 02 10:02:32.842472 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 02 10:02:32.844562 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 02 10:02:32.844562 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 02 10:02:32.844562 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 02 10:02:32.844562 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 02 10:02:32.844562 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 02 10:02:32.844562 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 02 10:02:32.844562 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 02 10:02:32.844562 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 02 10:02:32.844562 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 02 10:02:32.844562 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application return self.get_or_create( Aug 02 10:02:32.844562 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 02 10:02:32.844562 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application with Lock( Aug 02 10:02:32.844562 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 02 10:02:32.844562 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application return self._enter() Aug 02 10:02:32.844562 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 02 10:02:32.844562 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 02 10:02:32.847801 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 02 10:02:32.847801 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application return self.creator() Aug 02 10:02:32.847801 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 02 10:02:32.847801 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application created_value = creator( Aug 02 10:02:32.847801 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 02 10:02:32.847801 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 02 10:02:32.847801 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 02 10:02:32.847801 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 02 10:02:32.847801 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 02 10:02:32.847801 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 02 10:02:32.847801 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Aug 02 10:02:32.847801 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application Aug 02 10:02:32.856176 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: [pid: 80377|app: 0|req: 321/638] 10.222.0.82 () {56 vars in 1102 bytes} [Tue Aug 2 10:02:32 2022] GET /identity/v3/roles/service => generated 85 bytes in 231 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 02 10:02:32.856176 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7cfba23-2758-4f1b-b250-20cdf7c2e6a5 None admin] Authenticating user token {{(pid=80377) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:02:32.939785 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07fda3a5-e2da-4f5a-b5e3-2570ae7d36e2 None admin] Validating token access rules against request {{(pid=80378) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:02:32.994647 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b4fd299-68c0-4266-91de-8604fef96660 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '891005fa5ff74041b75747d16cc66798', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80378) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:02:32.995553 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b4fd299-68c0-4266-91de-8604fef96660 None admin] REQUEST_METHOD: `GET` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:02:32.996200 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b4fd299-68c0-4266-91de-8604fef96660 None admin] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:02:32.996772 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b4fd299-68c0-4266-91de-8604fef96660 None admin] PATH_INFO: `/v3/domains` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:02:32.998204 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b4fd299-68c0-4266-91de-8604fef96660 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80378) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 02 10:02:32.998780 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b4fd299-68c0-4266-91de-8604fef96660 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:02:33.000802 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b4fd299-68c0-4266-91de-8604fef96660 None admin] RBAC: Authorization granted {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:02:33.021707 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 318/639] 10.222.0.82 () {56 vars in 1109 bytes} [Tue Aug 2 10:02:32 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 222 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 02 10:02:33.023022 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b4fd299-68c0-4266-91de-8604fef96660 None admin] Authenticating user token {{(pid=80378) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:02:33.035784 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7cfba23-2758-4f1b-b250-20cdf7c2e6a5 None admin] Validating token access rules against request {{(pid=80377) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:02:33.061955 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG dbcounter [-] [80378] Writing DB stats keystone:SELECT=100 {{(pid=80378) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 10:02:33.086141 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ec0598f-1ad3-4e60-8912-6e7464f5ecd2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '891005fa5ff74041b75747d16cc66798', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80377) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:02:33.087141 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ec0598f-1ad3-4e60-8912-6e7464f5ecd2 None admin] REQUEST_METHOD: `GET` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:02:33.087666 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ec0598f-1ad3-4e60-8912-6e7464f5ecd2 None admin] SCRIPT_NAME: `/identity` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:02:33.088117 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ec0598f-1ad3-4e60-8912-6e7464f5ecd2 None admin] PATH_INFO: `/v3/domains` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:02:33.089049 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ec0598f-1ad3-4e60-8912-6e7464f5ecd2 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80377) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 02 10:02:33.089537 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ec0598f-1ad3-4e60-8912-6e7464f5ecd2 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:02:33.091777 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ec0598f-1ad3-4e60-8912-6e7464f5ecd2 None admin] RBAC: Authorization granted {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:02:33.109387 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: [pid: 80377|app: 0|req: 322/640] 10.222.0.82 () {56 vars in 1109 bytes} [Tue Aug 2 10:02:32 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 258 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 02 10:02:33.111271 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ec0598f-1ad3-4e60-8912-6e7464f5ecd2 None admin] Authenticating user token {{(pid=80377) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:02:33.182382 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b4fd299-68c0-4266-91de-8604fef96660 None admin] Validating token access rules against request {{(pid=80378) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:02:33.197067 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG dbcounter [-] [80377] Writing DB stats keystone:SELECT=100 {{(pid=80377) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 10:02:33.244247 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07e2b4ff-e6ae-4541-9857-fd092d948e96 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '891005fa5ff74041b75747d16cc66798', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80378) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:02:33.244887 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07e2b4ff-e6ae-4541-9857-fd092d948e96 None admin] REQUEST_METHOD: `GET` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:02:33.245172 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07e2b4ff-e6ae-4541-9857-fd092d948e96 None admin] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:02:33.245446 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07e2b4ff-e6ae-4541-9857-fd092d948e96 None admin] PATH_INFO: `/v3/roles` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:02:33.246216 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07e2b4ff-e6ae-4541-9857-fd092d948e96 None admin] RBAC: Adding query filter params (name=service) {{(pid=80378) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 02 10:02:33.246612 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07e2b4ff-e6ae-4541-9857-fd092d948e96 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:02:33.248778 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07e2b4ff-e6ae-4541-9857-fd092d948e96 None admin] RBAC: Authorization granted {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:02:33.265081 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 319/641] 10.222.0.82 () {56 vars in 1103 bytes} [Tue Aug 2 10:02:33 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 243 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 02 10:02:33.266739 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07e2b4ff-e6ae-4541-9857-fd092d948e96 None admin] Authenticating user token {{(pid=80378) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:02:33.267219 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ec0598f-1ad3-4e60-8912-6e7464f5ecd2 None admin] Validating token access rules against request {{(pid=80377) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:02:33.334370 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3496e5c3-329d-4476-80a3-0ae186bda10c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '891005fa5ff74041b75747d16cc66798', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80377) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:02:33.335468 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3496e5c3-329d-4476-80a3-0ae186bda10c None admin] REQUEST_METHOD: `GET` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:02:33.335964 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3496e5c3-329d-4476-80a3-0ae186bda10c None admin] SCRIPT_NAME: `/identity` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:02:33.336488 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3496e5c3-329d-4476-80a3-0ae186bda10c None admin] PATH_INFO: `/v3/projects/service` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:02:33.347324 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3496e5c3-329d-4476-80a3-0ae186bda10c None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:02:33.348329 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3496e5c3-329d-4476-80a3-0ae186bda10c None admin] RBAC: Authorization granted {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:02:33.364844 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application [None req-3496e5c3-329d-4476-80a3-0ae186bda10c None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 02 10:02:33.364844 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 02 10:02:33.364844 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 02 10:02:33.364844 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 02 10:02:33.364844 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 02 10:02:33.364844 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 02 10:02:33.364844 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 02 10:02:33.364844 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 02 10:02:33.364844 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 02 10:02:33.364844 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 02 10:02:33.364844 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 02 10:02:33.364844 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 02 10:02:33.364844 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 02 10:02:33.364844 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 02 10:02:33.364844 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 02 10:02:33.366650 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 02 10:02:33.366650 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 02 10:02:33.366650 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 02 10:02:33.366650 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 02 10:02:33.366650 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 02 10:02:33.366650 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 02 10:02:33.366650 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application return self.get_or_create( Aug 02 10:02:33.366650 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 02 10:02:33.366650 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application with Lock( Aug 02 10:02:33.366650 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 02 10:02:33.366650 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application return self._enter() Aug 02 10:02:33.366650 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 02 10:02:33.366650 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 02 10:02:33.366650 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 02 10:02:33.366650 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application return self.creator() Aug 02 10:02:33.366650 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 02 10:02:33.366650 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application created_value = creator( Aug 02 10:02:33.367693 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 02 10:02:33.367693 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 02 10:02:33.367693 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 02 10:02:33.367693 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 02 10:02:33.367693 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 02 10:02:33.367693 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 02 10:02:33.367693 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 02 10:02:33.367693 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application Aug 02 10:02:33.367693 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: [pid: 80377|app: 0|req: 323/642] 10.222.0.82 () {56 vars in 1111 bytes} [Tue Aug 2 10:02:33 2022] GET /identity/v3/projects/service => generated 88 bytes in 256 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 02 10:02:33.369363 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1c2a0a6-7902-429d-a52d-d23a6035a176 None None] REQUEST_METHOD: `GET` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:02:33.369363 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1c2a0a6-7902-429d-a52d-d23a6035a176 None None] SCRIPT_NAME: `/identity` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:02:33.369665 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1c2a0a6-7902-429d-a52d-d23a6035a176 None None] PATH_INFO: `/` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:02:33.376633 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: [pid: 80377|app: 0|req: 324/643] 10.222.0.82 () {54 vars in 926 bytes} [Tue Aug 2 10:02:33 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 02 10:02:33.376633 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1c2a0a6-7902-429d-a52d-d23a6035a176 None None] Authenticating user token {{(pid=80377) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:02:33.407407 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG dbcounter [-] [80378] Writing DB stats keystone:SELECT=100 {{(pid=80378) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 10:02:33.431328 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07e2b4ff-e6ae-4541-9857-fd092d948e96 None admin] Validating token access rules against request {{(pid=80378) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:02:33.483986 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e374d9ee-5c0c-4b3d-8281-eaa55934b388 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '891005fa5ff74041b75747d16cc66798', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80378) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:02:33.484612 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e374d9ee-5c0c-4b3d-8281-eaa55934b388 None admin] REQUEST_METHOD: `GET` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:02:33.485020 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e374d9ee-5c0c-4b3d-8281-eaa55934b388 None admin] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:02:33.485020 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e374d9ee-5c0c-4b3d-8281-eaa55934b388 None admin] PATH_INFO: `/v3/users/cinder` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:02:33.494908 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e374d9ee-5c0c-4b3d-8281-eaa55934b388 None admin] RBAC: Authorizing `identity:get_user(user_id=cinder)` {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:02:33.496277 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e374d9ee-5c0c-4b3d-8281-eaa55934b388 None admin] RBAC: Authorization granted {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:02:33.514569 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application [None req-e374d9ee-5c0c-4b3d-8281-eaa55934b388 None admin] Could not find user: cinder.: keystone.exception.UserNotFound: Could not find user: cinder. Aug 02 10:02:33.514569 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 02 10:02:33.514569 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 02 10:02:33.514569 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 02 10:02:33.514569 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 02 10:02:33.514569 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 02 10:02:33.514569 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 02 10:02:33.514569 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 02 10:02:33.514569 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 02 10:02:33.514569 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 02 10:02:33.514569 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 02 10:02:33.514569 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 02 10:02:33.514569 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 02 10:02:33.514569 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 02 10:02:33.514569 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 02 10:02:33.514569 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 02 10:02:33.516039 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 02 10:02:33.516039 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 02 10:02:33.516039 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 02 10:02:33.516039 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 02 10:02:33.516039 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 02 10:02:33.516039 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 02 10:02:33.516039 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 02 10:02:33.516039 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 02 10:02:33.516039 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 02 10:02:33.516039 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application return self.get_or_create( Aug 02 10:02:33.516039 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 02 10:02:33.516039 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application with Lock( Aug 02 10:02:33.516039 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 02 10:02:33.516039 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application return self._enter() Aug 02 10:02:33.516039 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 02 10:02:33.516039 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 02 10:02:33.516039 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 02 10:02:33.516039 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application return self.creator() Aug 02 10:02:33.520059 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 02 10:02:33.520059 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application created_value = creator( Aug 02 10:02:33.520059 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 02 10:02:33.520059 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 02 10:02:33.520059 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 02 10:02:33.520059 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 02 10:02:33.520059 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 02 10:02:33.520059 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 02 10:02:33.520059 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: cinder. Aug 02 10:02:33.520059 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application Aug 02 10:02:33.520059 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 320/644] 10.222.0.82 () {56 vars in 1099 bytes} [Tue Aug 2 10:02:33 2022] GET /identity/v3/users/cinder => generated 84 bytes in 252 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 02 10:02:33.520059 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e374d9ee-5c0c-4b3d-8281-eaa55934b388 None admin] Authenticating user token {{(pid=80378) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:02:33.558306 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1c2a0a6-7902-429d-a52d-d23a6035a176 None None] Validating token access rules against request {{(pid=80377) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:02:33.579670 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG dbcounter [-] [80377] Writing DB stats keystone:SELECT=100 {{(pid=80377) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 10:02:33.611662 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21b7e7a6-f6c8-4bbc-b818-221fb06d4473 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '891005fa5ff74041b75747d16cc66798', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80377) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:02:33.612290 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21b7e7a6-f6c8-4bbc-b818-221fb06d4473 None admin] REQUEST_METHOD: `GET` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:02:33.613000 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21b7e7a6-f6c8-4bbc-b818-221fb06d4473 None admin] SCRIPT_NAME: `/identity` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:02:33.613498 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21b7e7a6-f6c8-4bbc-b818-221fb06d4473 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:02:33.635651 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21b7e7a6-f6c8-4bbc-b818-221fb06d4473 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:02:33.635651 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21b7e7a6-f6c8-4bbc-b818-221fb06d4473 None admin] RBAC: Authorization granted {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:02:33.651804 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application [None req-21b7e7a6-f6c8-4bbc-b818-221fb06d4473 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 02 10:02:33.651804 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 02 10:02:33.651804 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 02 10:02:33.651804 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 02 10:02:33.651804 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 02 10:02:33.651804 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 02 10:02:33.651804 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 02 10:02:33.651804 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 02 10:02:33.651804 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 02 10:02:33.651804 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 02 10:02:33.651804 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 02 10:02:33.651804 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 02 10:02:33.651804 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 02 10:02:33.651804 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 02 10:02:33.651804 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 02 10:02:33.653510 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 02 10:02:33.653510 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 02 10:02:33.653510 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 02 10:02:33.653510 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 02 10:02:33.653510 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 02 10:02:33.653510 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 02 10:02:33.653510 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application return self.get_or_create( Aug 02 10:02:33.653510 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 02 10:02:33.653510 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application with Lock( Aug 02 10:02:33.653510 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 02 10:02:33.653510 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application return self._enter() Aug 02 10:02:33.653510 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 02 10:02:33.653510 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 02 10:02:33.653510 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 02 10:02:33.653510 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application return self.creator() Aug 02 10:02:33.653510 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 02 10:02:33.653510 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application created_value = creator( Aug 02 10:02:33.655630 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 02 10:02:33.655630 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 02 10:02:33.655630 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 02 10:02:33.655630 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application Aug 02 10:02:33.655630 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: [pid: 80377|app: 0|req: 325/645] 10.222.0.82 () {56 vars in 1108 bytes} [Tue Aug 2 10:02:33 2022] GET /identity/v3/domains/Default => generated 87 bytes in 280 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 02 10:02:33.655630 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ed89884-9c34-4654-9407-cccacb47dfa8 None None] REQUEST_METHOD: `POST` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:02:33.655630 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ed89884-9c34-4654-9407-cccacb47dfa8 None None] SCRIPT_NAME: `/identity` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:02:33.655630 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ed89884-9c34-4654-9407-cccacb47dfa8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:02:33.679863 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.auth.core [None req-4ed89884-9c34-4654-9407-cccacb47dfa8 None None] MFA Rules not processed for user `891005fa5ff74041b75747d16cc66798`. Rule list: `[]` (Enabled: `True`). {{(pid=80377) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 02 10:02:33.681089 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.fernet_utils [None req-4ed89884-9c34-4654-9407-cccacb47dfa8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80377) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 10:02:33.695882 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e374d9ee-5c0c-4b3d-8281-eaa55934b388 None admin] Validating token access rules against request {{(pid=80378) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:02:33.750267 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70d764fa-4283-44a5-9d94-e805d2886f6f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '891005fa5ff74041b75747d16cc66798', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80378) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:02:33.750894 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70d764fa-4283-44a5-9d94-e805d2886f6f None admin] REQUEST_METHOD: `GET` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:02:33.751773 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70d764fa-4283-44a5-9d94-e805d2886f6f None admin] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:02:33.752015 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70d764fa-4283-44a5-9d94-e805d2886f6f None admin] PATH_INFO: `/v3/projects` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:02:33.753192 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70d764fa-4283-44a5-9d94-e805d2886f6f None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80378) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 02 10:02:33.753722 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70d764fa-4283-44a5-9d94-e805d2886f6f None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:02:33.756201 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70d764fa-4283-44a5-9d94-e805d2886f6f None admin] RBAC: Authorization granted {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:02:33.785878 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 321/646] 10.222.0.82 () {56 vars in 1148 bytes} [Tue Aug 2 10:02:33 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 263 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 02 10:02:33.785878 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70d764fa-4283-44a5-9d94-e805d2886f6f None admin] Authenticating user token {{(pid=80378) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:02:33.788813 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG dbcounter [-] [80378] Writing DB stats keystone:SELECT=100 {{(pid=80378) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 10:02:33.790980 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: [pid: 80377|app: 0|req: 326/647] 10.222.0.82 () {58 vars in 1023 bytes} [Tue Aug 2 10:02:33 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 137 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 02 10:02:33.802200 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ed89884-9c34-4654-9407-cccacb47dfa8 None None] Authenticating user token {{(pid=80377) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:02:33.955409 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70d764fa-4283-44a5-9d94-e805d2886f6f None admin] Validating token access rules against request {{(pid=80378) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:02:33.965081 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG dbcounter [-] [80377] Writing DB stats keystone:SELECT=100 {{(pid=80377) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 10:02:33.978906 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ed89884-9c34-4654-9407-cccacb47dfa8 None None] Validating token access rules against request {{(pid=80377) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:02:34.023828 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24cb544e-da4f-447d-b5b4-a46e530374e2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '891005fa5ff74041b75747d16cc66798', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80378) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:02:34.025369 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24cb544e-da4f-447d-b5b4-a46e530374e2 None admin] REQUEST_METHOD: `GET` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:02:34.025903 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24cb544e-da4f-447d-b5b4-a46e530374e2 None admin] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:02:34.026522 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24cb544e-da4f-447d-b5b4-a46e530374e2 None admin] PATH_INFO: `/v3/users` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:02:34.027986 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24cb544e-da4f-447d-b5b4-a46e530374e2 None admin] RBAC: Adding query filter params (domain_id=default, name=cinder) {{(pid=80378) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 02 10:02:34.028659 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24cb544e-da4f-447d-b5b4-a46e530374e2 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:02:34.029513 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08e05ab9-52bb-4653-849c-88f42f297613 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '891005fa5ff74041b75747d16cc66798', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80377) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:02:34.030190 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08e05ab9-52bb-4653-849c-88f42f297613 None admin] REQUEST_METHOD: `GET` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:02:34.030476 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08e05ab9-52bb-4653-849c-88f42f297613 None admin] SCRIPT_NAME: `/identity` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:02:34.030756 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08e05ab9-52bb-4653-849c-88f42f297613 None admin] PATH_INFO: `/v3/domains` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:02:34.031472 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08e05ab9-52bb-4653-849c-88f42f297613 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80377) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 02 10:02:34.031824 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08e05ab9-52bb-4653-849c-88f42f297613 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:02:34.033244 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24cb544e-da4f-447d-b5b4-a46e530374e2 None admin] RBAC: Authorization granted {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:02:34.034055 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08e05ab9-52bb-4653-849c-88f42f297613 None admin] RBAC: Authorization granted {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:02:34.050888 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: [pid: 80377|app: 0|req: 327/648] 10.222.0.82 () {56 vars in 1109 bytes} [Tue Aug 2 10:02:33 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 259 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 02 10:02:34.052770 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08e05ab9-52bb-4653-849c-88f42f297613 None admin] Authenticating user token {{(pid=80377) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:02:34.070179 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 322/649] 10.222.0.82 () {56 vars in 1137 bytes} [Tue Aug 2 10:02:33 2022] GET /identity/v3/users?domain_id=default&name=cinder => generated 372 bytes in 289 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 02 10:02:34.072044 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24cb544e-da4f-447d-b5b4-a46e530374e2 None admin] Authenticating user token {{(pid=80378) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:02:34.171094 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG dbcounter [-] [80378] Writing DB stats keystone:SELECT=100 {{(pid=80378) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 10:02:34.208331 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08e05ab9-52bb-4653-849c-88f42f297613 None admin] Validating token access rules against request {{(pid=80377) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:02:34.245678 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24cb544e-da4f-447d-b5b4-a46e530374e2 None admin] Validating token access rules against request {{(pid=80378) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:02:34.250713 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-293bc9c7-f08e-496c-b67a-f03e2fbce745 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '891005fa5ff74041b75747d16cc66798', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80377) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:02:34.251423 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-293bc9c7-f08e-496c-b67a-f03e2fbce745 None admin] REQUEST_METHOD: `GET` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:02:34.251851 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-293bc9c7-f08e-496c-b67a-f03e2fbce745 None admin] SCRIPT_NAME: `/identity` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:02:34.252570 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-293bc9c7-f08e-496c-b67a-f03e2fbce745 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:02:34.252799 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-293bc9c7-f08e-496c-b67a-f03e2fbce745 None admin] RBAC: Adding query filter params (role.id=02003a0be60f47f2a8dfea17053af52c, scope.project.id=f1b38ebb804c41c69772c998aae72c41, user.id=88af518196674fe593ad88d30111fb7d) {{(pid=80377) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 02 10:02:34.253331 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-293bc9c7-f08e-496c-b67a-f03e2fbce745 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:02:34.254676 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-293bc9c7-f08e-496c-b67a-f03e2fbce745 None admin] RBAC: Authorization granted {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:02:34.270049 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: [pid: 80377|app: 0|req: 328/650] 10.222.0.82 () {56 vars in 1374 bytes} [Tue Aug 2 10:02:34 2022] GET /identity/v3/role_assignments?user.id=88af518196674fe593ad88d30111fb7d&scope.project.id=f1b38ebb804c41c69772c998aae72c41&role.id=02003a0be60f47f2a8dfea17053af52c => generated 608 bytes in 210 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 02 10:02:34.270049 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-293bc9c7-f08e-496c-b67a-f03e2fbce745 None admin] Authenticating user token {{(pid=80377) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:02:34.309108 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-889e57e4-4b50-468f-9488-bc76656f8352 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '891005fa5ff74041b75747d16cc66798', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80378) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:02:34.309799 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-889e57e4-4b50-468f-9488-bc76656f8352 None admin] REQUEST_METHOD: `GET` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:02:34.310092 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-889e57e4-4b50-468f-9488-bc76656f8352 None admin] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:02:34.310363 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-889e57e4-4b50-468f-9488-bc76656f8352 None admin] PATH_INFO: `/v3/roles/service` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:02:34.325876 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG dbcounter [-] [80377] Writing DB stats keystone:SELECT=100 {{(pid=80377) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 10:02:34.326116 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-889e57e4-4b50-468f-9488-bc76656f8352 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 02 10:02:34.334076 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application [None req-889e57e4-4b50-468f-9488-bc76656f8352 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Aug 02 10:02:34.334076 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 02 10:02:34.334076 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 02 10:02:34.334076 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 02 10:02:34.334076 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 02 10:02:34.334076 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 02 10:02:34.334076 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 02 10:02:34.334076 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 02 10:02:34.334076 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 02 10:02:34.334076 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 02 10:02:34.334076 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 02 10:02:34.334076 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 02 10:02:34.334076 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 02 10:02:34.334076 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 02 10:02:34.334076 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 02 10:02:34.334076 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 02 10:02:34.335162 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 02 10:02:34.335162 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 02 10:02:34.335162 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 02 10:02:34.335162 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 02 10:02:34.335162 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 02 10:02:34.335162 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 02 10:02:34.335162 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 02 10:02:34.335162 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 02 10:02:34.335162 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 02 10:02:34.335162 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application return self.get_or_create( Aug 02 10:02:34.335162 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 02 10:02:34.335162 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application with Lock( Aug 02 10:02:34.335162 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 02 10:02:34.335162 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application return self._enter() Aug 02 10:02:34.335162 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 02 10:02:34.335162 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 02 10:02:34.336012 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 02 10:02:34.336012 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application return self.creator() Aug 02 10:02:34.336012 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 02 10:02:34.336012 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application created_value = creator( Aug 02 10:02:34.336012 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 02 10:02:34.336012 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 02 10:02:34.336012 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 02 10:02:34.336012 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 02 10:02:34.336012 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 02 10:02:34.336012 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 02 10:02:34.336012 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Aug 02 10:02:34.336012 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application Aug 02 10:02:34.342383 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 323/651] 10.222.0.82 () {56 vars in 1102 bytes} [Tue Aug 2 10:02:34 2022] GET /identity/v3/roles/service => generated 85 bytes in 271 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 02 10:02:34.343918 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-889e57e4-4b50-468f-9488-bc76656f8352 None admin] Authenticating user token {{(pid=80378) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:02:34.423975 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-293bc9c7-f08e-496c-b67a-f03e2fbce745 None admin] Validating token access rules against request {{(pid=80377) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:02:34.469010 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe43a179-4f7b-4269-b3b6-3dbbc2484775 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '891005fa5ff74041b75747d16cc66798', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80377) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:02:34.470875 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe43a179-4f7b-4269-b3b6-3dbbc2484775 None admin] REQUEST_METHOD: `GET` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:02:34.471589 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe43a179-4f7b-4269-b3b6-3dbbc2484775 None admin] SCRIPT_NAME: `/identity` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:02:34.472246 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe43a179-4f7b-4269-b3b6-3dbbc2484775 None admin] PATH_INFO: `/v3/users/glance` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:02:34.481743 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-889e57e4-4b50-468f-9488-bc76656f8352 None admin] Validating token access rules against request {{(pid=80378) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:02:34.484029 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe43a179-4f7b-4269-b3b6-3dbbc2484775 None admin] RBAC: Authorizing `identity:get_user(user_id=glance)` {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:02:34.486370 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe43a179-4f7b-4269-b3b6-3dbbc2484775 None admin] RBAC: Authorization granted {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:02:34.503642 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application [None req-fe43a179-4f7b-4269-b3b6-3dbbc2484775 None admin] Could not find user: glance.: keystone.exception.UserNotFound: Could not find user: glance. Aug 02 10:02:34.503642 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 02 10:02:34.503642 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 02 10:02:34.503642 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 02 10:02:34.503642 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 02 10:02:34.503642 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 02 10:02:34.503642 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 02 10:02:34.503642 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 02 10:02:34.503642 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 02 10:02:34.503642 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 02 10:02:34.503642 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 02 10:02:34.503642 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 02 10:02:34.503642 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 02 10:02:34.503642 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 02 10:02:34.503642 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 02 10:02:34.503642 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 02 10:02:34.504721 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 02 10:02:34.504721 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 02 10:02:34.504721 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 02 10:02:34.504721 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 02 10:02:34.504721 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 02 10:02:34.504721 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 02 10:02:34.504721 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 02 10:02:34.504721 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 02 10:02:34.504721 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 02 10:02:34.504721 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application return self.get_or_create( Aug 02 10:02:34.504721 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 02 10:02:34.504721 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application with Lock( Aug 02 10:02:34.504721 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 02 10:02:34.504721 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application return self._enter() Aug 02 10:02:34.504721 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 02 10:02:34.504721 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 02 10:02:34.504721 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 02 10:02:34.504721 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application return self.creator() Aug 02 10:02:34.505713 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 02 10:02:34.505713 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application created_value = creator( Aug 02 10:02:34.505713 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 02 10:02:34.505713 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 02 10:02:34.505713 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 02 10:02:34.505713 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 02 10:02:34.505713 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 02 10:02:34.505713 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 02 10:02:34.505713 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance. Aug 02 10:02:34.505713 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application Aug 02 10:02:34.505713 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: [pid: 80377|app: 0|req: 329/652] 10.222.0.82 () {56 vars in 1099 bytes} [Tue Aug 2 10:02:34 2022] GET /identity/v3/users/glance => generated 84 bytes in 242 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 02 10:02:34.506445 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe43a179-4f7b-4269-b3b6-3dbbc2484775 None admin] Authenticating user token {{(pid=80377) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:02:34.518457 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG dbcounter [-] [80378] Writing DB stats keystone:SELECT=100 {{(pid=80378) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 10:02:34.536197 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a17f74b-d498-49b2-8f46-580d180fe0ab None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '891005fa5ff74041b75747d16cc66798', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80378) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:02:34.537282 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a17f74b-d498-49b2-8f46-580d180fe0ab None admin] REQUEST_METHOD: `GET` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:02:34.537748 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a17f74b-d498-49b2-8f46-580d180fe0ab None admin] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:02:34.538259 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a17f74b-d498-49b2-8f46-580d180fe0ab None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:02:34.622814 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG dbcounter [-] [80377] Writing DB stats keystone:SELECT=100 {{(pid=80377) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 10:02:34.645475 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a17f74b-d498-49b2-8f46-580d180fe0ab None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:02:34.647613 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a17f74b-d498-49b2-8f46-580d180fe0ab None admin] RBAC: Authorization granted {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:02:34.660190 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe43a179-4f7b-4269-b3b6-3dbbc2484775 None admin] Validating token access rules against request {{(pid=80377) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:02:34.712066 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fe85e65-bb3c-4b4f-b42e-a2c860f5d214 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '891005fa5ff74041b75747d16cc66798', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80377) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:02:34.713179 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fe85e65-bb3c-4b4f-b42e-a2c860f5d214 None admin] REQUEST_METHOD: `GET` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:02:34.713691 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fe85e65-bb3c-4b4f-b42e-a2c860f5d214 None admin] SCRIPT_NAME: `/identity` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:02:34.714192 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fe85e65-bb3c-4b4f-b42e-a2c860f5d214 None admin] PATH_INFO: `/v3/roles` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:02:34.715257 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2fe85e65-bb3c-4b4f-b42e-a2c860f5d214 None admin] RBAC: Adding query filter params (name=service) {{(pid=80377) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 02 10:02:34.715774 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2fe85e65-bb3c-4b4f-b42e-a2c860f5d214 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:02:34.717470 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2fe85e65-bb3c-4b4f-b42e-a2c860f5d214 None admin] RBAC: Authorization granted {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:02:34.737760 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: [pid: 80377|app: 0|req: 330/653] 10.222.0.82 () {56 vars in 1103 bytes} [Tue Aug 2 10:02:34 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 232 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 02 10:02:34.739265 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fe85e65-bb3c-4b4f-b42e-a2c860f5d214 None admin] Authenticating user token {{(pid=80377) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:02:34.789337 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 324/654] 10.222.0.82 () {58 vars in 1282 bytes} [Tue Aug 2 10:02:34 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 446 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 02 10:02:34.790315 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a17f74b-d498-49b2-8f46-580d180fe0ab None admin] Authenticating user token {{(pid=80378) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:02:34.818993 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG dbcounter [-] [80378] Writing DB stats keystone:SELECT=100 {{(pid=80378) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 10:02:34.877039 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fe85e65-bb3c-4b4f-b42e-a2c860f5d214 None admin] Validating token access rules against request {{(pid=80377) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:02:34.925595 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a17f74b-d498-49b2-8f46-580d180fe0ab None admin] Validating token access rules against request {{(pid=80378) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:02:34.931367 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e62387b7-a835-45d0-ac51-d022701a0f9e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '891005fa5ff74041b75747d16cc66798', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80377) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:02:34.932251 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e62387b7-a835-45d0-ac51-d022701a0f9e None admin] REQUEST_METHOD: `GET` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:02:34.932687 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e62387b7-a835-45d0-ac51-d022701a0f9e None admin] SCRIPT_NAME: `/identity` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:02:34.933124 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e62387b7-a835-45d0-ac51-d022701a0f9e None admin] PATH_INFO: `/v3/users` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:02:34.934671 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e62387b7-a835-45d0-ac51-d022701a0f9e None admin] RBAC: Adding query filter params (domain_id=default, name=glance) {{(pid=80377) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 02 10:02:34.934995 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e62387b7-a835-45d0-ac51-d022701a0f9e None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:02:34.936553 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e62387b7-a835-45d0-ac51-d022701a0f9e None admin] RBAC: Authorization granted {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:02:34.950536 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG dbcounter [-] [80377] Writing DB stats keystone:SELECT=100 {{(pid=80377) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 10:02:34.968436 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: [pid: 80377|app: 0|req: 331/655] 10.222.0.82 () {56 vars in 1137 bytes} [Tue Aug 2 10:02:34 2022] GET /identity/v3/users?domain_id=default&name=glance => generated 372 bytes in 225 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 02 10:02:34.968436 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adaf2814-99fc-4adc-a4f2-5a0e07a1a8ea None None] REQUEST_METHOD: `GET` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:02:34.968436 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adaf2814-99fc-4adc-a4f2-5a0e07a1a8ea None None] SCRIPT_NAME: `/identity` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:02:34.968436 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adaf2814-99fc-4adc-a4f2-5a0e07a1a8ea None None] PATH_INFO: `/` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:02:34.968436 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: [pid: 80377|app: 0|req: 332/656] 10.222.0.82 () {54 vars in 926 bytes} [Tue Aug 2 10:02:34 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 02 10:02:34.968436 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-adaf2814-99fc-4adc-a4f2-5a0e07a1a8ea None None] Authenticating user token {{(pid=80377) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:02:34.975323 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5204de51-2268-410e-bf97-c6cca3bff2e4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '891005fa5ff74041b75747d16cc66798', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80378) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:02:34.975941 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5204de51-2268-410e-bf97-c6cca3bff2e4 None admin] REQUEST_METHOD: `GET` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:02:34.976290 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5204de51-2268-410e-bf97-c6cca3bff2e4 None admin] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:02:34.976635 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5204de51-2268-410e-bf97-c6cca3bff2e4 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:02:34.989660 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5204de51-2268-410e-bf97-c6cca3bff2e4 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:02:34.991196 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5204de51-2268-410e-bf97-c6cca3bff2e4 None admin] RBAC: Authorization granted {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:02:35.009681 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application [None req-5204de51-2268-410e-bf97-c6cca3bff2e4 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 02 10:02:35.009681 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 02 10:02:35.009681 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 02 10:02:35.009681 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 02 10:02:35.009681 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 02 10:02:35.009681 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 02 10:02:35.009681 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 02 10:02:35.009681 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 02 10:02:35.009681 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 02 10:02:35.009681 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 02 10:02:35.009681 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 02 10:02:35.009681 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 02 10:02:35.009681 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 02 10:02:35.009681 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 02 10:02:35.009681 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 02 10:02:35.010816 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 02 10:02:35.010816 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 02 10:02:35.010816 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 02 10:02:35.010816 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 02 10:02:35.010816 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 02 10:02:35.010816 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 02 10:02:35.010816 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application return self.get_or_create( Aug 02 10:02:35.010816 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 02 10:02:35.010816 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application with Lock( Aug 02 10:02:35.010816 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 02 10:02:35.010816 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application return self._enter() Aug 02 10:02:35.010816 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 02 10:02:35.010816 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 02 10:02:35.010816 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 02 10:02:35.010816 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application return self.creator() Aug 02 10:02:35.010816 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 02 10:02:35.010816 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application created_value = creator( Aug 02 10:02:35.011712 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 02 10:02:35.011712 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 02 10:02:35.011712 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 02 10:02:35.011712 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application Aug 02 10:02:35.016591 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 325/657] 10.222.0.82 () {56 vars in 1108 bytes} [Tue Aug 2 10:02:34 2022] GET /identity/v3/domains/Default => generated 87 bytes in 226 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 02 10:02:35.018149 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ca6b341-34d8-4c84-801b-faa6199ed358 None None] REQUEST_METHOD: `POST` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:02:35.018448 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ca6b341-34d8-4c84-801b-faa6199ed358 None None] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:02:35.018729 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ca6b341-34d8-4c84-801b-faa6199ed358 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:02:35.043183 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.auth.core [None req-1ca6b341-34d8-4c84-801b-faa6199ed358 None None] MFA Rules not processed for user `891005fa5ff74041b75747d16cc66798`. Rule list: `[]` (Enabled: `True`). {{(pid=80378) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 02 10:02:35.044425 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.fernet_utils [None req-1ca6b341-34d8-4c84-801b-faa6199ed358 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80378) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 10:02:35.108926 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-adaf2814-99fc-4adc-a4f2-5a0e07a1a8ea None None] Validating token access rules against request {{(pid=80377) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:02:35.138142 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 326/658] 10.222.0.82 () {58 vars in 1023 bytes} [Tue Aug 2 10:02:35 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 121 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 02 10:02:35.139924 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ca6b341-34d8-4c84-801b-faa6199ed358 None None] Authenticating user token {{(pid=80378) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:02:35.145236 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG dbcounter [-] [80378] Writing DB stats keystone:SELECT=100 {{(pid=80378) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 10:02:35.156899 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be40b8c3-8849-4149-84d6-0300ffd8aeab None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '891005fa5ff74041b75747d16cc66798', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80377) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:02:35.158292 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be40b8c3-8849-4149-84d6-0300ffd8aeab None admin] REQUEST_METHOD: `GET` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:02:35.158781 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be40b8c3-8849-4149-84d6-0300ffd8aeab None admin] SCRIPT_NAME: `/identity` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:02:35.159258 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be40b8c3-8849-4149-84d6-0300ffd8aeab None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:02:35.175514 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be40b8c3-8849-4149-84d6-0300ffd8aeab None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:02:35.177279 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be40b8c3-8849-4149-84d6-0300ffd8aeab None admin] RBAC: Authorization granted {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:02:35.196661 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application [None req-be40b8c3-8849-4149-84d6-0300ffd8aeab None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 02 10:02:35.196661 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 02 10:02:35.196661 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 02 10:02:35.196661 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 02 10:02:35.196661 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 02 10:02:35.196661 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 02 10:02:35.196661 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 02 10:02:35.196661 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 02 10:02:35.196661 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 02 10:02:35.196661 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 02 10:02:35.196661 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 02 10:02:35.196661 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 02 10:02:35.196661 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 02 10:02:35.196661 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 02 10:02:35.196661 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 02 10:02:35.197760 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 02 10:02:35.197760 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 02 10:02:35.197760 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 02 10:02:35.197760 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 02 10:02:35.197760 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 02 10:02:35.197760 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 02 10:02:35.197760 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application return self.get_or_create( Aug 02 10:02:35.197760 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 02 10:02:35.197760 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application with Lock( Aug 02 10:02:35.197760 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 02 10:02:35.197760 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application return self._enter() Aug 02 10:02:35.197760 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 02 10:02:35.197760 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 02 10:02:35.197760 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 02 10:02:35.197760 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application return self.creator() Aug 02 10:02:35.197760 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 02 10:02:35.197760 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application created_value = creator( Aug 02 10:02:35.198702 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 02 10:02:35.198702 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 02 10:02:35.198702 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 02 10:02:35.198702 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application Aug 02 10:02:35.198702 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: [pid: 80377|app: 0|req: 333/659] 10.222.0.82 () {56 vars in 1108 bytes} [Tue Aug 2 10:02:34 2022] GET /identity/v3/domains/Default => generated 87 bytes in 230 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 02 10:02:35.198702 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be40b8c3-8849-4149-84d6-0300ffd8aeab None admin] Authenticating user token {{(pid=80377) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:02:35.276928 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG dbcounter [-] [80377] Writing DB stats keystone:SELECT=100 {{(pid=80377) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 10:02:35.289023 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ca6b341-34d8-4c84-801b-faa6199ed358 None None] Validating token access rules against request {{(pid=80378) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:02:35.326834 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-deb5c8a3-c27b-400a-a35c-52fe1784e86a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '891005fa5ff74041b75747d16cc66798', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80378) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:02:35.327465 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-deb5c8a3-c27b-400a-a35c-52fe1784e86a None admin] REQUEST_METHOD: `GET` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:02:35.327747 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-deb5c8a3-c27b-400a-a35c-52fe1784e86a None admin] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:02:35.328079 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-deb5c8a3-c27b-400a-a35c-52fe1784e86a None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:02:35.350372 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be40b8c3-8849-4149-84d6-0300ffd8aeab None admin] Validating token access rules against request {{(pid=80377) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:02:35.397837 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ada5639d-b632-4608-afe9-8e0edafcc8e5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '891005fa5ff74041b75747d16cc66798', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80377) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:02:35.398469 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ada5639d-b632-4608-afe9-8e0edafcc8e5 None admin] REQUEST_METHOD: `GET` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:02:35.398753 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ada5639d-b632-4608-afe9-8e0edafcc8e5 None admin] SCRIPT_NAME: `/identity` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:02:35.399026 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ada5639d-b632-4608-afe9-8e0edafcc8e5 None admin] PATH_INFO: `/v3/domains` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:02:35.399725 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ada5639d-b632-4608-afe9-8e0edafcc8e5 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80377) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 02 10:02:35.400033 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ada5639d-b632-4608-afe9-8e0edafcc8e5 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:02:35.401521 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ada5639d-b632-4608-afe9-8e0edafcc8e5 None admin] RBAC: Authorization granted {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:02:35.415276 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-deb5c8a3-c27b-400a-a35c-52fe1784e86a None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:02:35.416944 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-deb5c8a3-c27b-400a-a35c-52fe1784e86a None admin] RBAC: Authorization granted {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:02:35.424425 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: [pid: 80377|app: 0|req: 334/660] 10.222.0.82 () {56 vars in 1109 bytes} [Tue Aug 2 10:02:35 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 226 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 02 10:02:35.425390 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ada5639d-b632-4608-afe9-8e0edafcc8e5 None admin] Authenticating user token {{(pid=80377) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:02:35.427146 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG dbcounter [-] [80378] Writing DB stats keystone:SELECT=100 {{(pid=80378) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 10:02:35.560596 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 327/661] 10.222.0.82 () {58 vars in 1282 bytes} [Tue Aug 2 10:02:35 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 421 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 02 10:02:35.561462 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-deb5c8a3-c27b-400a-a35c-52fe1784e86a None admin] Authenticating user token {{(pid=80378) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:02:35.580739 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ada5639d-b632-4608-afe9-8e0edafcc8e5 None admin] Validating token access rules against request {{(pid=80377) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:02:35.594849 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG dbcounter [-] [80377] Writing DB stats keystone:SELECT=100 {{(pid=80377) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 10:02:35.630723 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbb92d43-2a8c-44ce-b816-24a2ad7fb571 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '891005fa5ff74041b75747d16cc66798', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80377) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:02:35.631377 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbb92d43-2a8c-44ce-b816-24a2ad7fb571 None admin] REQUEST_METHOD: `GET` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:02:35.631659 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbb92d43-2a8c-44ce-b816-24a2ad7fb571 None admin] SCRIPT_NAME: `/identity` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:02:35.631927 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbb92d43-2a8c-44ce-b816-24a2ad7fb571 None admin] PATH_INFO: `/v3/roles/service` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:02:35.656444 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cbb92d43-2a8c-44ce-b816-24a2ad7fb571 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 02 10:02:35.662299 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application [None req-cbb92d43-2a8c-44ce-b816-24a2ad7fb571 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Aug 02 10:02:35.662299 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 02 10:02:35.662299 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 02 10:02:35.662299 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 02 10:02:35.662299 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 02 10:02:35.662299 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 02 10:02:35.662299 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 02 10:02:35.662299 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 02 10:02:35.662299 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 02 10:02:35.662299 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 02 10:02:35.662299 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 02 10:02:35.662299 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 02 10:02:35.662299 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 02 10:02:35.662299 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 02 10:02:35.662299 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 02 10:02:35.662299 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 02 10:02:35.665026 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 02 10:02:35.665026 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 02 10:02:35.665026 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 02 10:02:35.665026 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 02 10:02:35.665026 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 02 10:02:35.665026 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 02 10:02:35.665026 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 02 10:02:35.665026 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 02 10:02:35.665026 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 02 10:02:35.665026 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application return self.get_or_create( Aug 02 10:02:35.665026 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 02 10:02:35.665026 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application with Lock( Aug 02 10:02:35.665026 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 02 10:02:35.665026 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application return self._enter() Aug 02 10:02:35.665026 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 02 10:02:35.665026 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 02 10:02:35.665959 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 02 10:02:35.665959 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application return self.creator() Aug 02 10:02:35.665959 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 02 10:02:35.665959 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application created_value = creator( Aug 02 10:02:35.665959 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 02 10:02:35.665959 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 02 10:02:35.665959 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 02 10:02:35.665959 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 02 10:02:35.665959 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 02 10:02:35.665959 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 02 10:02:35.665959 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Aug 02 10:02:35.665959 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application Aug 02 10:02:35.665959 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: [pid: 80377|app: 0|req: 335/662] 10.222.0.82 () {56 vars in 1102 bytes} [Tue Aug 2 10:02:35 2022] GET /identity/v3/roles/service => generated 85 bytes in 239 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 02 10:02:35.665959 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbb92d43-2a8c-44ce-b816-24a2ad7fb571 None admin] Authenticating user token {{(pid=80377) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:02:35.725366 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-deb5c8a3-c27b-400a-a35c-52fe1784e86a None admin] Validating token access rules against request {{(pid=80378) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:02:35.730625 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG dbcounter [-] [80378] Writing DB stats keystone:SELECT=100 {{(pid=80378) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 10:02:35.798678 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d240744-938f-4e63-9278-dbdb8f2e28a0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '891005fa5ff74041b75747d16cc66798', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80378) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:02:35.799682 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d240744-938f-4e63-9278-dbdb8f2e28a0 None admin] REQUEST_METHOD: `GET` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:02:35.800148 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d240744-938f-4e63-9278-dbdb8f2e28a0 None admin] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:02:35.800605 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d240744-938f-4e63-9278-dbdb8f2e28a0 None admin] PATH_INFO: `/v3/domains` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:02:35.801800 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d240744-938f-4e63-9278-dbdb8f2e28a0 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80378) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 02 10:02:35.802319 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d240744-938f-4e63-9278-dbdb8f2e28a0 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:02:35.804451 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d240744-938f-4e63-9278-dbdb8f2e28a0 None admin] RBAC: Authorization granted {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:02:35.818387 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbb92d43-2a8c-44ce-b816-24a2ad7fb571 None admin] Validating token access rules against request {{(pid=80377) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:02:35.822767 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 328/663] 10.222.0.82 () {56 vars in 1109 bytes} [Tue Aug 2 10:02:35 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 262 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 02 10:02:35.824254 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d240744-938f-4e63-9278-dbdb8f2e28a0 None admin] Authenticating user token {{(pid=80378) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:02:35.889532 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2424789-c56f-4409-a18f-4a7fb7f29c41 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '891005fa5ff74041b75747d16cc66798', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80377) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:02:35.890257 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2424789-c56f-4409-a18f-4a7fb7f29c41 None admin] REQUEST_METHOD: `GET` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:02:35.890868 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2424789-c56f-4409-a18f-4a7fb7f29c41 None admin] SCRIPT_NAME: `/identity` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:02:35.890868 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2424789-c56f-4409-a18f-4a7fb7f29c41 None admin] PATH_INFO: `/v3/users/neutron` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:02:35.902544 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2424789-c56f-4409-a18f-4a7fb7f29c41 None admin] RBAC: Authorizing `identity:get_user(user_id=neutron)` {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:02:35.904869 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2424789-c56f-4409-a18f-4a7fb7f29c41 None admin] RBAC: Authorization granted {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:02:35.918911 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application [None req-b2424789-c56f-4409-a18f-4a7fb7f29c41 None admin] Could not find user: neutron.: keystone.exception.UserNotFound: Could not find user: neutron. Aug 02 10:02:35.918911 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 02 10:02:35.918911 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 02 10:02:35.918911 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 02 10:02:35.918911 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 02 10:02:35.918911 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 02 10:02:35.918911 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 02 10:02:35.918911 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 02 10:02:35.918911 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 02 10:02:35.918911 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 02 10:02:35.918911 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 02 10:02:35.918911 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 02 10:02:35.918911 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 02 10:02:35.918911 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 02 10:02:35.918911 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 02 10:02:35.918911 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 02 10:02:35.920814 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 02 10:02:35.920814 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 02 10:02:35.920814 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 02 10:02:35.920814 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 02 10:02:35.920814 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 02 10:02:35.920814 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 02 10:02:35.920814 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 02 10:02:35.920814 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 02 10:02:35.920814 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 02 10:02:35.920814 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application return self.get_or_create( Aug 02 10:02:35.920814 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 02 10:02:35.920814 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application with Lock( Aug 02 10:02:35.920814 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 02 10:02:35.920814 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application return self._enter() Aug 02 10:02:35.920814 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 02 10:02:35.920814 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 02 10:02:35.920814 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 02 10:02:35.920814 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application return self.creator() Aug 02 10:02:35.922744 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 02 10:02:35.922744 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application created_value = creator( Aug 02 10:02:35.922744 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 02 10:02:35.922744 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 02 10:02:35.922744 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 02 10:02:35.922744 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 02 10:02:35.922744 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 02 10:02:35.922744 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 02 10:02:35.922744 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: neutron. Aug 02 10:02:35.922744 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application Aug 02 10:02:35.922744 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: [pid: 80377|app: 0|req: 336/664] 10.222.0.82 () {56 vars in 1102 bytes} [Tue Aug 2 10:02:35 2022] GET /identity/v3/users/neutron => generated 85 bytes in 256 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 02 10:02:35.922744 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2424789-c56f-4409-a18f-4a7fb7f29c41 None admin] Authenticating user token {{(pid=80377) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:02:35.975934 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG dbcounter [-] [80377] Writing DB stats keystone:SELECT=100 {{(pid=80377) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 10:02:35.999523 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d240744-938f-4e63-9278-dbdb8f2e28a0 None admin] Validating token access rules against request {{(pid=80378) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:02:36.048950 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1cc246d-c57d-4030-ac04-4e12c002d5b3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '891005fa5ff74041b75747d16cc66798', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80378) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:02:36.050729 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1cc246d-c57d-4030-ac04-4e12c002d5b3 None admin] REQUEST_METHOD: `GET` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:02:36.050729 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1cc246d-c57d-4030-ac04-4e12c002d5b3 None admin] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:02:36.051053 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1cc246d-c57d-4030-ac04-4e12c002d5b3 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:02:36.063018 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1cc246d-c57d-4030-ac04-4e12c002d5b3 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:02:36.065242 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1cc246d-c57d-4030-ac04-4e12c002d5b3 None admin] RBAC: Authorization granted {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:02:36.077978 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2424789-c56f-4409-a18f-4a7fb7f29c41 None admin] Validating token access rules against request {{(pid=80377) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:02:36.082504 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application [None req-b1cc246d-c57d-4030-ac04-4e12c002d5b3 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 02 10:02:36.082504 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 02 10:02:36.082504 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 02 10:02:36.082504 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 02 10:02:36.082504 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 02 10:02:36.082504 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 02 10:02:36.082504 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 02 10:02:36.082504 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 02 10:02:36.082504 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 02 10:02:36.082504 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 02 10:02:36.082504 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 02 10:02:36.082504 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 02 10:02:36.082504 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 02 10:02:36.082504 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 02 10:02:36.082504 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 02 10:02:36.083403 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 02 10:02:36.083403 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 02 10:02:36.083403 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 02 10:02:36.083403 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 02 10:02:36.083403 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 02 10:02:36.083403 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 02 10:02:36.083403 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application return self.get_or_create( Aug 02 10:02:36.083403 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 02 10:02:36.083403 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application with Lock( Aug 02 10:02:36.083403 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 02 10:02:36.083403 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application return self._enter() Aug 02 10:02:36.083403 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 02 10:02:36.083403 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 02 10:02:36.083403 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 02 10:02:36.083403 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application return self.creator() Aug 02 10:02:36.083403 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 02 10:02:36.083403 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application created_value = creator( Aug 02 10:02:36.084300 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 02 10:02:36.084300 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 02 10:02:36.084300 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 02 10:02:36.084300 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application Aug 02 10:02:36.084300 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 329/665] 10.222.0.82 () {56 vars in 1108 bytes} [Tue Aug 2 10:02:35 2022] GET /identity/v3/domains/Default => generated 87 bytes in 260 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 02 10:02:36.084597 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1cc246d-c57d-4030-ac04-4e12c002d5b3 None admin] Authenticating user token {{(pid=80378) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:02:36.112271 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG dbcounter [-] [80378] Writing DB stats keystone:SELECT=100 {{(pid=80378) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 10:02:36.133499 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3c296a9-62f6-479b-a644-2948b856e73a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '891005fa5ff74041b75747d16cc66798', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80377) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:02:36.134925 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3c296a9-62f6-479b-a644-2948b856e73a None admin] REQUEST_METHOD: `GET` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:02:36.135630 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3c296a9-62f6-479b-a644-2948b856e73a None admin] SCRIPT_NAME: `/identity` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:02:36.136302 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3c296a9-62f6-479b-a644-2948b856e73a None admin] PATH_INFO: `/v3/roles` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:02:36.137418 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3c296a9-62f6-479b-a644-2948b856e73a None admin] RBAC: Adding query filter params (name=service) {{(pid=80377) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 02 10:02:36.137967 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3c296a9-62f6-479b-a644-2948b856e73a None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:02:36.139803 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3c296a9-62f6-479b-a644-2948b856e73a None admin] RBAC: Authorization granted {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:02:36.149700 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: [pid: 80377|app: 0|req: 337/666] 10.222.0.82 () {56 vars in 1103 bytes} [Tue Aug 2 10:02:35 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 229 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 02 10:02:36.150891 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3c296a9-62f6-479b-a644-2948b856e73a None admin] Authenticating user token {{(pid=80377) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:02:36.226694 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1cc246d-c57d-4030-ac04-4e12c002d5b3 None admin] Validating token access rules against request {{(pid=80378) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:02:36.284262 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8b43606-c678-4b31-816a-e4e4a371e3f2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '891005fa5ff74041b75747d16cc66798', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80378) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:02:36.284262 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8b43606-c678-4b31-816a-e4e4a371e3f2 None admin] REQUEST_METHOD: `GET` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:02:36.284262 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8b43606-c678-4b31-816a-e4e4a371e3f2 None admin] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:02:36.284262 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8b43606-c678-4b31-816a-e4e4a371e3f2 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:02:36.293898 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG dbcounter [-] [80377] Writing DB stats keystone:SELECT=100 {{(pid=80377) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 10:02:36.294349 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8b43606-c678-4b31-816a-e4e4a371e3f2 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:02:36.296427 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8b43606-c678-4b31-816a-e4e4a371e3f2 None admin] RBAC: Authorization granted {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:02:36.306649 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3c296a9-62f6-479b-a644-2948b856e73a None admin] Validating token access rules against request {{(pid=80377) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:02:36.311180 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application [None req-b8b43606-c678-4b31-816a-e4e4a371e3f2 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 02 10:02:36.311180 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 02 10:02:36.311180 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 02 10:02:36.311180 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 02 10:02:36.311180 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 02 10:02:36.311180 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 02 10:02:36.311180 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 02 10:02:36.311180 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 02 10:02:36.311180 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 02 10:02:36.311180 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 02 10:02:36.311180 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 02 10:02:36.311180 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 02 10:02:36.311180 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 02 10:02:36.311180 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 02 10:02:36.311180 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 02 10:02:36.312132 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 02 10:02:36.312132 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 02 10:02:36.312132 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 02 10:02:36.312132 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 02 10:02:36.312132 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 02 10:02:36.312132 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 02 10:02:36.312132 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application return self.get_or_create( Aug 02 10:02:36.312132 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 02 10:02:36.312132 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application with Lock( Aug 02 10:02:36.312132 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 02 10:02:36.312132 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application return self._enter() Aug 02 10:02:36.312132 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 02 10:02:36.312132 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 02 10:02:36.312132 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 02 10:02:36.312132 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application return self.creator() Aug 02 10:02:36.312132 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 02 10:02:36.312132 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application created_value = creator( Aug 02 10:02:36.313033 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 02 10:02:36.313033 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 02 10:02:36.313033 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 02 10:02:36.313033 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 02 10:02:36.313033 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 02 10:02:36.313033 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 02 10:02:36.313033 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 02 10:02:36.313033 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application Aug 02 10:02:36.313033 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 330/667] 10.222.0.82 () {56 vars in 1111 bytes} [Tue Aug 2 10:02:36 2022] GET /identity/v3/projects/service => generated 88 bytes in 228 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 02 10:02:36.313537 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8b43606-c678-4b31-816a-e4e4a371e3f2 None admin] Authenticating user token {{(pid=80378) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:02:36.357419 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6ad3b99-762c-4940-9753-18b12b2bb1d5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '891005fa5ff74041b75747d16cc66798', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80377) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:02:36.358366 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6ad3b99-762c-4940-9753-18b12b2bb1d5 None admin] REQUEST_METHOD: `GET` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:02:36.358815 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6ad3b99-762c-4940-9753-18b12b2bb1d5 None admin] SCRIPT_NAME: `/identity` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:02:36.359097 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6ad3b99-762c-4940-9753-18b12b2bb1d5 None admin] PATH_INFO: `/v3/users` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:02:36.359908 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6ad3b99-762c-4940-9753-18b12b2bb1d5 None admin] RBAC: Adding query filter params (domain_id=default, name=neutron) {{(pid=80377) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 02 10:02:36.360228 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6ad3b99-762c-4940-9753-18b12b2bb1d5 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:02:36.361724 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6ad3b99-762c-4940-9753-18b12b2bb1d5 None admin] RBAC: Authorization granted {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:02:36.388426 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: [pid: 80377|app: 0|req: 338/668] 10.222.0.82 () {56 vars in 1139 bytes} [Tue Aug 2 10:02:36 2022] GET /identity/v3/users?domain_id=default&name=neutron => generated 374 bytes in 231 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 02 10:02:36.388426 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6ad3b99-762c-4940-9753-18b12b2bb1d5 None admin] Authenticating user token {{(pid=80377) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:02:36.426494 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG dbcounter [-] [80378] Writing DB stats keystone:SELECT=100 {{(pid=80378) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 10:02:36.468734 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8b43606-c678-4b31-816a-e4e4a371e3f2 None admin] Validating token access rules against request {{(pid=80378) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:02:36.522570 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6ad3b99-762c-4940-9753-18b12b2bb1d5 None admin] Validating token access rules against request {{(pid=80377) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:02:36.527852 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9ac7dca-66e8-4542-99b6-e83e22118da8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '891005fa5ff74041b75747d16cc66798', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80378) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:02:36.528495 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9ac7dca-66e8-4542-99b6-e83e22118da8 None admin] REQUEST_METHOD: `GET` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:02:36.528773 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9ac7dca-66e8-4542-99b6-e83e22118da8 None admin] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:02:36.529047 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9ac7dca-66e8-4542-99b6-e83e22118da8 None admin] PATH_INFO: `/v3/domains` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:02:36.529952 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9ac7dca-66e8-4542-99b6-e83e22118da8 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80378) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 02 10:02:36.530292 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9ac7dca-66e8-4542-99b6-e83e22118da8 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:02:36.533143 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9ac7dca-66e8-4542-99b6-e83e22118da8 None admin] RBAC: Authorization granted {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:02:36.555238 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 331/669] 10.222.0.82 () {56 vars in 1109 bytes} [Tue Aug 2 10:02:36 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 236 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 02 10:02:36.555238 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9ac7dca-66e8-4542-99b6-e83e22118da8 None admin] Authenticating user token {{(pid=80378) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:02:36.577313 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22053707-7c8d-4312-9c50-16e3d687c2b1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '891005fa5ff74041b75747d16cc66798', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80377) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:02:36.578088 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22053707-7c8d-4312-9c50-16e3d687c2b1 None admin] REQUEST_METHOD: `GET` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:02:36.578431 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22053707-7c8d-4312-9c50-16e3d687c2b1 None admin] SCRIPT_NAME: `/identity` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:02:36.578712 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22053707-7c8d-4312-9c50-16e3d687c2b1 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:02:36.593374 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22053707-7c8d-4312-9c50-16e3d687c2b1 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:02:36.595552 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22053707-7c8d-4312-9c50-16e3d687c2b1 None admin] RBAC: Authorization granted {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:02:36.617903 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application [None req-22053707-7c8d-4312-9c50-16e3d687c2b1 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 02 10:02:36.617903 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 02 10:02:36.617903 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 02 10:02:36.617903 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 02 10:02:36.617903 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 02 10:02:36.617903 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 02 10:02:36.617903 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 02 10:02:36.617903 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 02 10:02:36.617903 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 02 10:02:36.617903 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 02 10:02:36.617903 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 02 10:02:36.617903 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 02 10:02:36.617903 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 02 10:02:36.617903 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 02 10:02:36.617903 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 02 10:02:36.619487 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 02 10:02:36.619487 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 02 10:02:36.619487 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 02 10:02:36.619487 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 02 10:02:36.619487 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 02 10:02:36.619487 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 02 10:02:36.619487 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application return self.get_or_create( Aug 02 10:02:36.619487 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 02 10:02:36.619487 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application with Lock( Aug 02 10:02:36.619487 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 02 10:02:36.619487 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application return self._enter() Aug 02 10:02:36.619487 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 02 10:02:36.619487 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 02 10:02:36.619487 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 02 10:02:36.619487 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application return self.creator() Aug 02 10:02:36.619487 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 02 10:02:36.619487 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application created_value = creator( Aug 02 10:02:36.628648 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 02 10:02:36.628648 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 02 10:02:36.628648 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 02 10:02:36.628648 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application Aug 02 10:02:36.628648 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: [pid: 80377|app: 0|req: 339/670] 10.222.0.82 () {56 vars in 1108 bytes} [Tue Aug 2 10:02:36 2022] GET /identity/v3/domains/Default => generated 87 bytes in 237 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 02 10:02:36.628648 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22053707-7c8d-4312-9c50-16e3d687c2b1 None admin] Authenticating user token {{(pid=80377) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:02:36.629544 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG dbcounter [-] [80377] Writing DB stats keystone:SELECT=100 {{(pid=80377) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 10:02:36.681076 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9ac7dca-66e8-4542-99b6-e83e22118da8 None admin] Validating token access rules against request {{(pid=80378) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:02:36.726508 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ed8c2b2-24d9-44fd-8ca0-af175b92afd5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '891005fa5ff74041b75747d16cc66798', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80378) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:02:36.727144 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ed8c2b2-24d9-44fd-8ca0-af175b92afd5 None admin] REQUEST_METHOD: `GET` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:02:36.727423 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ed8c2b2-24d9-44fd-8ca0-af175b92afd5 None admin] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:02:36.727692 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ed8c2b2-24d9-44fd-8ca0-af175b92afd5 None admin] PATH_INFO: `/v3/projects` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:02:36.728447 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ed8c2b2-24d9-44fd-8ca0-af175b92afd5 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80378) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 02 10:02:36.729083 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ed8c2b2-24d9-44fd-8ca0-af175b92afd5 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:02:36.730218 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ed8c2b2-24d9-44fd-8ca0-af175b92afd5 None admin] RBAC: Authorization granted {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:02:36.733012 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG dbcounter [-] [80378] Writing DB stats keystone:SELECT=100 {{(pid=80378) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 10:02:36.746143 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 332/671] 10.222.0.82 () {56 vars in 1148 bytes} [Tue Aug 2 10:02:36 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 196 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 02 10:02:36.747000 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ed8c2b2-24d9-44fd-8ca0-af175b92afd5 None admin] Authenticating user token {{(pid=80378) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:02:36.748610 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22053707-7c8d-4312-9c50-16e3d687c2b1 None admin] Validating token access rules against request {{(pid=80377) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:02:36.785576 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05f62c68-7cf2-4523-88a0-1eff3fd878f9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '891005fa5ff74041b75747d16cc66798', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80377) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:02:36.786351 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05f62c68-7cf2-4523-88a0-1eff3fd878f9 None admin] REQUEST_METHOD: `GET` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:02:36.786635 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05f62c68-7cf2-4523-88a0-1eff3fd878f9 None admin] SCRIPT_NAME: `/identity` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:02:36.786912 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05f62c68-7cf2-4523-88a0-1eff3fd878f9 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:02:36.862547 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ed8c2b2-24d9-44fd-8ca0-af175b92afd5 None admin] Validating token access rules against request {{(pid=80378) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:02:36.873468 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05f62c68-7cf2-4523-88a0-1eff3fd878f9 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:02:36.877361 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05f62c68-7cf2-4523-88a0-1eff3fd878f9 None admin] RBAC: Authorization granted {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:02:36.879884 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG dbcounter [-] [80377] Writing DB stats keystone:SELECT=100 {{(pid=80377) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 10:02:36.910196 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62cca8b7-5075-422f-8756-e5e529921ec4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '891005fa5ff74041b75747d16cc66798', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80378) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:02:36.910867 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62cca8b7-5075-422f-8756-e5e529921ec4 None admin] REQUEST_METHOD: `GET` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:02:36.911155 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62cca8b7-5075-422f-8756-e5e529921ec4 None admin] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:02:36.911499 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62cca8b7-5075-422f-8756-e5e529921ec4 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:02:36.920096 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62cca8b7-5075-422f-8756-e5e529921ec4 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:02:36.921573 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62cca8b7-5075-422f-8756-e5e529921ec4 None admin] RBAC: Authorization granted {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:02:36.934934 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application [None req-62cca8b7-5075-422f-8756-e5e529921ec4 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 02 10:02:36.934934 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 02 10:02:36.934934 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 02 10:02:36.934934 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 02 10:02:36.934934 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 02 10:02:36.934934 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 02 10:02:36.934934 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 02 10:02:36.934934 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 02 10:02:36.934934 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 02 10:02:36.934934 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 02 10:02:36.934934 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 02 10:02:36.934934 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 02 10:02:36.934934 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 02 10:02:36.934934 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 02 10:02:36.934934 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 02 10:02:36.936115 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 02 10:02:36.936115 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 02 10:02:36.936115 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 02 10:02:36.936115 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 02 10:02:36.936115 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 02 10:02:36.936115 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 02 10:02:36.936115 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application return self.get_or_create( Aug 02 10:02:36.936115 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 02 10:02:36.936115 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application with Lock( Aug 02 10:02:36.936115 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 02 10:02:36.936115 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application return self._enter() Aug 02 10:02:36.936115 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 02 10:02:36.936115 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 02 10:02:36.936115 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 02 10:02:36.936115 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application return self.creator() Aug 02 10:02:36.936115 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 02 10:02:36.936115 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application created_value = creator( Aug 02 10:02:36.937015 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 02 10:02:36.937015 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 02 10:02:36.937015 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 02 10:02:36.937015 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 02 10:02:36.937015 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 02 10:02:36.937015 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 02 10:02:36.937015 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 02 10:02:36.937015 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application Aug 02 10:02:36.937015 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 333/672] 10.222.0.82 () {56 vars in 1111 bytes} [Tue Aug 2 10:02:36 2022] GET /identity/v3/projects/service => generated 88 bytes in 189 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 02 10:02:36.937015 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62cca8b7-5075-422f-8756-e5e529921ec4 None admin] Authenticating user token {{(pid=80378) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:02:37.004101 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: [pid: 80377|app: 0|req: 340/673] 10.222.0.82 () {58 vars in 1282 bytes} [Tue Aug 2 10:02:36 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 383 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 02 10:02:37.005676 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b83b862e-c01d-4355-a819-aa7855d1b1e9 None None] REQUEST_METHOD: `GET` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:02:37.005676 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b83b862e-c01d-4355-a819-aa7855d1b1e9 None None] SCRIPT_NAME: `/identity` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:02:37.005903 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b83b862e-c01d-4355-a819-aa7855d1b1e9 None None] PATH_INFO: `/` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:02:37.007278 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: [pid: 80377|app: 0|req: 341/674] 10.222.0.82 () {54 vars in 926 bytes} [Tue Aug 2 10:02:37 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 02 10:02:37.009831 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b83b862e-c01d-4355-a819-aa7855d1b1e9 None None] Authenticating user token {{(pid=80377) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:02:37.029914 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG dbcounter [-] [80378] Writing DB stats keystone:SELECT=100 {{(pid=80378) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 10:02:37.084906 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62cca8b7-5075-422f-8756-e5e529921ec4 None admin] Validating token access rules against request {{(pid=80378) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:02:37.125002 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5380154b-e4a3-4164-9531-1dd1bda46ab7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '891005fa5ff74041b75747d16cc66798', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80378) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:02:37.125561 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5380154b-e4a3-4164-9531-1dd1bda46ab7 None admin] REQUEST_METHOD: `GET` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:02:37.125965 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5380154b-e4a3-4164-9531-1dd1bda46ab7 None admin] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:02:37.126844 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5380154b-e4a3-4164-9531-1dd1bda46ab7 None admin] PATH_INFO: `/v3/domains` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:02:37.127462 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b83b862e-c01d-4355-a819-aa7855d1b1e9 None None] Validating token access rules against request {{(pid=80377) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:02:37.127748 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5380154b-e4a3-4164-9531-1dd1bda46ab7 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80378) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 02 10:02:37.127849 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5380154b-e4a3-4164-9531-1dd1bda46ab7 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:02:37.129203 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5380154b-e4a3-4164-9531-1dd1bda46ab7 None admin] RBAC: Authorization granted {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:02:37.129618 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG dbcounter [-] [80377] Writing DB stats keystone:SELECT=100 {{(pid=80377) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 10:02:37.150838 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 334/675] 10.222.0.82 () {56 vars in 1109 bytes} [Tue Aug 2 10:02:36 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 214 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 02 10:02:37.152290 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5380154b-e4a3-4164-9531-1dd1bda46ab7 None admin] Authenticating user token {{(pid=80378) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:02:37.170704 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6e6883c-31b7-4499-a165-3df19f790b9a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '891005fa5ff74041b75747d16cc66798', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80377) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:02:37.171493 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6e6883c-31b7-4499-a165-3df19f790b9a None admin] REQUEST_METHOD: `GET` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:02:37.171779 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6e6883c-31b7-4499-a165-3df19f790b9a None admin] SCRIPT_NAME: `/identity` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:02:37.172065 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6e6883c-31b7-4499-a165-3df19f790b9a None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:02:37.172860 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6e6883c-31b7-4499-a165-3df19f790b9a None admin] RBAC: Adding query filter params (role.id=02003a0be60f47f2a8dfea17053af52c, scope.project.id=f1b38ebb804c41c69772c998aae72c41, user.id=1023de281ae74e9d8a6d6c312c1c97a7) {{(pid=80377) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 02 10:02:37.173690 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6e6883c-31b7-4499-a165-3df19f790b9a None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:02:37.175265 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6e6883c-31b7-4499-a165-3df19f790b9a None admin] RBAC: Authorization granted {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:02:37.181938 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: [pid: 80377|app: 0|req: 342/676] 10.222.0.82 () {56 vars in 1374 bytes} [Tue Aug 2 10:02:37 2022] GET /identity/v3/role_assignments?user.id=1023de281ae74e9d8a6d6c312c1c97a7&scope.project.id=f1b38ebb804c41c69772c998aae72c41&role.id=02003a0be60f47f2a8dfea17053af52c => generated 608 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 02 10:02:37.181938 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6e6883c-31b7-4499-a165-3df19f790b9a None admin] Authenticating user token {{(pid=80377) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:02:37.317858 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5380154b-e4a3-4164-9531-1dd1bda46ab7 None admin] Validating token access rules against request {{(pid=80378) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:02:37.344105 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG dbcounter [-] [80378] Writing DB stats keystone:SELECT=100 {{(pid=80378) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 10:02:37.345255 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6e6883c-31b7-4499-a165-3df19f790b9a None admin] Validating token access rules against request {{(pid=80377) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:02:37.371392 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51140100-356c-4255-95a4-2313bc2bd36d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '891005fa5ff74041b75747d16cc66798', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80378) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:02:37.372028 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51140100-356c-4255-95a4-2313bc2bd36d None admin] REQUEST_METHOD: `GET` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:02:37.372308 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51140100-356c-4255-95a4-2313bc2bd36d None admin] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:02:37.372577 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51140100-356c-4255-95a4-2313bc2bd36d None admin] PATH_INFO: `/v3/projects` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:02:37.373349 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51140100-356c-4255-95a4-2313bc2bd36d None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80378) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 02 10:02:37.373658 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51140100-356c-4255-95a4-2313bc2bd36d None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:02:37.375281 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51140100-356c-4255-95a4-2313bc2bd36d None admin] RBAC: Authorization granted {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:02:37.389347 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 335/677] 10.222.0.82 () {56 vars in 1148 bytes} [Tue Aug 2 10:02:37 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 238 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 02 10:02:37.390674 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96de4651-8c92-42d6-b0f6-cdd355e5d752 None None] REQUEST_METHOD: `POST` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:02:37.391069 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96de4651-8c92-42d6-b0f6-cdd355e5d752 None None] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:02:37.391467 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96de4651-8c92-42d6-b0f6-cdd355e5d752 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:02:37.404774 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c8d1ae7-fbf0-4747-bf81-dc896ea0e29e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '891005fa5ff74041b75747d16cc66798', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80377) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:02:37.405561 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c8d1ae7-fbf0-4747-bf81-dc896ea0e29e None admin] REQUEST_METHOD: `GET` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:02:37.405876 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c8d1ae7-fbf0-4747-bf81-dc896ea0e29e None admin] SCRIPT_NAME: `/identity` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:02:37.406155 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c8d1ae7-fbf0-4747-bf81-dc896ea0e29e None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:02:37.411977 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.auth.core [None req-96de4651-8c92-42d6-b0f6-cdd355e5d752 None None] MFA Rules not processed for user `891005fa5ff74041b75747d16cc66798`. Rule list: `[]` (Enabled: `True`). {{(pid=80378) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 02 10:02:37.413277 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.fernet_utils [None req-96de4651-8c92-42d6-b0f6-cdd355e5d752 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80378) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 10:02:37.420208 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c8d1ae7-fbf0-4747-bf81-dc896ea0e29e None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:02:37.421677 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c8d1ae7-fbf0-4747-bf81-dc896ea0e29e None admin] RBAC: Authorization granted {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:02:37.439821 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application [None req-7c8d1ae7-fbf0-4747-bf81-dc896ea0e29e None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 02 10:02:37.439821 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 02 10:02:37.439821 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 02 10:02:37.439821 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 02 10:02:37.439821 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 02 10:02:37.439821 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 02 10:02:37.439821 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 02 10:02:37.439821 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 02 10:02:37.439821 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 02 10:02:37.439821 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 02 10:02:37.439821 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 02 10:02:37.439821 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 02 10:02:37.439821 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 02 10:02:37.439821 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 02 10:02:37.439821 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 02 10:02:37.441840 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 02 10:02:37.441840 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 02 10:02:37.441840 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 02 10:02:37.441840 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 02 10:02:37.441840 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 02 10:02:37.441840 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 02 10:02:37.441840 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application return self.get_or_create( Aug 02 10:02:37.441840 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 02 10:02:37.441840 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application with Lock( Aug 02 10:02:37.441840 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 02 10:02:37.441840 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application return self._enter() Aug 02 10:02:37.441840 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 02 10:02:37.441840 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 02 10:02:37.441840 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 02 10:02:37.441840 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application return self.creator() Aug 02 10:02:37.441840 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 02 10:02:37.441840 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application created_value = creator( Aug 02 10:02:37.443071 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 02 10:02:37.443071 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 02 10:02:37.443071 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 02 10:02:37.443071 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application Aug 02 10:02:37.443071 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: [pid: 80377|app: 0|req: 343/678] 10.222.0.82 () {56 vars in 1108 bytes} [Tue Aug 2 10:02:37 2022] GET /identity/v3/domains/Default => generated 87 bytes in 259 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 02 10:02:37.444304 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c8d1ae7-fbf0-4747-bf81-dc896ea0e29e None admin] Authenticating user token {{(pid=80377) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:02:37.476298 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG dbcounter [-] [80377] Writing DB stats keystone:SELECT=100 {{(pid=80377) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 10:02:37.501040 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 336/679] 10.222.0.82 () {58 vars in 1023 bytes} [Tue Aug 2 10:02:37 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 111 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 02 10:02:37.502416 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96de4651-8c92-42d6-b0f6-cdd355e5d752 None None] Authenticating user token {{(pid=80378) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:02:37.606343 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c8d1ae7-fbf0-4747-bf81-dc896ea0e29e None admin] Validating token access rules against request {{(pid=80377) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:02:37.650294 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-406ac40e-1ac6-4787-adae-2532c4fa6006 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '891005fa5ff74041b75747d16cc66798', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80377) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:02:37.651295 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-406ac40e-1ac6-4787-adae-2532c4fa6006 None admin] REQUEST_METHOD: `GET` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:02:37.651777 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-406ac40e-1ac6-4787-adae-2532c4fa6006 None admin] SCRIPT_NAME: `/identity` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:02:37.652240 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-406ac40e-1ac6-4787-adae-2532c4fa6006 None admin] PATH_INFO: `/v3/users/swift` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:02:37.657158 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96de4651-8c92-42d6-b0f6-cdd355e5d752 None None] Validating token access rules against request {{(pid=80378) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:02:37.668534 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-406ac40e-1ac6-4787-adae-2532c4fa6006 None admin] RBAC: Authorizing `identity:get_user(user_id=swift)` {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:02:37.669902 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG dbcounter [-] [80378] Writing DB stats keystone:SELECT=100 {{(pid=80378) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 10:02:37.670221 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-406ac40e-1ac6-4787-adae-2532c4fa6006 None admin] RBAC: Authorization granted {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:02:37.687535 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application [None req-406ac40e-1ac6-4787-adae-2532c4fa6006 None admin] Could not find user: swift.: keystone.exception.UserNotFound: Could not find user: swift. Aug 02 10:02:37.687535 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 02 10:02:37.687535 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 02 10:02:37.687535 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 02 10:02:37.687535 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 02 10:02:37.687535 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 02 10:02:37.687535 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 02 10:02:37.687535 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 02 10:02:37.687535 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 02 10:02:37.687535 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 02 10:02:37.687535 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 02 10:02:37.687535 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 02 10:02:37.687535 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 02 10:02:37.687535 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 02 10:02:37.687535 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 02 10:02:37.687535 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 02 10:02:37.688603 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 02 10:02:37.688603 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 02 10:02:37.688603 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 02 10:02:37.688603 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 02 10:02:37.688603 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 02 10:02:37.688603 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 02 10:02:37.688603 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 02 10:02:37.688603 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 02 10:02:37.688603 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 02 10:02:37.688603 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application return self.get_or_create( Aug 02 10:02:37.688603 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 02 10:02:37.688603 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application with Lock( Aug 02 10:02:37.688603 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 02 10:02:37.688603 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application return self._enter() Aug 02 10:02:37.688603 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 02 10:02:37.688603 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 02 10:02:37.688603 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 02 10:02:37.688603 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application return self.creator() Aug 02 10:02:37.690132 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 02 10:02:37.690132 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application created_value = creator( Aug 02 10:02:37.690132 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 02 10:02:37.690132 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 02 10:02:37.690132 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 02 10:02:37.690132 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 02 10:02:37.690132 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 02 10:02:37.690132 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 02 10:02:37.690132 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: swift. Aug 02 10:02:37.690132 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application Aug 02 10:02:37.698174 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: [pid: 80377|app: 0|req: 344/680] 10.222.0.82 () {56 vars in 1096 bytes} [Tue Aug 2 10:02:37 2022] GET /identity/v3/users/swift => generated 83 bytes in 256 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 02 10:02:37.699708 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-406ac40e-1ac6-4787-adae-2532c4fa6006 None admin] Authenticating user token {{(pid=80377) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:02:37.716843 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e503bef-a188-4e51-8300-3e9d05c65124 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '891005fa5ff74041b75747d16cc66798', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80378) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:02:37.717903 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e503bef-a188-4e51-8300-3e9d05c65124 None admin] REQUEST_METHOD: `GET` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:02:37.718888 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e503bef-a188-4e51-8300-3e9d05c65124 None admin] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:02:37.719649 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e503bef-a188-4e51-8300-3e9d05c65124 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:02:37.721319 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e503bef-a188-4e51-8300-3e9d05c65124 None admin] RBAC: Adding query filter params (role.id=02003a0be60f47f2a8dfea17053af52c, scope.project.id=f1b38ebb804c41c69772c998aae72c41, user.id=fa964af53a2b4fb9ace6afb544d54d40) {{(pid=80378) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 02 10:02:37.722165 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e503bef-a188-4e51-8300-3e9d05c65124 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:02:37.724565 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e503bef-a188-4e51-8300-3e9d05c65124 None admin] RBAC: Authorization granted {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:02:37.741818 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 337/681] 10.222.0.82 () {56 vars in 1374 bytes} [Tue Aug 2 10:02:37 2022] GET /identity/v3/role_assignments?user.id=fa964af53a2b4fb9ace6afb544d54d40&scope.project.id=f1b38ebb804c41c69772c998aae72c41&role.id=02003a0be60f47f2a8dfea17053af52c => generated 608 bytes in 229 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 02 10:02:37.741818 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e503bef-a188-4e51-8300-3e9d05c65124 None admin] Authenticating user token {{(pid=80378) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:02:37.847956 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG dbcounter [-] [80377] Writing DB stats keystone:SELECT=100 {{(pid=80377) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 10:02:37.890277 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-406ac40e-1ac6-4787-adae-2532c4fa6006 None admin] Validating token access rules against request {{(pid=80377) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:02:37.929269 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e503bef-a188-4e51-8300-3e9d05c65124 None admin] Validating token access rules against request {{(pid=80378) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:02:37.943592 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c84d6e44-2651-4cdd-be28-e2e157d4fffb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '891005fa5ff74041b75747d16cc66798', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80377) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:02:37.944472 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c84d6e44-2651-4cdd-be28-e2e157d4fffb None admin] REQUEST_METHOD: `GET` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:02:37.944918 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c84d6e44-2651-4cdd-be28-e2e157d4fffb None admin] SCRIPT_NAME: `/identity` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:02:37.945351 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c84d6e44-2651-4cdd-be28-e2e157d4fffb None admin] PATH_INFO: `/v3/domains` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:02:37.946252 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c84d6e44-2651-4cdd-be28-e2e157d4fffb None admin] RBAC: Adding query filter params (name=Default) {{(pid=80377) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 02 10:02:37.946703 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c84d6e44-2651-4cdd-be28-e2e157d4fffb None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:02:37.948304 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c84d6e44-2651-4cdd-be28-e2e157d4fffb None admin] RBAC: Authorization granted {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:02:37.963616 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: [pid: 80377|app: 0|req: 345/682] 10.222.0.82 () {56 vars in 1109 bytes} [Tue Aug 2 10:02:37 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 265 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 02 10:02:37.964474 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c84d6e44-2651-4cdd-be28-e2e157d4fffb None admin] Authenticating user token {{(pid=80377) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:02:37.976952 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fdadc3c-a0e2-4c91-b814-2aa455f8de23 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '891005fa5ff74041b75747d16cc66798', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80378) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:02:37.978038 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fdadc3c-a0e2-4c91-b814-2aa455f8de23 None admin] REQUEST_METHOD: `GET` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:02:37.978528 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fdadc3c-a0e2-4c91-b814-2aa455f8de23 None admin] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:02:37.978991 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fdadc3c-a0e2-4c91-b814-2aa455f8de23 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:02:37.997575 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9fdadc3c-a0e2-4c91-b814-2aa455f8de23 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 02 10:02:38.003843 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application [None req-9fdadc3c-a0e2-4c91-b814-2aa455f8de23 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 02 10:02:38.003843 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 02 10:02:38.003843 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 02 10:02:38.003843 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 02 10:02:38.003843 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 02 10:02:38.003843 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 02 10:02:38.003843 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 02 10:02:38.003843 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 02 10:02:38.003843 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 02 10:02:38.003843 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 02 10:02:38.003843 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 02 10:02:38.003843 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 02 10:02:38.003843 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 02 10:02:38.003843 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 02 10:02:38.003843 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 02 10:02:38.003843 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 02 10:02:38.005041 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 02 10:02:38.005041 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 02 10:02:38.005041 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 02 10:02:38.005041 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 02 10:02:38.005041 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 02 10:02:38.005041 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 02 10:02:38.005041 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 02 10:02:38.005041 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 02 10:02:38.005041 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 02 10:02:38.005041 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application return self.get_or_create( Aug 02 10:02:38.005041 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 02 10:02:38.005041 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application with Lock( Aug 02 10:02:38.005041 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 02 10:02:38.005041 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application return self._enter() Aug 02 10:02:38.005041 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 02 10:02:38.005041 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 02 10:02:38.006127 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 02 10:02:38.006127 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application return self.creator() Aug 02 10:02:38.006127 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 02 10:02:38.006127 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application created_value = creator( Aug 02 10:02:38.006127 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 02 10:02:38.006127 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 02 10:02:38.006127 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 02 10:02:38.006127 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 02 10:02:38.006127 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 02 10:02:38.006127 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 02 10:02:38.006127 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 02 10:02:38.006127 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application Aug 02 10:02:38.010552 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 338/683] 10.222.0.82 () {56 vars in 1096 bytes} [Tue Aug 2 10:02:37 2022] GET /identity/v3/roles/admin => generated 83 bytes in 273 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 02 10:02:38.010929 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fdadc3c-a0e2-4c91-b814-2aa455f8de23 None admin] Authenticating user token {{(pid=80378) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:02:38.060876 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG dbcounter [-] [80378] Writing DB stats keystone:SELECT=100 {{(pid=80378) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 10:02:38.122509 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c84d6e44-2651-4cdd-be28-e2e157d4fffb None admin] Validating token access rules against request {{(pid=80377) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:02:38.166729 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fdadc3c-a0e2-4c91-b814-2aa455f8de23 None admin] Validating token access rules against request {{(pid=80378) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:02:38.176207 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22732c63-0706-450f-a4ab-e21f4cf507f9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '891005fa5ff74041b75747d16cc66798', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80377) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:02:38.176951 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22732c63-0706-450f-a4ab-e21f4cf507f9 None admin] REQUEST_METHOD: `GET` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:02:38.177388 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22732c63-0706-450f-a4ab-e21f4cf507f9 None admin] SCRIPT_NAME: `/identity` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:02:38.177980 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22732c63-0706-450f-a4ab-e21f4cf507f9 None admin] PATH_INFO: `/v3/users` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:02:38.179773 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22732c63-0706-450f-a4ab-e21f4cf507f9 None admin] RBAC: Adding query filter params (domain_id=default, name=swift) {{(pid=80377) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 02 10:02:38.180278 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22732c63-0706-450f-a4ab-e21f4cf507f9 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:02:38.182664 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22732c63-0706-450f-a4ab-e21f4cf507f9 None admin] RBAC: Authorization granted {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:02:38.194684 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG dbcounter [-] [80377] Writing DB stats keystone:SELECT=100 {{(pid=80377) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 10:02:38.216648 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: [pid: 80377|app: 0|req: 346/684] 10.222.0.82 () {56 vars in 1135 bytes} [Tue Aug 2 10:02:37 2022] GET /identity/v3/users?domain_id=default&name=swift => generated 370 bytes in 252 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 02 10:02:38.218382 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22732c63-0706-450f-a4ab-e21f4cf507f9 None admin] Authenticating user token {{(pid=80377) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:02:38.223982 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2bd2ca6-8486-4244-bef6-026659a95866 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '891005fa5ff74041b75747d16cc66798', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80378) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:02:38.223982 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2bd2ca6-8486-4244-bef6-026659a95866 None admin] REQUEST_METHOD: `GET` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:02:38.223982 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2bd2ca6-8486-4244-bef6-026659a95866 None admin] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:02:38.223982 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2bd2ca6-8486-4244-bef6-026659a95866 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:02:38.232962 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2bd2ca6-8486-4244-bef6-026659a95866 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:02:38.234558 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2bd2ca6-8486-4244-bef6-026659a95866 None admin] RBAC: Authorization granted {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:02:38.253392 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application [None req-d2bd2ca6-8486-4244-bef6-026659a95866 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 02 10:02:38.253392 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 02 10:02:38.253392 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 02 10:02:38.253392 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 02 10:02:38.253392 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 02 10:02:38.253392 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 02 10:02:38.253392 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 02 10:02:38.253392 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 02 10:02:38.253392 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 02 10:02:38.253392 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 02 10:02:38.253392 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 02 10:02:38.253392 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 02 10:02:38.253392 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 02 10:02:38.253392 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 02 10:02:38.253392 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 02 10:02:38.255263 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 02 10:02:38.255263 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 02 10:02:38.255263 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 02 10:02:38.255263 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 02 10:02:38.255263 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 02 10:02:38.255263 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 02 10:02:38.255263 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application return self.get_or_create( Aug 02 10:02:38.255263 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 02 10:02:38.255263 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application with Lock( Aug 02 10:02:38.255263 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 02 10:02:38.255263 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application return self._enter() Aug 02 10:02:38.255263 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 02 10:02:38.255263 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 02 10:02:38.255263 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 02 10:02:38.255263 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application return self.creator() Aug 02 10:02:38.255263 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 02 10:02:38.255263 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application created_value = creator( Aug 02 10:02:38.256676 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 02 10:02:38.256676 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 02 10:02:38.256676 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 02 10:02:38.256676 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 02 10:02:38.256676 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 02 10:02:38.256676 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 02 10:02:38.256676 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 02 10:02:38.256676 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application Aug 02 10:02:38.256676 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 339/685] 10.222.0.82 () {56 vars in 1111 bytes} [Tue Aug 2 10:02:38 2022] GET /identity/v3/projects/service => generated 88 bytes in 245 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 02 10:02:38.256676 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2bd2ca6-8486-4244-bef6-026659a95866 None admin] Authenticating user token {{(pid=80378) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:02:38.368200 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22732c63-0706-450f-a4ab-e21f4cf507f9 None admin] Validating token access rules against request {{(pid=80377) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:02:38.415168 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fab0e6f-7c40-4eb7-bff8-6e891f528e6d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '891005fa5ff74041b75747d16cc66798', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80377) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:02:38.416036 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fab0e6f-7c40-4eb7-bff8-6e891f528e6d None admin] REQUEST_METHOD: `GET` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:02:38.416418 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fab0e6f-7c40-4eb7-bff8-6e891f528e6d None admin] SCRIPT_NAME: `/identity` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:02:38.416877 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fab0e6f-7c40-4eb7-bff8-6e891f528e6d None admin] PATH_INFO: `/v3/roles` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:02:38.417766 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5fab0e6f-7c40-4eb7-bff8-6e891f528e6d None admin] RBAC: Adding query filter params (name=admin) {{(pid=80377) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 02 10:02:38.418213 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5fab0e6f-7c40-4eb7-bff8-6e891f528e6d None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:02:38.420833 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5fab0e6f-7c40-4eb7-bff8-6e891f528e6d None admin] RBAC: Authorization granted {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:02:38.424047 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG dbcounter [-] [80378] Writing DB stats keystone:SELECT=100 {{(pid=80378) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 10:02:38.434601 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: [pid: 80377|app: 0|req: 347/686] 10.222.0.82 () {56 vars in 1099 bytes} [Tue Aug 2 10:02:38 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 217 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 02 10:02:38.435607 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fab0e6f-7c40-4eb7-bff8-6e891f528e6d None admin] Authenticating user token {{(pid=80377) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:02:38.436336 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2bd2ca6-8486-4244-bef6-026659a95866 None admin] Validating token access rules against request {{(pid=80378) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:02:38.479873 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a5dd84d-80d3-4022-814c-1d90cfcef4fb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '891005fa5ff74041b75747d16cc66798', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80378) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:02:38.480930 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a5dd84d-80d3-4022-814c-1d90cfcef4fb None admin] REQUEST_METHOD: `GET` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:02:38.481403 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a5dd84d-80d3-4022-814c-1d90cfcef4fb None admin] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:02:38.481923 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a5dd84d-80d3-4022-814c-1d90cfcef4fb None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:02:38.504337 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG dbcounter [-] [80377] Writing DB stats keystone:SELECT=100 {{(pid=80377) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 10:02:38.555952 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fab0e6f-7c40-4eb7-bff8-6e891f528e6d None admin] Validating token access rules against request {{(pid=80377) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:02:38.571431 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a5dd84d-80d3-4022-814c-1d90cfcef4fb None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:02:38.573935 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a5dd84d-80d3-4022-814c-1d90cfcef4fb None admin] RBAC: Authorization granted {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:02:38.597890 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2498c93-0d2c-472d-927e-8dda4381710c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '891005fa5ff74041b75747d16cc66798', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80377) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:02:38.598687 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2498c93-0d2c-472d-927e-8dda4381710c None admin] REQUEST_METHOD: `GET` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:02:38.599457 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2498c93-0d2c-472d-927e-8dda4381710c None admin] SCRIPT_NAME: `/identity` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:02:38.599457 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2498c93-0d2c-472d-927e-8dda4381710c None admin] PATH_INFO: `/v3/projects` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:02:38.600255 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2498c93-0d2c-472d-927e-8dda4381710c None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80377) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 02 10:02:38.600644 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2498c93-0d2c-472d-927e-8dda4381710c None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:02:38.602275 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2498c93-0d2c-472d-927e-8dda4381710c None admin] RBAC: Authorization granted {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:02:38.618292 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: [pid: 80377|app: 0|req: 348/687] 10.222.0.82 () {56 vars in 1148 bytes} [Tue Aug 2 10:02:38 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 183 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 02 10:02:38.619406 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2498c93-0d2c-472d-927e-8dda4381710c None admin] Authenticating user token {{(pid=80377) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:02:38.697742 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG dbcounter [-] [80378] Writing DB stats keystone:SELECT=100 {{(pid=80378) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 10:02:38.717836 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 340/688] 10.222.0.82 () {58 vars in 1282 bytes} [Tue Aug 2 10:02:38 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 456 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 02 10:02:38.719315 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a5dd84d-80d3-4022-814c-1d90cfcef4fb None admin] Authenticating user token {{(pid=80378) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:02:38.742593 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2498c93-0d2c-472d-927e-8dda4381710c None admin] Validating token access rules against request {{(pid=80377) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:02:38.766515 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG dbcounter [-] [80377] Writing DB stats keystone:SELECT=100 {{(pid=80377) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 10:02:38.788522 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-438d857f-455a-4eb1-a099-6d0aba9331f3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '891005fa5ff74041b75747d16cc66798', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80377) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:02:38.789314 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-438d857f-455a-4eb1-a099-6d0aba9331f3 None admin] REQUEST_METHOD: `GET` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:02:38.789663 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-438d857f-455a-4eb1-a099-6d0aba9331f3 None admin] SCRIPT_NAME: `/identity` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:02:38.790042 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-438d857f-455a-4eb1-a099-6d0aba9331f3 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:02:38.802504 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-438d857f-455a-4eb1-a099-6d0aba9331f3 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:02:38.804211 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-438d857f-455a-4eb1-a099-6d0aba9331f3 None admin] RBAC: Authorization granted {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:02:38.823279 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application [None req-438d857f-455a-4eb1-a099-6d0aba9331f3 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 02 10:02:38.823279 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 02 10:02:38.823279 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 02 10:02:38.823279 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 02 10:02:38.823279 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 02 10:02:38.823279 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 02 10:02:38.823279 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 02 10:02:38.823279 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 02 10:02:38.823279 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 02 10:02:38.823279 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 02 10:02:38.823279 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 02 10:02:38.823279 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 02 10:02:38.823279 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 02 10:02:38.823279 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 02 10:02:38.823279 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 02 10:02:38.824261 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 02 10:02:38.824261 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 02 10:02:38.824261 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 02 10:02:38.824261 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 02 10:02:38.824261 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 02 10:02:38.824261 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 02 10:02:38.824261 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application return self.get_or_create( Aug 02 10:02:38.824261 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 02 10:02:38.824261 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application with Lock( Aug 02 10:02:38.824261 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 02 10:02:38.824261 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application return self._enter() Aug 02 10:02:38.824261 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 02 10:02:38.824261 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 02 10:02:38.824261 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 02 10:02:38.824261 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application return self.creator() Aug 02 10:02:38.824261 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 02 10:02:38.824261 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application created_value = creator( Aug 02 10:02:38.825188 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 02 10:02:38.825188 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 02 10:02:38.825188 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 02 10:02:38.825188 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application Aug 02 10:02:38.825188 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: [pid: 80377|app: 0|req: 349/689] 10.222.0.82 () {56 vars in 1108 bytes} [Tue Aug 2 10:02:38 2022] GET /identity/v3/domains/Default => generated 87 bytes in 206 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 02 10:02:38.826267 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-438d857f-455a-4eb1-a099-6d0aba9331f3 None admin] Authenticating user token {{(pid=80377) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:02:38.854926 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a5dd84d-80d3-4022-814c-1d90cfcef4fb None admin] Validating token access rules against request {{(pid=80378) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:02:38.904662 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2544f9f-d04a-4739-902c-9b2adbb4f6bc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '891005fa5ff74041b75747d16cc66798', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80378) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:02:38.905707 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2544f9f-d04a-4739-902c-9b2adbb4f6bc None admin] REQUEST_METHOD: `GET` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:02:38.906320 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2544f9f-d04a-4739-902c-9b2adbb4f6bc None admin] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:02:38.906778 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2544f9f-d04a-4739-902c-9b2adbb4f6bc None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:02:38.908595 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2544f9f-d04a-4739-902c-9b2adbb4f6bc None admin] RBAC: Adding query filter params (role.id=02003a0be60f47f2a8dfea17053af52c, scope.project.id=f1b38ebb804c41c69772c998aae72c41, user.id=33b87170f0634b67bce60750847b5cbe) {{(pid=80378) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 02 10:02:38.909098 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2544f9f-d04a-4739-902c-9b2adbb4f6bc None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:02:38.910910 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2544f9f-d04a-4739-902c-9b2adbb4f6bc None admin] RBAC: Authorization granted {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:02:38.918376 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 341/690] 10.222.0.82 () {56 vars in 1374 bytes} [Tue Aug 2 10:02:38 2022] GET /identity/v3/role_assignments?user.id=33b87170f0634b67bce60750847b5cbe&scope.project.id=f1b38ebb804c41c69772c998aae72c41&role.id=02003a0be60f47f2a8dfea17053af52c => generated 608 bytes in 200 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 02 10:02:38.919674 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2544f9f-d04a-4739-902c-9b2adbb4f6bc None admin] Authenticating user token {{(pid=80378) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:02:38.967927 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-438d857f-455a-4eb1-a099-6d0aba9331f3 None admin] Validating token access rules against request {{(pid=80377) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:02:38.986245 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG dbcounter [-] [80378] Writing DB stats keystone:SELECT=100 {{(pid=80378) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 10:02:39.039922 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2544f9f-d04a-4739-902c-9b2adbb4f6bc None admin] Validating token access rules against request {{(pid=80378) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:02:39.045749 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06394ea7-8c16-4871-b108-47ef80784280 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '891005fa5ff74041b75747d16cc66798', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80377) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:02:39.046512 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06394ea7-8c16-4871-b108-47ef80784280 None admin] REQUEST_METHOD: `GET` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:02:39.046801 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06394ea7-8c16-4871-b108-47ef80784280 None admin] SCRIPT_NAME: `/identity` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:02:39.047076 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06394ea7-8c16-4871-b108-47ef80784280 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:02:39.070900 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06394ea7-8c16-4871-b108-47ef80784280 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:02:39.072476 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06394ea7-8c16-4871-b108-47ef80784280 None admin] RBAC: Authorization granted {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:02:39.084366 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e2c49d7-57bb-49d8-8421-26c14b11ed2b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '891005fa5ff74041b75747d16cc66798', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80378) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:02:39.085924 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e2c49d7-57bb-49d8-8421-26c14b11ed2b None admin] REQUEST_METHOD: `GET` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:02:39.086446 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e2c49d7-57bb-49d8-8421-26c14b11ed2b None admin] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:02:39.086955 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e2c49d7-57bb-49d8-8421-26c14b11ed2b None admin] PATH_INFO: `/v3/domains` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:02:39.087916 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e2c49d7-57bb-49d8-8421-26c14b11ed2b None admin] RBAC: Adding query filter params (name=Default) {{(pid=80378) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 02 10:02:39.088484 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e2c49d7-57bb-49d8-8421-26c14b11ed2b None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:02:39.090197 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e2c49d7-57bb-49d8-8421-26c14b11ed2b None admin] RBAC: Authorization granted {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:02:39.097724 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application [None req-06394ea7-8c16-4871-b108-47ef80784280 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 02 10:02:39.097724 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 02 10:02:39.097724 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 02 10:02:39.097724 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 02 10:02:39.097724 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 02 10:02:39.097724 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 02 10:02:39.097724 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 02 10:02:39.097724 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 02 10:02:39.097724 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 02 10:02:39.097724 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 02 10:02:39.097724 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 02 10:02:39.097724 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 02 10:02:39.097724 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 02 10:02:39.097724 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 02 10:02:39.097724 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 02 10:02:39.098834 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 02 10:02:39.098834 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 02 10:02:39.098834 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 02 10:02:39.098834 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 02 10:02:39.098834 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 02 10:02:39.098834 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 02 10:02:39.098834 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application return self.get_or_create( Aug 02 10:02:39.098834 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 02 10:02:39.098834 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application with Lock( Aug 02 10:02:39.098834 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 02 10:02:39.098834 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application return self._enter() Aug 02 10:02:39.098834 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 02 10:02:39.098834 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 02 10:02:39.098834 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 02 10:02:39.098834 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application return self.creator() Aug 02 10:02:39.098834 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 02 10:02:39.098834 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application created_value = creator( Aug 02 10:02:39.099750 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 02 10:02:39.099750 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 02 10:02:39.099750 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 02 10:02:39.099750 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application Aug 02 10:02:39.105544 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: [pid: 80377|app: 0|req: 350/691] 10.222.0.82 () {56 vars in 1108 bytes} [Tue Aug 2 10:02:38 2022] GET /identity/v3/domains/Default => generated 87 bytes in 274 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 02 10:02:39.107164 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06394ea7-8c16-4871-b108-47ef80784280 None admin] Authenticating user token {{(pid=80377) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:02:39.113638 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 342/692] 10.222.0.82 () {56 vars in 1109 bytes} [Tue Aug 2 10:02:38 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 194 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 02 10:02:39.118894 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e2c49d7-57bb-49d8-8421-26c14b11ed2b None admin] Authenticating user token {{(pid=80378) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:02:39.157681 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG dbcounter [-] [80377] Writing DB stats keystone:SELECT=100 {{(pid=80377) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 10:02:39.242584 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e2c49d7-57bb-49d8-8421-26c14b11ed2b None admin] Validating token access rules against request {{(pid=80378) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:02:39.262778 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG dbcounter [-] [80378] Writing DB stats keystone:SELECT=100 {{(pid=80378) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 10:02:39.269444 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06394ea7-8c16-4871-b108-47ef80784280 None admin] Validating token access rules against request {{(pid=80377) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:02:39.295292 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23a1b2fe-cdab-41b2-84b4-5d02aef1ffec None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '891005fa5ff74041b75747d16cc66798', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80378) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:02:39.296596 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23a1b2fe-cdab-41b2-84b4-5d02aef1ffec None admin] REQUEST_METHOD: `GET` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:02:39.297159 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23a1b2fe-cdab-41b2-84b4-5d02aef1ffec None admin] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:02:39.297570 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23a1b2fe-cdab-41b2-84b4-5d02aef1ffec None admin] PATH_INFO: `/v3/users/nova` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:02:39.306659 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23a1b2fe-cdab-41b2-84b4-5d02aef1ffec None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:02:39.308306 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23a1b2fe-cdab-41b2-84b4-5d02aef1ffec None admin] RBAC: Authorization granted {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:02:39.313427 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55819131-849b-4453-b797-ce313878498c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '891005fa5ff74041b75747d16cc66798', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80377) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:02:39.314449 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55819131-849b-4453-b797-ce313878498c None admin] REQUEST_METHOD: `GET` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:02:39.314874 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55819131-849b-4453-b797-ce313878498c None admin] SCRIPT_NAME: `/identity` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:02:39.315289 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55819131-849b-4453-b797-ce313878498c None admin] PATH_INFO: `/v3/domains` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:02:39.316137 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55819131-849b-4453-b797-ce313878498c None admin] RBAC: Adding query filter params (name=Default) {{(pid=80377) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 02 10:02:39.316589 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55819131-849b-4453-b797-ce313878498c None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:02:39.318181 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55819131-849b-4453-b797-ce313878498c None admin] RBAC: Authorization granted {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:02:39.323175 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application [None req-23a1b2fe-cdab-41b2-84b4-5d02aef1ffec None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Aug 02 10:02:39.323175 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 02 10:02:39.323175 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 02 10:02:39.323175 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 02 10:02:39.323175 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 02 10:02:39.323175 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 02 10:02:39.323175 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 02 10:02:39.323175 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 02 10:02:39.323175 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 02 10:02:39.323175 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 02 10:02:39.323175 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 02 10:02:39.323175 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 02 10:02:39.323175 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 02 10:02:39.323175 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 02 10:02:39.323175 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 02 10:02:39.323175 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 02 10:02:39.324376 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 02 10:02:39.324376 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 02 10:02:39.324376 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 02 10:02:39.324376 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 02 10:02:39.324376 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 02 10:02:39.324376 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 02 10:02:39.324376 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 02 10:02:39.324376 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 02 10:02:39.324376 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 02 10:02:39.324376 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application return self.get_or_create( Aug 02 10:02:39.324376 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 02 10:02:39.324376 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application with Lock( Aug 02 10:02:39.324376 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 02 10:02:39.324376 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application return self._enter() Aug 02 10:02:39.324376 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 02 10:02:39.324376 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 02 10:02:39.324376 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 02 10:02:39.324376 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application return self.creator() Aug 02 10:02:39.325753 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 02 10:02:39.325753 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application created_value = creator( Aug 02 10:02:39.325753 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 02 10:02:39.325753 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 02 10:02:39.325753 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 02 10:02:39.325753 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 02 10:02:39.325753 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 02 10:02:39.325753 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 02 10:02:39.325753 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Aug 02 10:02:39.325753 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application Aug 02 10:02:39.327862 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 343/693] 10.222.0.82 () {56 vars in 1093 bytes} [Tue Aug 2 10:02:39 2022] GET /identity/v3/users/nova => generated 82 bytes in 209 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 02 10:02:39.337081 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23a1b2fe-cdab-41b2-84b4-5d02aef1ffec None admin] Authenticating user token {{(pid=80378) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:02:39.341735 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: [pid: 80377|app: 0|req: 351/694] 10.222.0.82 () {56 vars in 1109 bytes} [Tue Aug 2 10:02:39 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 235 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 02 10:02:39.351403 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55819131-849b-4453-b797-ce313878498c None admin] Authenticating user token {{(pid=80377) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:02:39.452997 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG dbcounter [-] [80377] Writing DB stats keystone:SELECT=100 {{(pid=80377) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 10:02:39.479305 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55819131-849b-4453-b797-ce313878498c None admin] Validating token access rules against request {{(pid=80377) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:02:39.537996 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56cc15ff-73da-4870-94bb-822e44034073 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '891005fa5ff74041b75747d16cc66798', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80377) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:02:39.539969 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56cc15ff-73da-4870-94bb-822e44034073 None admin] REQUEST_METHOD: `GET` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:02:39.541036 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56cc15ff-73da-4870-94bb-822e44034073 None admin] SCRIPT_NAME: `/identity` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:02:39.542371 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56cc15ff-73da-4870-94bb-822e44034073 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:02:39.567138 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56cc15ff-73da-4870-94bb-822e44034073 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:02:39.570100 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56cc15ff-73da-4870-94bb-822e44034073 None admin] RBAC: Authorization granted {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:02:39.574712 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23a1b2fe-cdab-41b2-84b4-5d02aef1ffec None admin] Validating token access rules against request {{(pid=80378) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:02:39.604047 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application [None req-56cc15ff-73da-4870-94bb-822e44034073 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 02 10:02:39.604047 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 02 10:02:39.604047 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 02 10:02:39.604047 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 02 10:02:39.604047 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 02 10:02:39.604047 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 02 10:02:39.604047 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 02 10:02:39.604047 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 02 10:02:39.604047 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 02 10:02:39.604047 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 02 10:02:39.604047 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 02 10:02:39.604047 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 02 10:02:39.604047 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 02 10:02:39.604047 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 02 10:02:39.604047 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 02 10:02:39.605130 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 02 10:02:39.605130 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 02 10:02:39.605130 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 02 10:02:39.605130 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 02 10:02:39.605130 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 02 10:02:39.605130 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 02 10:02:39.605130 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application return self.get_or_create( Aug 02 10:02:39.605130 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 02 10:02:39.605130 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application with Lock( Aug 02 10:02:39.605130 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 02 10:02:39.605130 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application return self._enter() Aug 02 10:02:39.605130 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 02 10:02:39.605130 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 02 10:02:39.605130 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 02 10:02:39.605130 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application return self.creator() Aug 02 10:02:39.605130 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 02 10:02:39.605130 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application created_value = creator( Aug 02 10:02:39.606306 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 02 10:02:39.606306 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 02 10:02:39.606306 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 02 10:02:39.606306 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 02 10:02:39.606306 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 02 10:02:39.606306 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 02 10:02:39.606306 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 02 10:02:39.606306 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application Aug 02 10:02:39.609939 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: [pid: 80377|app: 0|req: 352/695] 10.222.0.82 () {56 vars in 1111 bytes} [Tue Aug 2 10:02:39 2022] GET /identity/v3/projects/service => generated 88 bytes in 255 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 02 10:02:39.612804 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56cc15ff-73da-4870-94bb-822e44034073 None admin] Authenticating user token {{(pid=80377) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:02:39.642499 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c207a10-1594-45c2-b8b9-55dafcd97a85 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '891005fa5ff74041b75747d16cc66798', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80378) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:02:39.643741 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c207a10-1594-45c2-b8b9-55dafcd97a85 None admin] REQUEST_METHOD: `GET` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:02:39.644343 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c207a10-1594-45c2-b8b9-55dafcd97a85 None admin] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:02:39.644961 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c207a10-1594-45c2-b8b9-55dafcd97a85 None admin] PATH_INFO: `/v3/users` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:02:39.646268 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c207a10-1594-45c2-b8b9-55dafcd97a85 None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=80378) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 02 10:02:39.646894 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c207a10-1594-45c2-b8b9-55dafcd97a85 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:02:39.651036 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c207a10-1594-45c2-b8b9-55dafcd97a85 None admin] RBAC: Authorization granted {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:02:39.694724 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 344/696] 10.222.0.82 () {56 vars in 1133 bytes} [Tue Aug 2 10:02:39 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 358 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 02 10:02:39.701011 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c207a10-1594-45c2-b8b9-55dafcd97a85 None admin] Authenticating user token {{(pid=80378) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:02:39.799031 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG dbcounter [-] [80378] Writing DB stats keystone:SELECT=100 {{(pid=80378) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 10:02:39.816669 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56cc15ff-73da-4870-94bb-822e44034073 None admin] Validating token access rules against request {{(pid=80377) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:02:39.887220 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8dcce1df-fcc2-412a-8ec0-ecd90f42eae2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '891005fa5ff74041b75747d16cc66798', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80377) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:02:39.887957 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8dcce1df-fcc2-412a-8ec0-ecd90f42eae2 None admin] REQUEST_METHOD: `GET` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:02:39.888640 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8dcce1df-fcc2-412a-8ec0-ecd90f42eae2 None admin] SCRIPT_NAME: `/identity` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:02:39.888640 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8dcce1df-fcc2-412a-8ec0-ecd90f42eae2 None admin] PATH_INFO: `/v3/projects` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:02:39.889273 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8dcce1df-fcc2-412a-8ec0-ecd90f42eae2 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80377) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 02 10:02:39.889586 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8dcce1df-fcc2-412a-8ec0-ecd90f42eae2 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:02:39.891080 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8dcce1df-fcc2-412a-8ec0-ecd90f42eae2 None admin] RBAC: Authorization granted {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:02:39.905903 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c207a10-1594-45c2-b8b9-55dafcd97a85 None admin] Validating token access rules against request {{(pid=80378) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:02:39.912699 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: [pid: 80377|app: 0|req: 353/697] 10.222.0.82 () {56 vars in 1148 bytes} [Tue Aug 2 10:02:39 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 294 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 02 10:02:39.914415 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdb6d642-5b27-4d14-a240-a009b2490443 None None] REQUEST_METHOD: `GET` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:02:39.914715 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdb6d642-5b27-4d14-a240-a009b2490443 None None] SCRIPT_NAME: `/identity` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:02:39.915086 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdb6d642-5b27-4d14-a240-a009b2490443 None None] PATH_INFO: `/` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:02:39.921700 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: [pid: 80377|app: 0|req: 354/698] 10.222.0.82 () {54 vars in 926 bytes} [Tue Aug 2 10:02:39 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 02 10:02:39.921700 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fdb6d642-5b27-4d14-a240-a009b2490443 None None] Authenticating user token {{(pid=80377) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:02:39.922517 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG dbcounter [-] [80377] Writing DB stats keystone:SELECT=100 {{(pid=80377) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 10:02:39.972822 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c30c0d74-9433-4683-b513-14eb6fd584ba None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '891005fa5ff74041b75747d16cc66798', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80378) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:02:39.974128 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c30c0d74-9433-4683-b513-14eb6fd584ba None admin] REQUEST_METHOD: `GET` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:02:39.975344 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c30c0d74-9433-4683-b513-14eb6fd584ba None admin] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:02:39.975909 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c30c0d74-9433-4683-b513-14eb6fd584ba None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:02:40.098903 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fdb6d642-5b27-4d14-a240-a009b2490443 None None] Validating token access rules against request {{(pid=80377) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:02:40.115302 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c30c0d74-9433-4683-b513-14eb6fd584ba None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:02:40.116982 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c30c0d74-9433-4683-b513-14eb6fd584ba None admin] RBAC: Authorization granted {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:02:40.146940 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a89134ee-8594-4975-8995-6a7433de0635 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '891005fa5ff74041b75747d16cc66798', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80377) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:02:40.147844 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a89134ee-8594-4975-8995-6a7433de0635 None admin] REQUEST_METHOD: `GET` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:02:40.148297 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a89134ee-8594-4975-8995-6a7433de0635 None admin] SCRIPT_NAME: `/identity` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:02:40.148724 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a89134ee-8594-4975-8995-6a7433de0635 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:02:40.149688 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a89134ee-8594-4975-8995-6a7433de0635 None admin] RBAC: Adding query filter params (role.id=02003a0be60f47f2a8dfea17053af52c, scope.project.id=f1b38ebb804c41c69772c998aae72c41, user.id=bef1b2e714cb45c4a51eb6d3e15aad6d) {{(pid=80377) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 02 10:02:40.151025 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a89134ee-8594-4975-8995-6a7433de0635 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:02:40.153727 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a89134ee-8594-4975-8995-6a7433de0635 None admin] RBAC: Authorization granted {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:02:40.165593 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG dbcounter [-] [80378] Writing DB stats keystone:SELECT=100 {{(pid=80378) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 10:02:40.170838 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: [pid: 80377|app: 0|req: 355/699] 10.222.0.82 () {56 vars in 1374 bytes} [Tue Aug 2 10:02:39 2022] GET /identity/v3/role_assignments?user.id=bef1b2e714cb45c4a51eb6d3e15aad6d&scope.project.id=f1b38ebb804c41c69772c998aae72c41&role.id=02003a0be60f47f2a8dfea17053af52c => generated 608 bytes in 253 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 02 10:02:40.173502 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2230cebe-d47f-4ccf-995a-b85d24cb608f None None] REQUEST_METHOD: `POST` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:02:40.174251 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2230cebe-d47f-4ccf-995a-b85d24cb608f None None] SCRIPT_NAME: `/identity` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:02:40.174949 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2230cebe-d47f-4ccf-995a-b85d24cb608f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:02:40.206119 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.auth.core [None req-2230cebe-d47f-4ccf-995a-b85d24cb608f None None] MFA Rules not processed for user `891005fa5ff74041b75747d16cc66798`. Rule list: `[]` (Enabled: `True`). {{(pid=80377) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 02 10:02:40.207850 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.fernet_utils [None req-2230cebe-d47f-4ccf-995a-b85d24cb608f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80377) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 10:02:40.312596 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG dbcounter [-] [80377] Writing DB stats keystone:SELECT=100 {{(pid=80377) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 10:02:40.334378 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: [pid: 80377|app: 0|req: 356/700] 10.222.0.82 () {58 vars in 1023 bytes} [Tue Aug 2 10:02:40 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 157 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 02 10:02:40.356368 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 345/701] 10.222.0.82 () {58 vars in 1282 bytes} [Tue Aug 2 10:02:39 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 656 msecs (HTTP/1.1 200) 6 headers in 358 bytes (2 switches on core 0) Aug 02 10:02:40.363189 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2230cebe-d47f-4ccf-995a-b85d24cb608f None None] Authenticating user token {{(pid=80377) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:02:40.387571 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c30c0d74-9433-4683-b513-14eb6fd584ba None admin] Authenticating user token {{(pid=80378) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:02:40.561075 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c30c0d74-9433-4683-b513-14eb6fd584ba None admin] Validating token access rules against request {{(pid=80378) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:02:40.572635 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2230cebe-d47f-4ccf-995a-b85d24cb608f None None] Validating token access rules against request {{(pid=80377) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:02:40.611958 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG dbcounter [-] [80378] Writing DB stats keystone:SELECT=100 {{(pid=80378) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 10:02:40.622571 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe4b94e3-4b5f-400e-ba5f-db10b2010d1c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '891005fa5ff74041b75747d16cc66798', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80378) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:02:40.623843 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe4b94e3-4b5f-400e-ba5f-db10b2010d1c None admin] REQUEST_METHOD: `GET` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:02:40.624302 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe4b94e3-4b5f-400e-ba5f-db10b2010d1c None admin] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:02:40.624718 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe4b94e3-4b5f-400e-ba5f-db10b2010d1c None admin] PATH_INFO: `/v3/services/block-storage` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:02:40.625801 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe4b94e3-4b5f-400e-ba5f-db10b2010d1c None admin] RBAC: Authorizing `identity:get_service(service_id=block-storage)` {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:02:40.627724 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe4b94e3-4b5f-400e-ba5f-db10b2010d1c None admin] RBAC: Authorization granted {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:02:40.640161 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6988465-3891-44fd-98a0-74c779918400 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '891005fa5ff74041b75747d16cc66798', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80377) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:02:40.640767 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6988465-3891-44fd-98a0-74c779918400 None admin] REQUEST_METHOD: `GET` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:02:40.641053 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6988465-3891-44fd-98a0-74c779918400 None admin] SCRIPT_NAME: `/identity` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:02:40.641432 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6988465-3891-44fd-98a0-74c779918400 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:02:40.655314 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6988465-3891-44fd-98a0-74c779918400 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:02:40.655509 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application [None req-fe4b94e3-4b5f-400e-ba5f-db10b2010d1c None admin] Could not find service: block-storage.: keystone.exception.ServiceNotFound: Could not find service: block-storage. Aug 02 10:02:40.655509 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 02 10:02:40.655509 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Aug 02 10:02:40.655509 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Aug 02 10:02:40.655509 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Aug 02 10:02:40.655509 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Aug 02 10:02:40.655509 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Aug 02 10:02:40.655509 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 02 10:02:40.655509 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: block-storage. Aug 02 10:02:40.655509 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application Aug 02 10:02:40.655509 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 02 10:02:40.655509 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application Aug 02 10:02:40.655509 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 02 10:02:40.655509 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 02 10:02:40.655509 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 02 10:02:40.655509 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 02 10:02:40.656538 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 02 10:02:40.656538 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 02 10:02:40.656538 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 02 10:02:40.656538 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 02 10:02:40.656538 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 02 10:02:40.656538 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 02 10:02:40.656538 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 02 10:02:40.656538 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Aug 02 10:02:40.656538 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application return self._get_service(service_id) Aug 02 10:02:40.656538 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Aug 02 10:02:40.656538 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Aug 02 10:02:40.656538 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 02 10:02:40.656538 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 02 10:02:40.656538 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 02 10:02:40.656538 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 02 10:02:40.656538 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 02 10:02:40.658886 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application return self.get_or_create( Aug 02 10:02:40.658886 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 02 10:02:40.658886 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application with Lock( Aug 02 10:02:40.658886 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 02 10:02:40.658886 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application return self._enter() Aug 02 10:02:40.658886 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 02 10:02:40.658886 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 02 10:02:40.658886 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 02 10:02:40.658886 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application return self.creator() Aug 02 10:02:40.658886 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 02 10:02:40.658886 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application created_value = creator( Aug 02 10:02:40.658886 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Aug 02 10:02:40.658886 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 02 10:02:40.658886 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: block-storage. Aug 02 10:02:40.658886 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application Aug 02 10:02:40.659797 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6988465-3891-44fd-98a0-74c779918400 None admin] RBAC: Authorization granted {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:02:40.659894 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 346/702] 10.222.0.82 () {56 vars in 1129 bytes} [Tue Aug 2 10:02:40 2022] GET /identity/v3/services/block-storage => generated 94 bytes in 268 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 02 10:02:40.659894 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90f23925-be68-46b0-8b5f-df4183113344 None None] REQUEST_METHOD: `GET` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:02:40.659894 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90f23925-be68-46b0-8b5f-df4183113344 None None] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:02:40.659894 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90f23925-be68-46b0-8b5f-df4183113344 None None] PATH_INFO: `/` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:02:40.659894 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 347/703] 10.222.0.82 () {54 vars in 926 bytes} [Tue Aug 2 10:02:40 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 02 10:02:40.666548 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90f23925-be68-46b0-8b5f-df4183113344 None None] Authenticating user token {{(pid=80378) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:02:40.674349 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application [None req-b6988465-3891-44fd-98a0-74c779918400 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 02 10:02:40.674349 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 02 10:02:40.674349 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 02 10:02:40.674349 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 02 10:02:40.674349 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 02 10:02:40.674349 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 02 10:02:40.674349 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 02 10:02:40.674349 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 02 10:02:40.674349 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 02 10:02:40.674349 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 02 10:02:40.674349 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 02 10:02:40.674349 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 02 10:02:40.674349 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 02 10:02:40.674349 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 02 10:02:40.674349 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 02 10:02:40.675643 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 02 10:02:40.675643 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 02 10:02:40.675643 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 02 10:02:40.675643 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 02 10:02:40.675643 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 02 10:02:40.675643 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 02 10:02:40.675643 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application return self.get_or_create( Aug 02 10:02:40.675643 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 02 10:02:40.675643 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application with Lock( Aug 02 10:02:40.675643 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 02 10:02:40.675643 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application return self._enter() Aug 02 10:02:40.675643 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 02 10:02:40.675643 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 02 10:02:40.675643 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 02 10:02:40.675643 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application return self.creator() Aug 02 10:02:40.675643 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 02 10:02:40.675643 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application created_value = creator( Aug 02 10:02:40.677465 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 02 10:02:40.677465 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 02 10:02:40.677465 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 02 10:02:40.677465 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application Aug 02 10:02:40.677465 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: [pid: 80377|app: 0|req: 357/704] 10.222.0.82 () {56 vars in 1108 bytes} [Tue Aug 2 10:02:40 2022] GET /identity/v3/domains/Default => generated 87 bytes in 313 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 02 10:02:40.677465 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7acaa3b4-7ca1-47d7-b1ab-12da4061761a None None] REQUEST_METHOD: `POST` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:02:40.678244 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7acaa3b4-7ca1-47d7-b1ab-12da4061761a None None] SCRIPT_NAME: `/identity` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:02:40.678244 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7acaa3b4-7ca1-47d7-b1ab-12da4061761a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:02:40.702532 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.auth.core [None req-7acaa3b4-7ca1-47d7-b1ab-12da4061761a None None] MFA Rules not processed for user `891005fa5ff74041b75747d16cc66798`. Rule list: `[]` (Enabled: `True`). {{(pid=80377) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 02 10:02:40.704190 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.fernet_utils [None req-7acaa3b4-7ca1-47d7-b1ab-12da4061761a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80377) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 10:02:40.778677 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG dbcounter [-] [80377] Writing DB stats keystone:SELECT=100 {{(pid=80377) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 10:02:40.818820 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: [pid: 80377|app: 0|req: 358/705] 10.222.0.82 () {58 vars in 1023 bytes} [Tue Aug 2 10:02:40 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 142 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 02 10:02:40.819537 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7acaa3b4-7ca1-47d7-b1ab-12da4061761a None None] Authenticating user token {{(pid=80377) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:02:40.836878 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90f23925-be68-46b0-8b5f-df4183113344 None None] Validating token access rules against request {{(pid=80378) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:02:40.892128 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed151ef1-101a-4422-83ea-579bedd6daf9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '891005fa5ff74041b75747d16cc66798', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80378) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:02:40.893071 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed151ef1-101a-4422-83ea-579bedd6daf9 None admin] REQUEST_METHOD: `GET` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:02:40.893502 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed151ef1-101a-4422-83ea-579bedd6daf9 None admin] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:02:40.893978 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed151ef1-101a-4422-83ea-579bedd6daf9 None admin] PATH_INFO: `/v3/services` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:02:40.894826 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed151ef1-101a-4422-83ea-579bedd6daf9 None admin] RBAC: Adding query filter params (name=block-storage) {{(pid=80378) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 02 10:02:40.895286 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed151ef1-101a-4422-83ea-579bedd6daf9 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:02:40.897078 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed151ef1-101a-4422-83ea-579bedd6daf9 None admin] RBAC: Authorization granted {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:02:40.908367 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 348/706] 10.222.0.82 () {56 vars in 1124 bytes} [Tue Aug 2 10:02:40 2022] GET /identity/v3/services?name=block-storage => generated 130 bytes in 238 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 02 10:02:40.909934 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed151ef1-101a-4422-83ea-579bedd6daf9 None admin] Authenticating user token {{(pid=80378) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:02:40.961546 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7acaa3b4-7ca1-47d7-b1ab-12da4061761a None None] Validating token access rules against request {{(pid=80377) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:02:41.010579 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bba5f35-3c18-4ba4-bc52-65b585373691 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '891005fa5ff74041b75747d16cc66798', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80377) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:02:41.011618 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bba5f35-3c18-4ba4-bc52-65b585373691 None admin] REQUEST_METHOD: `GET` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:02:41.011618 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bba5f35-3c18-4ba4-bc52-65b585373691 None admin] SCRIPT_NAME: `/identity` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:02:41.013353 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bba5f35-3c18-4ba4-bc52-65b585373691 None admin] PATH_INFO: `/v3/domains` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:02:41.014164 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1bba5f35-3c18-4ba4-bc52-65b585373691 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80377) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 02 10:02:41.014516 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1bba5f35-3c18-4ba4-bc52-65b585373691 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:02:41.015967 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1bba5f35-3c18-4ba4-bc52-65b585373691 None admin] RBAC: Authorization granted {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:02:41.033832 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: [pid: 80377|app: 0|req: 359/707] 10.222.0.82 () {56 vars in 1109 bytes} [Tue Aug 2 10:02:40 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 211 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 02 10:02:41.033832 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bba5f35-3c18-4ba4-bc52-65b585373691 None admin] Authenticating user token {{(pid=80377) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:02:41.051263 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG dbcounter [-] [80378] Writing DB stats keystone:SELECT=100 {{(pid=80378) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 10:02:41.081614 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed151ef1-101a-4422-83ea-579bedd6daf9 None admin] Validating token access rules against request {{(pid=80378) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:02:41.088504 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG dbcounter [-] [80377] Writing DB stats keystone:SELECT=100 {{(pid=80377) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 10:02:41.130691 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72daee8d-bd14-4e32-af23-ae835bea1216 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '891005fa5ff74041b75747d16cc66798', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80378) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:02:41.131648 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72daee8d-bd14-4e32-af23-ae835bea1216 None admin] REQUEST_METHOD: `GET` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:02:41.132215 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72daee8d-bd14-4e32-af23-ae835bea1216 None admin] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:02:41.132670 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72daee8d-bd14-4e32-af23-ae835bea1216 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:02:41.149038 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72daee8d-bd14-4e32-af23-ae835bea1216 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:02:41.151312 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72daee8d-bd14-4e32-af23-ae835bea1216 None admin] RBAC: Authorization granted {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:02:41.172365 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application [None req-72daee8d-bd14-4e32-af23-ae835bea1216 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 02 10:02:41.172365 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 02 10:02:41.172365 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 02 10:02:41.172365 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 02 10:02:41.172365 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 02 10:02:41.172365 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 02 10:02:41.172365 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 02 10:02:41.172365 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 02 10:02:41.172365 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 02 10:02:41.172365 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 02 10:02:41.172365 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 02 10:02:41.172365 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 02 10:02:41.172365 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 02 10:02:41.172365 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 02 10:02:41.172365 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 02 10:02:41.174289 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 02 10:02:41.174289 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 02 10:02:41.174289 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 02 10:02:41.174289 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 02 10:02:41.174289 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 02 10:02:41.174289 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 02 10:02:41.174289 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application return self.get_or_create( Aug 02 10:02:41.174289 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 02 10:02:41.174289 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application with Lock( Aug 02 10:02:41.174289 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 02 10:02:41.174289 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application return self._enter() Aug 02 10:02:41.174289 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 02 10:02:41.174289 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 02 10:02:41.174289 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 02 10:02:41.174289 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application return self.creator() Aug 02 10:02:41.174289 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 02 10:02:41.174289 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application created_value = creator( Aug 02 10:02:41.176398 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 02 10:02:41.176398 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 02 10:02:41.176398 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 02 10:02:41.176398 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application Aug 02 10:02:41.176794 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bba5f35-3c18-4ba4-bc52-65b585373691 None admin] Validating token access rules against request {{(pid=80377) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:02:41.179772 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 349/708] 10.222.0.82 () {56 vars in 1108 bytes} [Tue Aug 2 10:02:40 2022] GET /identity/v3/domains/Default => generated 87 bytes in 270 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 02 10:02:41.181165 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72daee8d-bd14-4e32-af23-ae835bea1216 None admin] Authenticating user token {{(pid=80378) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:02:41.241647 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5776ccc-ccfe-499c-b4c4-4c4369eacba8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '891005fa5ff74041b75747d16cc66798', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80377) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:02:41.242890 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5776ccc-ccfe-499c-b4c4-4c4369eacba8 None admin] REQUEST_METHOD: `GET` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:02:41.243360 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5776ccc-ccfe-499c-b4c4-4c4369eacba8 None admin] SCRIPT_NAME: `/identity` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:02:41.243794 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5776ccc-ccfe-499c-b4c4-4c4369eacba8 None admin] PATH_INFO: `/v3/services` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:02:41.244961 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5776ccc-ccfe-499c-b4c4-4c4369eacba8 None admin] RBAC: Adding query filter params (type=block-storage) {{(pid=80377) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 02 10:02:41.245435 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5776ccc-ccfe-499c-b4c4-4c4369eacba8 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:02:41.248055 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5776ccc-ccfe-499c-b4c4-4c4369eacba8 None admin] RBAC: Authorization granted {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:02:41.263511 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: [pid: 80377|app: 0|req: 360/709] 10.222.0.82 () {56 vars in 1124 bytes} [Tue Aug 2 10:02:41 2022] GET /identity/v3/services?type=block-storage => generated 130 bytes in 228 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 02 10:02:41.263511 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5776ccc-ccfe-499c-b4c4-4c4369eacba8 None admin] Authenticating user token {{(pid=80377) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:02:41.473599 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72daee8d-bd14-4e32-af23-ae835bea1216 None admin] Validating token access rules against request {{(pid=80378) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:02:41.500055 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG dbcounter [-] [80378] Writing DB stats keystone:SELECT=100 {{(pid=80378) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 10:02:41.542493 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d288967-aa9a-4bf9-81e2-c2c121599df1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '891005fa5ff74041b75747d16cc66798', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80378) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:02:41.543651 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d288967-aa9a-4bf9-81e2-c2c121599df1 None admin] REQUEST_METHOD: `GET` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:02:41.543938 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d288967-aa9a-4bf9-81e2-c2c121599df1 None admin] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:02:41.544320 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d288967-aa9a-4bf9-81e2-c2c121599df1 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:02:41.551970 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5776ccc-ccfe-499c-b4c4-4c4369eacba8 None admin] Validating token access rules against request {{(pid=80377) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:02:41.553667 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG dbcounter [-] [80377] Writing DB stats keystone:SELECT=100 {{(pid=80377) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 10:02:41.562728 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d288967-aa9a-4bf9-81e2-c2c121599df1 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:02:41.564932 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d288967-aa9a-4bf9-81e2-c2c121599df1 None admin] RBAC: Authorization granted {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:02:41.583456 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application [None req-9d288967-aa9a-4bf9-81e2-c2c121599df1 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 02 10:02:41.583456 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 02 10:02:41.583456 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 02 10:02:41.583456 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 02 10:02:41.583456 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 02 10:02:41.583456 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 02 10:02:41.583456 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 02 10:02:41.583456 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 02 10:02:41.583456 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 02 10:02:41.583456 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 02 10:02:41.583456 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 02 10:02:41.583456 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 02 10:02:41.583456 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 02 10:02:41.583456 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 02 10:02:41.583456 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 02 10:02:41.584692 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 02 10:02:41.584692 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 02 10:02:41.584692 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 02 10:02:41.584692 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 02 10:02:41.584692 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 02 10:02:41.584692 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 02 10:02:41.584692 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application return self.get_or_create( Aug 02 10:02:41.584692 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 02 10:02:41.584692 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application with Lock( Aug 02 10:02:41.584692 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 02 10:02:41.584692 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application return self._enter() Aug 02 10:02:41.584692 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 02 10:02:41.584692 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 02 10:02:41.584692 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 02 10:02:41.584692 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application return self.creator() Aug 02 10:02:41.584692 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 02 10:02:41.584692 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application created_value = creator( Aug 02 10:02:41.585774 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 02 10:02:41.585774 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 02 10:02:41.585774 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 02 10:02:41.585774 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 02 10:02:41.585774 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 02 10:02:41.585774 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 02 10:02:41.585774 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 02 10:02:41.585774 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application Aug 02 10:02:41.586553 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 350/710] 10.222.0.82 () {56 vars in 1111 bytes} [Tue Aug 2 10:02:41 2022] GET /identity/v3/projects/service => generated 88 bytes in 406 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 02 10:02:41.594631 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d288967-aa9a-4bf9-81e2-c2c121599df1 None admin] Authenticating user token {{(pid=80378) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:02:41.644499 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e407d2cf-e1b3-4b77-ad8e-b917dca9d1d8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '891005fa5ff74041b75747d16cc66798', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80377) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:02:41.645475 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e407d2cf-e1b3-4b77-ad8e-b917dca9d1d8 None admin] REQUEST_METHOD: `GET` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:02:41.646029 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e407d2cf-e1b3-4b77-ad8e-b917dca9d1d8 None admin] SCRIPT_NAME: `/identity` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:02:41.646420 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e407d2cf-e1b3-4b77-ad8e-b917dca9d1d8 None admin] PATH_INFO: `/v3/domains` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:02:41.647378 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e407d2cf-e1b3-4b77-ad8e-b917dca9d1d8 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80377) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 02 10:02:41.647769 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e407d2cf-e1b3-4b77-ad8e-b917dca9d1d8 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:02:41.649324 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e407d2cf-e1b3-4b77-ad8e-b917dca9d1d8 None admin] RBAC: Authorization granted {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:02:41.674977 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: [pid: 80377|app: 0|req: 361/711] 10.222.0.82 () {56 vars in 1109 bytes} [Tue Aug 2 10:02:41 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 412 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 02 10:02:41.681724 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e407d2cf-e1b3-4b77-ad8e-b917dca9d1d8 None admin] Authenticating user token {{(pid=80377) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:02:41.827698 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d288967-aa9a-4bf9-81e2-c2c121599df1 None admin] Validating token access rules against request {{(pid=80378) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:02:41.917831 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5ce821d-348c-439b-bbd0-43f41985b768 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '891005fa5ff74041b75747d16cc66798', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80378) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:02:41.918571 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5ce821d-348c-439b-bbd0-43f41985b768 None admin] REQUEST_METHOD: `GET` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:02:41.918860 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5ce821d-348c-439b-bbd0-43f41985b768 None admin] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:02:41.919314 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5ce821d-348c-439b-bbd0-43f41985b768 None admin] PATH_INFO: `/v3/projects` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:02:41.920096 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5ce821d-348c-439b-bbd0-43f41985b768 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80378) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 02 10:02:41.920408 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5ce821d-348c-439b-bbd0-43f41985b768 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:02:41.921958 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5ce821d-348c-439b-bbd0-43f41985b768 None admin] RBAC: Authorization granted {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:02:41.949145 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e407d2cf-e1b3-4b77-ad8e-b917dca9d1d8 None admin] Validating token access rules against request {{(pid=80377) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:02:41.955659 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 351/712] 10.222.0.82 () {56 vars in 1148 bytes} [Tue Aug 2 10:02:41 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 362 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 02 10:02:41.975085 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5ce821d-348c-439b-bbd0-43f41985b768 None admin] Authenticating user token {{(pid=80378) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:02:42.059661 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3ffe6b7-017e-4b19-81de-00a83504d3f4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '891005fa5ff74041b75747d16cc66798', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80377) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:02:42.060899 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3ffe6b7-017e-4b19-81de-00a83504d3f4 None admin] REQUEST_METHOD: `POST` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:02:42.061380 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3ffe6b7-017e-4b19-81de-00a83504d3f4 None admin] SCRIPT_NAME: `/identity` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:02:42.061869 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3ffe6b7-017e-4b19-81de-00a83504d3f4 None admin] PATH_INFO: `/v3/users` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:02:42.062831 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3ffe6b7-017e-4b19-81de-00a83504d3f4 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:02:42.065327 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3ffe6b7-017e-4b19-81de-00a83504d3f4 None admin] RBAC: Authorization granted {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:02:42.083784 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG dbcounter [-] [80378] Writing DB stats keystone:SELECT=100 {{(pid=80378) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 10:02:42.107445 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: [pid: 80377|app: 0|req: 362/713] 10.222.0.82 () {60 vars in 1132 bytes} [Tue Aug 2 10:02:41 2022] POST /identity/v3/users => generated 254 bytes in 426 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 02 10:02:42.116890 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fff58bd-2f89-46b9-a3fb-ed80ed641430 None None] REQUEST_METHOD: `GET` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:02:42.117348 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fff58bd-2f89-46b9-a3fb-ed80ed641430 None None] SCRIPT_NAME: `/identity` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:02:42.117803 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fff58bd-2f89-46b9-a3fb-ed80ed641430 None None] PATH_INFO: `/` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:02:42.120265 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: [pid: 80377|app: 0|req: 363/714] 10.222.0.82 () {54 vars in 926 bytes} [Tue Aug 2 10:02:42 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 02 10:02:42.135445 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-719dbdb8-c5ef-4505-8ac6-8e37a31612ec None None] REQUEST_METHOD: `POST` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:02:42.135839 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-719dbdb8-c5ef-4505-8ac6-8e37a31612ec None None] SCRIPT_NAME: `/identity` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:02:42.136209 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-719dbdb8-c5ef-4505-8ac6-8e37a31612ec None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:02:42.167141 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.auth.core [None req-719dbdb8-c5ef-4505-8ac6-8e37a31612ec None None] MFA Rules not processed for user `891005fa5ff74041b75747d16cc66798`. Rule list: `[]` (Enabled: `True`). {{(pid=80377) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 02 10:02:42.168849 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.fernet_utils [None req-719dbdb8-c5ef-4505-8ac6-8e37a31612ec None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80377) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 10:02:42.206155 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG dbcounter [-] [80377] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80377) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 10:02:42.236990 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5ce821d-348c-439b-bbd0-43f41985b768 None admin] Validating token access rules against request {{(pid=80378) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:02:42.319189 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3070e3cc-f7d4-4fdf-81d6-02c06aee177a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '891005fa5ff74041b75747d16cc66798', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80378) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:02:42.320001 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3070e3cc-f7d4-4fdf-81d6-02c06aee177a None admin] REQUEST_METHOD: `GET` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:02:42.320311 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3070e3cc-f7d4-4fdf-81d6-02c06aee177a None admin] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:02:42.320590 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3070e3cc-f7d4-4fdf-81d6-02c06aee177a None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:02:42.321517 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3070e3cc-f7d4-4fdf-81d6-02c06aee177a None admin] RBAC: Adding query filter params (role.id=237966442f05411191105bf336bc14f3, scope.project.id=f1b38ebb804c41c69772c998aae72c41, user.id=88af518196674fe593ad88d30111fb7d) {{(pid=80378) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 02 10:02:42.321798 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3070e3cc-f7d4-4fdf-81d6-02c06aee177a None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:02:42.323428 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3070e3cc-f7d4-4fdf-81d6-02c06aee177a None admin] RBAC: Authorization granted {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:02:42.336821 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 352/715] 10.222.0.82 () {56 vars in 1374 bytes} [Tue Aug 2 10:02:41 2022] GET /identity/v3/role_assignments?user.id=88af518196674fe593ad88d30111fb7d&scope.project.id=f1b38ebb804c41c69772c998aae72c41&role.id=237966442f05411191105bf336bc14f3 => generated 259 bytes in 361 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 02 10:02:42.339791 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: [pid: 80377|app: 0|req: 364/716] 10.222.0.82 () {58 vars in 1023 bytes} [Tue Aug 2 10:02:42 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 205 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 02 10:02:42.367584 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3070e3cc-f7d4-4fdf-81d6-02c06aee177a None admin] Authenticating user token {{(pid=80378) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:02:42.529588 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG dbcounter [-] [80378] Writing DB stats keystone:SELECT=100 {{(pid=80378) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 10:02:42.538354 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3070e3cc-f7d4-4fdf-81d6-02c06aee177a None admin] Validating token access rules against request {{(pid=80378) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:02:42.590282 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec97bcc0-2dd5-4fdf-a5f0-ec6b33226f74 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '891005fa5ff74041b75747d16cc66798', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80378) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:02:42.590858 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec97bcc0-2dd5-4fdf-a5f0-ec6b33226f74 None admin] REQUEST_METHOD: `GET` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:02:42.591240 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec97bcc0-2dd5-4fdf-a5f0-ec6b33226f74 None admin] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:02:42.591437 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec97bcc0-2dd5-4fdf-a5f0-ec6b33226f74 None admin] PATH_INFO: `/v3/services/network` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:02:42.592680 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec97bcc0-2dd5-4fdf-a5f0-ec6b33226f74 None admin] RBAC: Authorizing `identity:get_service(service_id=network)` {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:02:42.593662 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec97bcc0-2dd5-4fdf-a5f0-ec6b33226f74 None admin] RBAC: Authorization granted {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:02:42.607333 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application [None req-ec97bcc0-2dd5-4fdf-a5f0-ec6b33226f74 None admin] Could not find service: network.: keystone.exception.ServiceNotFound: Could not find service: network. Aug 02 10:02:42.607333 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 02 10:02:42.607333 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Aug 02 10:02:42.607333 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Aug 02 10:02:42.607333 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Aug 02 10:02:42.607333 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Aug 02 10:02:42.607333 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Aug 02 10:02:42.607333 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 02 10:02:42.607333 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: network. Aug 02 10:02:42.607333 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application Aug 02 10:02:42.607333 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 02 10:02:42.607333 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application Aug 02 10:02:42.607333 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 02 10:02:42.607333 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 02 10:02:42.607333 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 02 10:02:42.607333 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 02 10:02:42.607333 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 02 10:02:42.609229 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 02 10:02:42.609229 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 02 10:02:42.609229 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 02 10:02:42.609229 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 02 10:02:42.609229 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 02 10:02:42.609229 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 02 10:02:42.609229 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Aug 02 10:02:42.609229 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application return self._get_service(service_id) Aug 02 10:02:42.609229 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Aug 02 10:02:42.609229 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Aug 02 10:02:42.609229 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 02 10:02:42.609229 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 02 10:02:42.609229 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 02 10:02:42.609229 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 02 10:02:42.609229 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 02 10:02:42.609229 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application return self.get_or_create( Aug 02 10:02:42.610959 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 02 10:02:42.610959 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application with Lock( Aug 02 10:02:42.610959 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 02 10:02:42.610959 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application return self._enter() Aug 02 10:02:42.610959 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 02 10:02:42.610959 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 02 10:02:42.610959 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 02 10:02:42.610959 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application return self.creator() Aug 02 10:02:42.610959 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 02 10:02:42.610959 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application created_value = creator( Aug 02 10:02:42.610959 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Aug 02 10:02:42.610959 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 02 10:02:42.610959 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: network. Aug 02 10:02:42.610959 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application Aug 02 10:02:42.610959 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 353/717] 10.222.0.82 () {56 vars in 1111 bytes} [Tue Aug 2 10:02:42 2022] GET /identity/v3/services/network => generated 88 bytes in 242 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 02 10:02:42.629113 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-719dbdb8-c5ef-4505-8ac6-8e37a31612ec None None] Authenticating user token {{(pid=80377) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:02:42.824307 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-719dbdb8-c5ef-4505-8ac6-8e37a31612ec None None] Validating token access rules against request {{(pid=80377) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:02:42.895851 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-981bdba5-f157-4ae0-88e3-0784fc2b5bd9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '891005fa5ff74041b75747d16cc66798', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80377) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:02:42.896559 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-981bdba5-f157-4ae0-88e3-0784fc2b5bd9 None admin] REQUEST_METHOD: `GET` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:02:42.896873 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-981bdba5-f157-4ae0-88e3-0784fc2b5bd9 None admin] SCRIPT_NAME: `/identity` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:02:42.897181 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-981bdba5-f157-4ae0-88e3-0784fc2b5bd9 None admin] PATH_INFO: `/v3/services` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:02:42.897971 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-981bdba5-f157-4ae0-88e3-0784fc2b5bd9 None admin] RBAC: Adding query filter params (name=network) {{(pid=80377) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 02 10:02:42.898314 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-981bdba5-f157-4ae0-88e3-0784fc2b5bd9 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:02:42.899873 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-981bdba5-f157-4ae0-88e3-0784fc2b5bd9 None admin] RBAC: Authorization granted {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:02:42.906262 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: [pid: 80377|app: 0|req: 365/718] 10.222.0.82 () {56 vars in 1112 bytes} [Tue Aug 2 10:02:42 2022] GET /identity/v3/services?name=network => generated 124 bytes in 278 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 02 10:02:42.914945 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec97bcc0-2dd5-4fdf-a5f0-ec6b33226f74 None admin] Authenticating user token {{(pid=80378) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:02:43.152066 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec97bcc0-2dd5-4fdf-a5f0-ec6b33226f74 None admin] Validating token access rules against request {{(pid=80378) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:02:43.215849 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-772cd9fa-16c1-4036-9849-ca8045dc9d3d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '891005fa5ff74041b75747d16cc66798', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80378) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:02:43.216880 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-772cd9fa-16c1-4036-9849-ca8045dc9d3d None admin] REQUEST_METHOD: `GET` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:02:43.217397 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-772cd9fa-16c1-4036-9849-ca8045dc9d3d None admin] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:02:43.217988 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-772cd9fa-16c1-4036-9849-ca8045dc9d3d None admin] PATH_INFO: `/v3/services` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:02:43.219501 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-772cd9fa-16c1-4036-9849-ca8045dc9d3d None admin] RBAC: Adding query filter params (type=network) {{(pid=80378) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 02 10:02:43.220270 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-772cd9fa-16c1-4036-9849-ca8045dc9d3d None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:02:43.222811 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-772cd9fa-16c1-4036-9849-ca8045dc9d3d None admin] RBAC: Authorization granted {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:02:43.231066 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 354/719] 10.222.0.82 () {56 vars in 1112 bytes} [Tue Aug 2 10:02:42 2022] GET /identity/v3/services?type=network => generated 124 bytes in 317 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 02 10:02:43.294868 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fcc9db2-407b-4f16-8fd5-96d0a4244ef3 None None] REQUEST_METHOD: `GET` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:02:43.295596 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fcc9db2-407b-4f16-8fd5-96d0a4244ef3 None None] SCRIPT_NAME: `/identity` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:02:43.295724 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fcc9db2-407b-4f16-8fd5-96d0a4244ef3 None None] PATH_INFO: `/` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:02:43.297265 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: [pid: 80377|app: 0|req: 366/720] 10.222.0.82 () {54 vars in 926 bytes} [Tue Aug 2 10:02:43 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 02 10:02:43.318890 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66a63cbf-3a73-4de6-9e4f-705fc88a156e None None] REQUEST_METHOD: `POST` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:02:43.319876 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66a63cbf-3a73-4de6-9e4f-705fc88a156e None None] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:02:43.320458 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66a63cbf-3a73-4de6-9e4f-705fc88a156e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:02:43.348313 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.auth.core [None req-66a63cbf-3a73-4de6-9e4f-705fc88a156e None None] MFA Rules not processed for user `891005fa5ff74041b75747d16cc66798`. Rule list: `[]` (Enabled: `True`). {{(pid=80378) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 02 10:02:43.350016 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.fernet_utils [None req-66a63cbf-3a73-4de6-9e4f-705fc88a156e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80378) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 10:02:43.383780 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG dbcounter [-] [80378] Writing DB stats keystone:SELECT=100 {{(pid=80378) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 10:02:43.459095 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 355/721] 10.222.0.82 () {58 vars in 1023 bytes} [Tue Aug 2 10:02:43 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 143 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 02 10:02:43.493512 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fcc9db2-407b-4f16-8fd5-96d0a4244ef3 None None] Authenticating user token {{(pid=80377) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:02:43.516951 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG dbcounter [-] [80377] Writing DB stats keystone:SELECT=100 {{(pid=80377) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 10:02:43.639338 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fcc9db2-407b-4f16-8fd5-96d0a4244ef3 None None] Validating token access rules against request {{(pid=80377) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:02:43.693004 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-629a86cf-463e-4c1f-a111-fa0d6dc79942 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '891005fa5ff74041b75747d16cc66798', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80377) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:02:43.694416 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-629a86cf-463e-4c1f-a111-fa0d6dc79942 None admin] REQUEST_METHOD: `GET` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:02:43.695166 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-629a86cf-463e-4c1f-a111-fa0d6dc79942 None admin] SCRIPT_NAME: `/identity` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:02:43.695675 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-629a86cf-463e-4c1f-a111-fa0d6dc79942 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:02:43.709489 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-629a86cf-463e-4c1f-a111-fa0d6dc79942 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 02 10:02:43.715537 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application [None req-629a86cf-463e-4c1f-a111-fa0d6dc79942 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 02 10:02:43.715537 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 02 10:02:43.715537 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 02 10:02:43.715537 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 02 10:02:43.715537 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 02 10:02:43.715537 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 02 10:02:43.715537 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 02 10:02:43.715537 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 02 10:02:43.715537 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 02 10:02:43.715537 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 02 10:02:43.715537 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 02 10:02:43.715537 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 02 10:02:43.715537 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 02 10:02:43.715537 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 02 10:02:43.715537 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 02 10:02:43.715537 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 02 10:02:43.716544 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 02 10:02:43.716544 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 02 10:02:43.716544 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 02 10:02:43.716544 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 02 10:02:43.716544 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 02 10:02:43.716544 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 02 10:02:43.716544 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 02 10:02:43.716544 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 02 10:02:43.716544 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 02 10:02:43.716544 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application return self.get_or_create( Aug 02 10:02:43.716544 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 02 10:02:43.716544 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application with Lock( Aug 02 10:02:43.716544 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 02 10:02:43.716544 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application return self._enter() Aug 02 10:02:43.716544 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 02 10:02:43.716544 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 02 10:02:43.717531 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 02 10:02:43.717531 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application return self.creator() Aug 02 10:02:43.717531 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 02 10:02:43.717531 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application created_value = creator( Aug 02 10:02:43.717531 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 02 10:02:43.717531 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 02 10:02:43.717531 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 02 10:02:43.717531 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 02 10:02:43.717531 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 02 10:02:43.717531 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 02 10:02:43.717531 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 02 10:02:43.717531 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application Aug 02 10:02:43.720610 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: [pid: 80377|app: 0|req: 367/722] 10.222.0.82 () {56 vars in 1096 bytes} [Tue Aug 2 10:02:43 2022] GET /identity/v3/roles/admin => generated 83 bytes in 227 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 02 10:02:43.726937 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66a63cbf-3a73-4de6-9e4f-705fc88a156e None None] Authenticating user token {{(pid=80378) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:02:43.935398 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66a63cbf-3a73-4de6-9e4f-705fc88a156e None None] Validating token access rules against request {{(pid=80378) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:02:44.005503 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa54e6ba-3028-4023-a8ee-56e04af85048 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '891005fa5ff74041b75747d16cc66798', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80378) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:02:44.006654 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa54e6ba-3028-4023-a8ee-56e04af85048 None admin] REQUEST_METHOD: `GET` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:02:44.007148 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa54e6ba-3028-4023-a8ee-56e04af85048 None admin] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:02:44.007658 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa54e6ba-3028-4023-a8ee-56e04af85048 None admin] PATH_INFO: `/v3/roles` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:02:44.009004 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa54e6ba-3028-4023-a8ee-56e04af85048 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80378) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 02 10:02:44.009561 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa54e6ba-3028-4023-a8ee-56e04af85048 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:02:44.012237 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa54e6ba-3028-4023-a8ee-56e04af85048 None admin] RBAC: Authorization granted {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:02:44.038759 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 356/723] 10.222.0.82 () {56 vars in 1099 bytes} [Tue Aug 2 10:02:43 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 312 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 02 10:02:44.048701 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-629a86cf-463e-4c1f-a111-fa0d6dc79942 None admin] Authenticating user token {{(pid=80377) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:02:44.243295 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG dbcounter [-] [80377] Writing DB stats keystone:SELECT=100 {{(pid=80377) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 10:02:44.322575 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-629a86cf-463e-4c1f-a111-fa0d6dc79942 None admin] Validating token access rules against request {{(pid=80377) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:02:44.388504 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3bbd85b-2285-4d1f-b375-003e3f639634 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '891005fa5ff74041b75747d16cc66798', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80377) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:02:44.389423 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3bbd85b-2285-4d1f-b375-003e3f639634 None admin] REQUEST_METHOD: `GET` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:02:44.389799 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3bbd85b-2285-4d1f-b375-003e3f639634 None admin] SCRIPT_NAME: `/identity` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:02:44.390096 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3bbd85b-2285-4d1f-b375-003e3f639634 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:02:44.410524 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73e06cf0-73d5-41e5-8b9f-7189c8ab6727 None None] REQUEST_METHOD: `GET` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:02:44.410900 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73e06cf0-73d5-41e5-8b9f-7189c8ab6727 None None] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:02:44.411182 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73e06cf0-73d5-41e5-8b9f-7189c8ab6727 None None] PATH_INFO: `/` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:02:44.413101 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3bbd85b-2285-4d1f-b375-003e3f639634 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:02:44.413523 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 357/724] 10.222.0.82 () {54 vars in 926 bytes} [Tue Aug 2 10:02:44 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 02 10:02:44.415630 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3bbd85b-2285-4d1f-b375-003e3f639634 None admin] RBAC: Authorization granted {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:02:44.419825 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37dc83b2-ee4d-4324-94d9-b8fe4931543a None None] REQUEST_METHOD: `POST` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:02:44.420178 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37dc83b2-ee4d-4324-94d9-b8fe4931543a None None] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:02:44.420529 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37dc83b2-ee4d-4324-94d9-b8fe4931543a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:02:44.445486 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application [None req-d3bbd85b-2285-4d1f-b375-003e3f639634 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 02 10:02:44.445486 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 02 10:02:44.445486 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 02 10:02:44.445486 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 02 10:02:44.445486 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 02 10:02:44.445486 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 02 10:02:44.445486 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 02 10:02:44.445486 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 02 10:02:44.445486 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 02 10:02:44.445486 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 02 10:02:44.445486 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 02 10:02:44.445486 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 02 10:02:44.445486 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 02 10:02:44.445486 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 02 10:02:44.445486 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 02 10:02:44.446597 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 02 10:02:44.446597 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 02 10:02:44.446597 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 02 10:02:44.446597 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 02 10:02:44.446597 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 02 10:02:44.446597 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 02 10:02:44.446597 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application return self.get_or_create( Aug 02 10:02:44.446597 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 02 10:02:44.446597 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application with Lock( Aug 02 10:02:44.446597 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 02 10:02:44.446597 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application return self._enter() Aug 02 10:02:44.446597 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 02 10:02:44.446597 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 02 10:02:44.446597 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 02 10:02:44.446597 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application return self.creator() Aug 02 10:02:44.446597 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 02 10:02:44.446597 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application created_value = creator( Aug 02 10:02:44.447504 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 02 10:02:44.447504 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 02 10:02:44.447504 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 02 10:02:44.447504 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application Aug 02 10:02:44.448066 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: [pid: 80377|app: 0|req: 368/725] 10.222.0.82 () {56 vars in 1108 bytes} [Tue Aug 2 10:02:44 2022] GET /identity/v3/domains/Default => generated 87 bytes in 401 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 02 10:02:44.454030 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3bbd85b-2285-4d1f-b375-003e3f639634 None admin] Authenticating user token {{(pid=80377) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:02:44.466879 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.auth.core [None req-37dc83b2-ee4d-4324-94d9-b8fe4931543a None None] MFA Rules not processed for user `891005fa5ff74041b75747d16cc66798`. Rule list: `[]` (Enabled: `True`). {{(pid=80378) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 02 10:02:44.468811 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.fernet_utils [None req-37dc83b2-ee4d-4324-94d9-b8fe4931543a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80378) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 10:02:44.498341 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG dbcounter [-] [80378] Writing DB stats keystone:SELECT=100 {{(pid=80378) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 10:02:44.624645 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 358/726] 10.222.0.82 () {58 vars in 1023 bytes} [Tue Aug 2 10:02:44 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 206 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Aug 02 10:02:44.672501 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3bbd85b-2285-4d1f-b375-003e3f639634 None admin] Validating token access rules against request {{(pid=80377) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:02:44.718195 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG dbcounter [-] [80377] Writing DB stats keystone:SELECT=100 {{(pid=80377) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 10:02:44.723019 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37dc83b2-ee4d-4324-94d9-b8fe4931543a None None] Authenticating user token {{(pid=80378) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:02:44.725126 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6f56b6f-34f4-4b75-a106-1d4e85aa26dc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '891005fa5ff74041b75747d16cc66798', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80377) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:02:44.726278 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6f56b6f-34f4-4b75-a106-1d4e85aa26dc None admin] REQUEST_METHOD: `GET` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:02:44.726570 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6f56b6f-34f4-4b75-a106-1d4e85aa26dc None admin] SCRIPT_NAME: `/identity` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:02:44.727481 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6f56b6f-34f4-4b75-a106-1d4e85aa26dc None admin] PATH_INFO: `/v3/domains` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:02:44.728686 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6f56b6f-34f4-4b75-a106-1d4e85aa26dc None admin] RBAC: Adding query filter params (name=Default) {{(pid=80377) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 02 10:02:44.730305 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6f56b6f-34f4-4b75-a106-1d4e85aa26dc None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:02:44.732665 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6f56b6f-34f4-4b75-a106-1d4e85aa26dc None admin] RBAC: Authorization granted {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:02:44.761497 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: [pid: 80377|app: 0|req: 369/727] 10.222.0.82 () {56 vars in 1109 bytes} [Tue Aug 2 10:02:44 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 308 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 02 10:02:44.769200 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6f56b6f-34f4-4b75-a106-1d4e85aa26dc None admin] Authenticating user token {{(pid=80377) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:02:44.915174 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37dc83b2-ee4d-4324-94d9-b8fe4931543a None None] Validating token access rules against request {{(pid=80378) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:02:45.010766 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d810b9c3-e2fe-4e9b-8c36-440d5464f566 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '891005fa5ff74041b75747d16cc66798', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80378) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:02:45.011299 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d810b9c3-e2fe-4e9b-8c36-440d5464f566 None admin] REQUEST_METHOD: `POST` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:02:45.011586 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d810b9c3-e2fe-4e9b-8c36-440d5464f566 None admin] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:02:45.011912 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d810b9c3-e2fe-4e9b-8c36-440d5464f566 None admin] PATH_INFO: `/v3/services` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:02:45.012650 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d810b9c3-e2fe-4e9b-8c36-440d5464f566 None admin] RBAC: Authorizing `identity:create_service()` {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:02:45.014719 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d810b9c3-e2fe-4e9b-8c36-440d5464f566 None admin] RBAC: Authorization granted {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:02:45.040433 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6f56b6f-34f4-4b75-a106-1d4e85aa26dc None admin] Validating token access rules against request {{(pid=80377) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:02:45.051370 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 359/728] 10.222.0.82 () {60 vars in 1141 bytes} [Tue Aug 2 10:02:44 2022] POST /identity/v3/services => generated 251 bytes in 329 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 02 10:02:45.123414 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21f7f423-4601-4a1a-858b-c5d62dba5a61 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '891005fa5ff74041b75747d16cc66798', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80377) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:02:45.124527 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21f7f423-4601-4a1a-858b-c5d62dba5a61 None admin] REQUEST_METHOD: `GET` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:02:45.125052 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21f7f423-4601-4a1a-858b-c5d62dba5a61 None admin] SCRIPT_NAME: `/identity` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:02:45.125532 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21f7f423-4601-4a1a-858b-c5d62dba5a61 None admin] PATH_INFO: `/v3/users/swift` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:02:45.136031 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21f7f423-4601-4a1a-858b-c5d62dba5a61 None admin] RBAC: Authorizing `identity:get_user(user_id=swift)` {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:02:45.137988 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21f7f423-4601-4a1a-858b-c5d62dba5a61 None admin] RBAC: Authorization granted {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:02:45.160836 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application [None req-21f7f423-4601-4a1a-858b-c5d62dba5a61 None admin] Could not find user: swift.: keystone.exception.UserNotFound: Could not find user: swift. Aug 02 10:02:45.160836 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 02 10:02:45.160836 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 02 10:02:45.160836 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 02 10:02:45.160836 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 02 10:02:45.160836 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 02 10:02:45.160836 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 02 10:02:45.160836 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 02 10:02:45.160836 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 02 10:02:45.160836 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 02 10:02:45.160836 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 02 10:02:45.160836 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 02 10:02:45.160836 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 02 10:02:45.160836 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 02 10:02:45.160836 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 02 10:02:45.160836 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 02 10:02:45.166487 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 02 10:02:45.166487 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 02 10:02:45.166487 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 02 10:02:45.166487 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 02 10:02:45.166487 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 02 10:02:45.166487 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 02 10:02:45.166487 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 02 10:02:45.166487 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 02 10:02:45.166487 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 02 10:02:45.166487 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application return self.get_or_create( Aug 02 10:02:45.166487 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 02 10:02:45.166487 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application with Lock( Aug 02 10:02:45.166487 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 02 10:02:45.166487 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application return self._enter() Aug 02 10:02:45.166487 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 02 10:02:45.166487 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 02 10:02:45.166487 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 02 10:02:45.166487 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application return self.creator() Aug 02 10:02:45.167793 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 02 10:02:45.167793 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application created_value = creator( Aug 02 10:02:45.167793 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 02 10:02:45.167793 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 02 10:02:45.167793 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 02 10:02:45.167793 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 02 10:02:45.167793 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 02 10:02:45.167793 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 02 10:02:45.167793 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: swift. Aug 02 10:02:45.167793 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application Aug 02 10:02:45.170482 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: [pid: 80377|app: 0|req: 370/729] 10.222.0.82 () {56 vars in 1096 bytes} [Tue Aug 2 10:02:44 2022] GET /identity/v3/users/swift => generated 83 bytes in 402 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 02 10:02:45.178321 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d810b9c3-e2fe-4e9b-8c36-440d5464f566 None admin] Authenticating user token {{(pid=80378) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:02:45.199163 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG dbcounter [-] [80378] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80378) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 10:02:45.388650 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d810b9c3-e2fe-4e9b-8c36-440d5464f566 None admin] Validating token access rules against request {{(pid=80378) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:02:45.444982 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9dace06a-a6a8-47e3-b785-6d0c24b2d93b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '891005fa5ff74041b75747d16cc66798', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80378) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:02:45.445541 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9dace06a-a6a8-47e3-b785-6d0c24b2d93b None admin] REQUEST_METHOD: `GET` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:02:45.445847 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9dace06a-a6a8-47e3-b785-6d0c24b2d93b None admin] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:02:45.446123 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9dace06a-a6a8-47e3-b785-6d0c24b2d93b None admin] PATH_INFO: `/v3/users` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:02:45.446938 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9dace06a-a6a8-47e3-b785-6d0c24b2d93b None admin] RBAC: Adding query filter params (domain_id=default, name=swift) {{(pid=80378) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 02 10:02:45.447241 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9dace06a-a6a8-47e3-b785-6d0c24b2d93b None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:02:45.448712 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9dace06a-a6a8-47e3-b785-6d0c24b2d93b None admin] RBAC: Authorization granted {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:02:45.476553 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 360/730] 10.222.0.82 () {56 vars in 1135 bytes} [Tue Aug 2 10:02:45 2022] GET /identity/v3/users?domain_id=default&name=swift => generated 370 bytes in 299 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 02 10:02:45.482481 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21f7f423-4601-4a1a-858b-c5d62dba5a61 None admin] Authenticating user token {{(pid=80377) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:02:45.561645 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG dbcounter [-] [80377] Writing DB stats keystone:SELECT=100 {{(pid=80377) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 10:02:45.576507 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3542a4cf-fa0f-40f9-8021-df12c0bd48e0 None None] REQUEST_METHOD: `GET` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:02:45.576969 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3542a4cf-fa0f-40f9-8021-df12c0bd48e0 None None] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:02:45.577545 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3542a4cf-fa0f-40f9-8021-df12c0bd48e0 None None] PATH_INFO: `/` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:02:45.580955 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 361/731] 10.222.0.82 () {54 vars in 926 bytes} [Tue Aug 2 10:02:45 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 02 10:02:45.590006 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c36fb34-aa18-43d6-bb18-84404dc9c723 None None] REQUEST_METHOD: `POST` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:02:45.590306 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c36fb34-aa18-43d6-bb18-84404dc9c723 None None] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:02:45.590581 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c36fb34-aa18-43d6-bb18-84404dc9c723 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:02:45.616682 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.auth.core [None req-8c36fb34-aa18-43d6-bb18-84404dc9c723 None None] MFA Rules not processed for user `891005fa5ff74041b75747d16cc66798`. Rule list: `[]` (Enabled: `True`). {{(pid=80378) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 02 10:02:45.618087 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.fernet_utils [None req-8c36fb34-aa18-43d6-bb18-84404dc9c723 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80378) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 10:02:45.661125 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21f7f423-4601-4a1a-858b-c5d62dba5a61 None admin] Validating token access rules against request {{(pid=80377) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:02:45.708269 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG dbcounter [-] [80378] Writing DB stats keystone:SELECT=100 {{(pid=80378) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 10:02:45.713727 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6b95250-cb36-43b2-8bda-cfe2e9a38554 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '891005fa5ff74041b75747d16cc66798', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80377) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:02:45.714832 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6b95250-cb36-43b2-8bda-cfe2e9a38554 None admin] REQUEST_METHOD: `GET` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:02:45.715460 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6b95250-cb36-43b2-8bda-cfe2e9a38554 None admin] SCRIPT_NAME: `/identity` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:02:45.716154 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6b95250-cb36-43b2-8bda-cfe2e9a38554 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:02:45.718963 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 362/732] 10.222.0.82 () {58 vars in 1023 bytes} [Tue Aug 2 10:02:45 2022] POST /identity/v3/auth/tokens => generated 903 bytes in 130 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 02 10:02:45.721186 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00c1e8e8-aec4-4459-832e-a20c312e9dfa None None] REQUEST_METHOD: `GET` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:02:45.721743 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00c1e8e8-aec4-4459-832e-a20c312e9dfa None None] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:02:45.722245 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00c1e8e8-aec4-4459-832e-a20c312e9dfa None None] PATH_INFO: `/` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:02:45.724079 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 363/733] 10.222.0.82 () {54 vars in 926 bytes} [Tue Aug 2 10:02:45 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 02 10:02:45.744481 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9aa8abdc-c43c-4b19-baa7-2d571c1672f5 None None] REQUEST_METHOD: `POST` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:02:45.745138 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9aa8abdc-c43c-4b19-baa7-2d571c1672f5 None None] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:02:45.745680 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9aa8abdc-c43c-4b19-baa7-2d571c1672f5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:02:45.773863 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.auth.core [None req-9aa8abdc-c43c-4b19-baa7-2d571c1672f5 None None] MFA Rules not processed for user `891005fa5ff74041b75747d16cc66798`. Rule list: `[]` (Enabled: `True`). {{(pid=80378) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 02 10:02:45.775846 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.fernet_utils [None req-9aa8abdc-c43c-4b19-baa7-2d571c1672f5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80378) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 10:02:45.845441 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6b95250-cb36-43b2-8bda-cfe2e9a38554 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:02:45.847487 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6b95250-cb36-43b2-8bda-cfe2e9a38554 None admin] RBAC: Authorization granted {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:02:45.880567 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 364/734] 10.222.0.82 () {58 vars in 1023 bytes} [Tue Aug 2 10:02:45 2022] POST /identity/v3/auth/tokens => generated 903 bytes in 137 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Aug 02 10:02:45.887084 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9aa8abdc-c43c-4b19-baa7-2d571c1672f5 None None] Authenticating user token {{(pid=80378) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:02:45.902705 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG dbcounter [-] [80377] Writing DB stats keystone:SELECT=100 {{(pid=80377) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 10:02:46.021277 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: [pid: 80377|app: 0|req: 371/735] 10.222.0.82 () {58 vars in 1282 bytes} [Tue Aug 2 10:02:45 2022] GET /identity/v3/auth/tokens => generated 903 bytes in 539 msecs (HTTP/1.1 200) 6 headers in 358 bytes (2 switches on core 0) Aug 02 10:02:46.023244 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6b95250-cb36-43b2-8bda-cfe2e9a38554 None admin] Authenticating user token {{(pid=80377) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:02:46.077572 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9aa8abdc-c43c-4b19-baa7-2d571c1672f5 None None] Validating token access rules against request {{(pid=80378) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:02:46.126076 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG dbcounter [-] [80378] Writing DB stats keystone:SELECT=100 {{(pid=80378) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 10:02:46.144185 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f4ed9a3-3baf-4386-a48c-855753f524ac None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '891005fa5ff74041b75747d16cc66798', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80378) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:02:46.145442 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f4ed9a3-3baf-4386-a48c-855753f524ac None admin] REQUEST_METHOD: `GET` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:02:46.146143 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f4ed9a3-3baf-4386-a48c-855753f524ac None admin] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:02:46.146847 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f4ed9a3-3baf-4386-a48c-855753f524ac None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:02:46.164741 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f4ed9a3-3baf-4386-a48c-855753f524ac None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 02 10:02:46.172416 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application [None req-8f4ed9a3-3baf-4386-a48c-855753f524ac None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 02 10:02:46.172416 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 02 10:02:46.172416 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 02 10:02:46.172416 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 02 10:02:46.172416 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 02 10:02:46.172416 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 02 10:02:46.172416 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 02 10:02:46.172416 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 02 10:02:46.172416 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 02 10:02:46.172416 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 02 10:02:46.172416 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 02 10:02:46.172416 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 02 10:02:46.172416 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 02 10:02:46.172416 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 02 10:02:46.172416 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 02 10:02:46.172416 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 02 10:02:46.173526 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 02 10:02:46.173526 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 02 10:02:46.173526 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 02 10:02:46.173526 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 02 10:02:46.173526 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 02 10:02:46.173526 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 02 10:02:46.173526 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 02 10:02:46.173526 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 02 10:02:46.173526 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 02 10:02:46.173526 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application return self.get_or_create( Aug 02 10:02:46.173526 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 02 10:02:46.173526 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application with Lock( Aug 02 10:02:46.173526 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 02 10:02:46.173526 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application return self._enter() Aug 02 10:02:46.173526 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 02 10:02:46.173526 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 02 10:02:46.175199 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 02 10:02:46.175199 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application return self.creator() Aug 02 10:02:46.175199 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 02 10:02:46.175199 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application created_value = creator( Aug 02 10:02:46.175199 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 02 10:02:46.175199 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 02 10:02:46.175199 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 02 10:02:46.175199 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 02 10:02:46.175199 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 02 10:02:46.175199 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 02 10:02:46.175199 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 02 10:02:46.175199 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application Aug 02 10:02:46.182299 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 365/736] 10.222.0.82 () {56 vars in 1096 bytes} [Tue Aug 2 10:02:45 2022] GET /identity/v3/roles/admin => generated 83 bytes in 296 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 02 10:02:46.183740 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f4ed9a3-3baf-4386-a48c-855753f524ac None admin] Authenticating user token {{(pid=80378) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:02:46.186579 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6b95250-cb36-43b2-8bda-cfe2e9a38554 None admin] Validating token access rules against request {{(pid=80377) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:02:46.236511 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b465c5ad-c0d7-4590-9d89-88cbd9eac8fe None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '891005fa5ff74041b75747d16cc66798', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80377) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:02:46.237361 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b465c5ad-c0d7-4590-9d89-88cbd9eac8fe None admin] REQUEST_METHOD: `GET` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:02:46.237778 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b465c5ad-c0d7-4590-9d89-88cbd9eac8fe None admin] SCRIPT_NAME: `/identity` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:02:46.238249 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b465c5ad-c0d7-4590-9d89-88cbd9eac8fe None admin] PATH_INFO: `/v3/roles/service` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:02:46.248481 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG dbcounter [-] [80377] Writing DB stats keystone:SELECT=100 {{(pid=80377) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 10:02:46.255099 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b465c5ad-c0d7-4590-9d89-88cbd9eac8fe None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 02 10:02:46.261230 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application [None req-b465c5ad-c0d7-4590-9d89-88cbd9eac8fe None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Aug 02 10:02:46.261230 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 02 10:02:46.261230 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 02 10:02:46.261230 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 02 10:02:46.261230 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 02 10:02:46.261230 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 02 10:02:46.261230 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 02 10:02:46.261230 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 02 10:02:46.261230 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 02 10:02:46.261230 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 02 10:02:46.261230 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 02 10:02:46.261230 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 02 10:02:46.261230 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 02 10:02:46.261230 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 02 10:02:46.261230 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 02 10:02:46.261230 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 02 10:02:46.262560 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 02 10:02:46.262560 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 02 10:02:46.262560 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 02 10:02:46.262560 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 02 10:02:46.262560 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 02 10:02:46.262560 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 02 10:02:46.262560 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 02 10:02:46.262560 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 02 10:02:46.262560 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 02 10:02:46.262560 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application return self.get_or_create( Aug 02 10:02:46.262560 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 02 10:02:46.262560 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application with Lock( Aug 02 10:02:46.262560 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 02 10:02:46.262560 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application return self._enter() Aug 02 10:02:46.262560 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 02 10:02:46.262560 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 02 10:02:46.263430 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 02 10:02:46.263430 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application return self.creator() Aug 02 10:02:46.263430 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 02 10:02:46.263430 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application created_value = creator( Aug 02 10:02:46.263430 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 02 10:02:46.263430 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 02 10:02:46.263430 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 02 10:02:46.263430 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 02 10:02:46.263430 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 02 10:02:46.263430 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 02 10:02:46.263430 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Aug 02 10:02:46.263430 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application Aug 02 10:02:46.269626 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: [pid: 80377|app: 0|req: 372/737] 10.222.0.82 () {56 vars in 1102 bytes} [Tue Aug 2 10:02:46 2022] GET /identity/v3/roles/service => generated 85 bytes in 241 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 02 10:02:46.271014 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b465c5ad-c0d7-4590-9d89-88cbd9eac8fe None admin] Authenticating user token {{(pid=80377) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:02:46.321772 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f4ed9a3-3baf-4386-a48c-855753f524ac None admin] Validating token access rules against request {{(pid=80378) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:02:46.362748 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b133a3e-128f-4b1c-a404-9e5e28add7c1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '891005fa5ff74041b75747d16cc66798', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80378) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:02:46.363665 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b133a3e-128f-4b1c-a404-9e5e28add7c1 None admin] REQUEST_METHOD: `GET` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:02:46.364129 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b133a3e-128f-4b1c-a404-9e5e28add7c1 None admin] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:02:46.364571 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b133a3e-128f-4b1c-a404-9e5e28add7c1 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:02:46.377989 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b133a3e-128f-4b1c-a404-9e5e28add7c1 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:02:46.380923 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b133a3e-128f-4b1c-a404-9e5e28add7c1 None admin] RBAC: Authorization granted {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:02:46.398160 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application [None req-5b133a3e-128f-4b1c-a404-9e5e28add7c1 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 02 10:02:46.398160 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 02 10:02:46.398160 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 02 10:02:46.398160 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 02 10:02:46.398160 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 02 10:02:46.398160 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 02 10:02:46.398160 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 02 10:02:46.398160 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 02 10:02:46.398160 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 02 10:02:46.398160 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 02 10:02:46.398160 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 02 10:02:46.398160 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 02 10:02:46.398160 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 02 10:02:46.398160 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 02 10:02:46.398160 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 02 10:02:46.399293 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 02 10:02:46.399293 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 02 10:02:46.399293 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 02 10:02:46.399293 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 02 10:02:46.399293 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 02 10:02:46.399293 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 02 10:02:46.399293 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application return self.get_or_create( Aug 02 10:02:46.399293 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 02 10:02:46.399293 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application with Lock( Aug 02 10:02:46.399293 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 02 10:02:46.399293 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application return self._enter() Aug 02 10:02:46.399293 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 02 10:02:46.399293 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 02 10:02:46.399293 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 02 10:02:46.399293 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application return self.creator() Aug 02 10:02:46.399293 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 02 10:02:46.399293 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application created_value = creator( Aug 02 10:02:46.400353 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 02 10:02:46.400353 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 02 10:02:46.400353 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 02 10:02:46.400353 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application Aug 02 10:02:46.401935 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 366/738] 10.222.0.82 () {56 vars in 1108 bytes} [Tue Aug 2 10:02:46 2022] GET /identity/v3/domains/Default => generated 87 bytes in 219 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 02 10:02:46.403026 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b133a3e-128f-4b1c-a404-9e5e28add7c1 None admin] Authenticating user token {{(pid=80378) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:02:46.420291 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b465c5ad-c0d7-4590-9d89-88cbd9eac8fe None admin] Validating token access rules against request {{(pid=80377) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:02:46.446032 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG dbcounter [-] [80378] Writing DB stats keystone:SELECT=100 {{(pid=80378) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 10:02:46.468088 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38d89c7b-b6c2-4f82-a9f0-9f4616ae7306 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '891005fa5ff74041b75747d16cc66798', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80377) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:02:46.468826 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38d89c7b-b6c2-4f82-a9f0-9f4616ae7306 None admin] REQUEST_METHOD: `GET` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:02:46.469151 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38d89c7b-b6c2-4f82-a9f0-9f4616ae7306 None admin] SCRIPT_NAME: `/identity` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:02:46.469749 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38d89c7b-b6c2-4f82-a9f0-9f4616ae7306 None admin] PATH_INFO: `/v3/roles` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:02:46.470895 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38d89c7b-b6c2-4f82-a9f0-9f4616ae7306 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80377) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 02 10:02:46.471252 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38d89c7b-b6c2-4f82-a9f0-9f4616ae7306 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:02:46.472717 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38d89c7b-b6c2-4f82-a9f0-9f4616ae7306 None admin] RBAC: Authorization granted {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:02:46.487991 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: [pid: 80377|app: 0|req: 373/739] 10.222.0.82 () {56 vars in 1099 bytes} [Tue Aug 2 10:02:46 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 213 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 02 10:02:46.487991 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38d89c7b-b6c2-4f82-a9f0-9f4616ae7306 None admin] Authenticating user token {{(pid=80377) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:02:46.669543 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b133a3e-128f-4b1c-a404-9e5e28add7c1 None admin] Validating token access rules against request {{(pid=80378) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:02:46.712147 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70b972cb-fe88-4cd1-9aeb-9ece9a648e81 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '891005fa5ff74041b75747d16cc66798', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80378) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:02:46.712781 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70b972cb-fe88-4cd1-9aeb-9ece9a648e81 None admin] REQUEST_METHOD: `GET` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:02:46.713066 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70b972cb-fe88-4cd1-9aeb-9ece9a648e81 None admin] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:02:46.713344 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70b972cb-fe88-4cd1-9aeb-9ece9a648e81 None admin] PATH_INFO: `/v3/roles` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:02:46.714105 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70b972cb-fe88-4cd1-9aeb-9ece9a648e81 None admin] RBAC: Adding query filter params (name=service) {{(pid=80378) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 02 10:02:46.714461 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70b972cb-fe88-4cd1-9aeb-9ece9a648e81 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:02:46.715936 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70b972cb-fe88-4cd1-9aeb-9ece9a648e81 None admin] RBAC: Authorization granted {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:02:46.727535 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 367/740] 10.222.0.82 () {56 vars in 1103 bytes} [Tue Aug 2 10:02:46 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 325 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 02 10:02:46.729155 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70b972cb-fe88-4cd1-9aeb-9ece9a648e81 None admin] Authenticating user token {{(pid=80378) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:02:46.735125 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG dbcounter [-] [80377] Writing DB stats keystone:SELECT=100 {{(pid=80377) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 10:02:46.792207 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38d89c7b-b6c2-4f82-a9f0-9f4616ae7306 None admin] Validating token access rules against request {{(pid=80377) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:02:46.842362 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59614c93-dca5-4a0b-9f70-0529182270b2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '891005fa5ff74041b75747d16cc66798', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80377) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:02:46.843283 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59614c93-dca5-4a0b-9f70-0529182270b2 None admin] REQUEST_METHOD: `GET` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:02:46.843560 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59614c93-dca5-4a0b-9f70-0529182270b2 None admin] SCRIPT_NAME: `/identity` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:02:46.844162 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59614c93-dca5-4a0b-9f70-0529182270b2 None admin] PATH_INFO: `/v3/domains` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:02:46.844713 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59614c93-dca5-4a0b-9f70-0529182270b2 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80377) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 02 10:02:46.845747 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59614c93-dca5-4a0b-9f70-0529182270b2 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:02:46.847775 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59614c93-dca5-4a0b-9f70-0529182270b2 None admin] RBAC: Authorization granted {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:02:46.857907 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG dbcounter [-] [80378] Writing DB stats keystone:SELECT=100 {{(pid=80378) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 10:02:46.869163 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: [pid: 80377|app: 0|req: 374/741] 10.222.0.82 () {56 vars in 1109 bytes} [Tue Aug 2 10:02:46 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 382 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 02 10:02:46.869163 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20adfa5e-b575-4099-86ea-a519e8cd3546 None None] REQUEST_METHOD: `GET` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:02:46.869460 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70b972cb-fe88-4cd1-9aeb-9ece9a648e81 None admin] Validating token access rules against request {{(pid=80378) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:02:46.869542 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20adfa5e-b575-4099-86ea-a519e8cd3546 None None] SCRIPT_NAME: `/identity` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:02:46.869542 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20adfa5e-b575-4099-86ea-a519e8cd3546 None None] PATH_INFO: `/` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:02:46.874975 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: [pid: 80377|app: 0|req: 375/742] 10.222.0.82 () {54 vars in 926 bytes} [Tue Aug 2 10:02:46 2022] GET /identity => generated 270 bytes in 8 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 02 10:02:46.877086 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20adfa5e-b575-4099-86ea-a519e8cd3546 None None] Authenticating user token {{(pid=80377) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:02:46.926323 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17b73e37-a5c3-483b-a1c3-d79f79bfcbda None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '891005fa5ff74041b75747d16cc66798', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80378) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:02:46.927111 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17b73e37-a5c3-483b-a1c3-d79f79bfcbda None admin] REQUEST_METHOD: `GET` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:02:46.927504 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17b73e37-a5c3-483b-a1c3-d79f79bfcbda None admin] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:02:46.927823 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17b73e37-a5c3-483b-a1c3-d79f79bfcbda None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:02:46.944318 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17b73e37-a5c3-483b-a1c3-d79f79bfcbda None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:02:46.946950 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17b73e37-a5c3-483b-a1c3-d79f79bfcbda None admin] RBAC: Authorization granted {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:02:46.970517 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application [None req-17b73e37-a5c3-483b-a1c3-d79f79bfcbda None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 02 10:02:46.970517 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 02 10:02:46.970517 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 02 10:02:46.970517 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 02 10:02:46.970517 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 02 10:02:46.970517 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 02 10:02:46.970517 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 02 10:02:46.970517 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 02 10:02:46.970517 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 02 10:02:46.970517 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 02 10:02:46.970517 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 02 10:02:46.970517 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 02 10:02:46.970517 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 02 10:02:46.970517 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 02 10:02:46.970517 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 02 10:02:46.971637 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 02 10:02:46.971637 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 02 10:02:46.971637 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 02 10:02:46.971637 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 02 10:02:46.971637 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 02 10:02:46.971637 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 02 10:02:46.971637 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application return self.get_or_create( Aug 02 10:02:46.971637 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 02 10:02:46.971637 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application with Lock( Aug 02 10:02:46.971637 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 02 10:02:46.971637 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application return self._enter() Aug 02 10:02:46.971637 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 02 10:02:46.971637 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 02 10:02:46.971637 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 02 10:02:46.971637 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application return self.creator() Aug 02 10:02:46.971637 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 02 10:02:46.971637 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application created_value = creator( Aug 02 10:02:46.972802 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 02 10:02:46.972802 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 02 10:02:46.972802 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 02 10:02:46.972802 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application Aug 02 10:02:46.972802 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 368/743] 10.222.0.82 () {56 vars in 1108 bytes} [Tue Aug 2 10:02:46 2022] GET /identity/v3/domains/Default => generated 87 bytes in 244 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 02 10:02:46.972802 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17b73e37-a5c3-483b-a1c3-d79f79bfcbda None admin] Authenticating user token {{(pid=80378) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:02:47.020962 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20adfa5e-b575-4099-86ea-a519e8cd3546 None None] Validating token access rules against request {{(pid=80377) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:02:47.049191 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG dbcounter [-] [80377] Writing DB stats keystone:SELECT=100 {{(pid=80377) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 10:02:47.065434 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40adc34a-5cf5-4f74-896b-9cf04c018208 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '891005fa5ff74041b75747d16cc66798', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80377) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:02:47.066300 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40adc34a-5cf5-4f74-896b-9cf04c018208 None admin] REQUEST_METHOD: `GET` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:02:47.066599 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40adc34a-5cf5-4f74-896b-9cf04c018208 None admin] SCRIPT_NAME: `/identity` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:02:47.066881 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40adc34a-5cf5-4f74-896b-9cf04c018208 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:02:47.080532 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40adc34a-5cf5-4f74-896b-9cf04c018208 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:02:47.082329 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40adc34a-5cf5-4f74-896b-9cf04c018208 None admin] RBAC: Authorization granted {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:02:47.104852 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application [None req-40adc34a-5cf5-4f74-896b-9cf04c018208 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 02 10:02:47.104852 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 02 10:02:47.104852 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 02 10:02:47.104852 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 02 10:02:47.104852 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 02 10:02:47.104852 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 02 10:02:47.104852 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 02 10:02:47.104852 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 02 10:02:47.104852 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 02 10:02:47.104852 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 02 10:02:47.104852 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 02 10:02:47.104852 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 02 10:02:47.104852 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 02 10:02:47.104852 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 02 10:02:47.104852 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 02 10:02:47.106711 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 02 10:02:47.106711 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 02 10:02:47.106711 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 02 10:02:47.106711 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 02 10:02:47.106711 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 02 10:02:47.106711 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 02 10:02:47.106711 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application return self.get_or_create( Aug 02 10:02:47.106711 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 02 10:02:47.106711 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application with Lock( Aug 02 10:02:47.106711 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 02 10:02:47.106711 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application return self._enter() Aug 02 10:02:47.106711 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 02 10:02:47.106711 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 02 10:02:47.106711 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 02 10:02:47.106711 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application return self.creator() Aug 02 10:02:47.106711 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 02 10:02:47.106711 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application created_value = creator( Aug 02 10:02:47.108454 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 02 10:02:47.108454 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 02 10:02:47.108454 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 02 10:02:47.108454 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application Aug 02 10:02:47.108454 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: [pid: 80377|app: 0|req: 376/744] 10.222.0.82 () {56 vars in 1108 bytes} [Tue Aug 2 10:02:46 2022] GET /identity/v3/domains/Default => generated 87 bytes in 231 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 02 10:02:47.108454 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3289c9ca-832e-4b7c-8c82-a06d16ca268f None None] REQUEST_METHOD: `POST` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:02:47.108454 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3289c9ca-832e-4b7c-8c82-a06d16ca268f None None] SCRIPT_NAME: `/identity` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:02:47.108454 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3289c9ca-832e-4b7c-8c82-a06d16ca268f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:02:47.124882 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17b73e37-a5c3-483b-a1c3-d79f79bfcbda None admin] Validating token access rules against request {{(pid=80378) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:02:47.136663 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.auth.core [None req-3289c9ca-832e-4b7c-8c82-a06d16ca268f None None] MFA Rules not processed for user `891005fa5ff74041b75747d16cc66798`. Rule list: `[]` (Enabled: `True`). {{(pid=80377) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 02 10:02:47.138105 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.fernet_utils [None req-3289c9ca-832e-4b7c-8c82-a06d16ca268f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80377) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 10:02:47.171970 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c17a96f-2eb0-4abf-9518-40919ee062ea None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '891005fa5ff74041b75747d16cc66798', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80378) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:02:47.172947 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c17a96f-2eb0-4abf-9518-40919ee062ea None admin] REQUEST_METHOD: `GET` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:02:47.173969 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c17a96f-2eb0-4abf-9518-40919ee062ea None admin] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:02:47.174406 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c17a96f-2eb0-4abf-9518-40919ee062ea None admin] PATH_INFO: `/v3/projects/service` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:02:47.183606 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c17a96f-2eb0-4abf-9518-40919ee062ea None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:02:47.185272 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c17a96f-2eb0-4abf-9518-40919ee062ea None admin] RBAC: Authorization granted {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:02:47.202106 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application [None req-6c17a96f-2eb0-4abf-9518-40919ee062ea None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 02 10:02:47.202106 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 02 10:02:47.202106 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 02 10:02:47.202106 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 02 10:02:47.202106 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 02 10:02:47.202106 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 02 10:02:47.202106 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 02 10:02:47.202106 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 02 10:02:47.202106 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 02 10:02:47.202106 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 02 10:02:47.202106 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 02 10:02:47.202106 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 02 10:02:47.202106 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 02 10:02:47.202106 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 02 10:02:47.202106 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 02 10:02:47.203885 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 02 10:02:47.203885 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 02 10:02:47.203885 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 02 10:02:47.203885 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 02 10:02:47.203885 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 02 10:02:47.203885 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 02 10:02:47.203885 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application return self.get_or_create( Aug 02 10:02:47.203885 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 02 10:02:47.203885 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application with Lock( Aug 02 10:02:47.203885 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 02 10:02:47.203885 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application return self._enter() Aug 02 10:02:47.203885 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 02 10:02:47.203885 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 02 10:02:47.203885 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 02 10:02:47.203885 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application return self.creator() Aug 02 10:02:47.203885 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 02 10:02:47.203885 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application created_value = creator( Aug 02 10:02:47.205862 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 02 10:02:47.205862 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 02 10:02:47.205862 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 02 10:02:47.205862 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 02 10:02:47.205862 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 02 10:02:47.205862 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 02 10:02:47.205862 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 02 10:02:47.205862 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application Aug 02 10:02:47.205862 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 369/745] 10.222.0.82 () {56 vars in 1111 bytes} [Tue Aug 2 10:02:46 2022] GET /identity/v3/projects/service => generated 88 bytes in 232 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 02 10:02:47.205862 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c17a96f-2eb0-4abf-9518-40919ee062ea None admin] Authenticating user token {{(pid=80378) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:02:47.222470 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG dbcounter [-] [80378] Writing DB stats keystone:SELECT=100 {{(pid=80378) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 10:02:47.242414 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: [pid: 80377|app: 0|req: 377/746] 10.222.0.82 () {58 vars in 1023 bytes} [Tue Aug 2 10:02:47 2022] POST /identity/v3/auth/tokens => generated 903 bytes in 136 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 02 10:02:47.244092 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3289c9ca-832e-4b7c-8c82-a06d16ca268f None None] Authenticating user token {{(pid=80377) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:02:47.365904 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c17a96f-2eb0-4abf-9518-40919ee062ea None admin] Validating token access rules against request {{(pid=80378) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:02:47.409637 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3289c9ca-832e-4b7c-8c82-a06d16ca268f None None] Validating token access rules against request {{(pid=80377) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:02:47.414603 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG dbcounter [-] [80377] Writing DB stats keystone:SELECT=100 {{(pid=80377) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 10:02:47.421151 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2b8553d-9414-4fc2-95d2-bd163e1fa1fa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '891005fa5ff74041b75747d16cc66798', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80378) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:02:47.422558 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2b8553d-9414-4fc2-95d2-bd163e1fa1fa None admin] REQUEST_METHOD: `GET` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:02:47.422558 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2b8553d-9414-4fc2-95d2-bd163e1fa1fa None admin] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:02:47.422782 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2b8553d-9414-4fc2-95d2-bd163e1fa1fa None admin] PATH_INFO: `/v3/domains` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:02:47.423908 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2b8553d-9414-4fc2-95d2-bd163e1fa1fa None admin] RBAC: Adding query filter params (name=Default) {{(pid=80378) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 02 10:02:47.424212 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2b8553d-9414-4fc2-95d2-bd163e1fa1fa None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:02:47.425990 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2b8553d-9414-4fc2-95d2-bd163e1fa1fa None admin] RBAC: Authorization granted {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:02:47.442836 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 370/747] 10.222.0.82 () {56 vars in 1109 bytes} [Tue Aug 2 10:02:47 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 238 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 02 10:02:47.443562 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2b8553d-9414-4fc2-95d2-bd163e1fa1fa None admin] Authenticating user token {{(pid=80378) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:02:47.467785 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6045153b-6b7c-412d-9d08-26755de6f4f0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '891005fa5ff74041b75747d16cc66798', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80377) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:02:47.468832 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6045153b-6b7c-412d-9d08-26755de6f4f0 None admin] REQUEST_METHOD: `GET` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:02:47.469431 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6045153b-6b7c-412d-9d08-26755de6f4f0 None admin] SCRIPT_NAME: `/identity` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:02:47.469735 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6045153b-6b7c-412d-9d08-26755de6f4f0 None admin] PATH_INFO: `/v3/domains` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:02:47.470972 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6045153b-6b7c-412d-9d08-26755de6f4f0 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80377) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 02 10:02:47.471532 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6045153b-6b7c-412d-9d08-26755de6f4f0 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:02:47.474093 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6045153b-6b7c-412d-9d08-26755de6f4f0 None admin] RBAC: Authorization granted {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:02:47.497290 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: [pid: 80377|app: 0|req: 378/748] 10.222.0.82 () {56 vars in 1109 bytes} [Tue Aug 2 10:02:47 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 254 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 02 10:02:47.498340 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6045153b-6b7c-412d-9d08-26755de6f4f0 None admin] Authenticating user token {{(pid=80377) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:02:47.549646 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG dbcounter [-] [80378] Writing DB stats keystone:SELECT=100 {{(pid=80378) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 10:02:47.593112 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2b8553d-9414-4fc2-95d2-bd163e1fa1fa None admin] Validating token access rules against request {{(pid=80378) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:02:47.643890 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f702cff-38e1-4dcd-9423-2303e46319a6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '891005fa5ff74041b75747d16cc66798', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80378) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:02:47.644841 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6045153b-6b7c-412d-9d08-26755de6f4f0 None admin] Validating token access rules against request {{(pid=80377) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:02:47.645134 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f702cff-38e1-4dcd-9423-2303e46319a6 None admin] REQUEST_METHOD: `GET` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:02:47.645506 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f702cff-38e1-4dcd-9423-2303e46319a6 None admin] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:02:47.645925 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f702cff-38e1-4dcd-9423-2303e46319a6 None admin] PATH_INFO: `/v3/projects` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:02:47.645925 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f702cff-38e1-4dcd-9423-2303e46319a6 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80378) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 02 10:02:47.647230 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f702cff-38e1-4dcd-9423-2303e46319a6 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:02:47.648956 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f702cff-38e1-4dcd-9423-2303e46319a6 None admin] RBAC: Authorization granted {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:02:47.676329 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 371/749] 10.222.0.82 () {56 vars in 1148 bytes} [Tue Aug 2 10:02:47 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 227 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 02 10:02:47.676329 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f702cff-38e1-4dcd-9423-2303e46319a6 None admin] Authenticating user token {{(pid=80378) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:02:47.699910 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31eba492-bdf8-474f-a650-37fdffe1d1d0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '891005fa5ff74041b75747d16cc66798', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80377) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:02:47.700913 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31eba492-bdf8-474f-a650-37fdffe1d1d0 None admin] REQUEST_METHOD: `POST` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:02:47.701471 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31eba492-bdf8-474f-a650-37fdffe1d1d0 None admin] SCRIPT_NAME: `/identity` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:02:47.702167 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31eba492-bdf8-474f-a650-37fdffe1d1d0 None admin] PATH_INFO: `/v3/services` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:02:47.703416 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31eba492-bdf8-474f-a650-37fdffe1d1d0 None admin] RBAC: Authorizing `identity:create_service()` {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:02:47.705823 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31eba492-bdf8-474f-a650-37fdffe1d1d0 None admin] RBAC: Authorization granted {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:02:47.728076 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: [pid: 80377|app: 0|req: 379/750] 10.222.0.82 () {60 vars in 1141 bytes} [Tue Aug 2 10:02:47 2022] POST /identity/v3/services => generated 240 bytes in 230 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 02 10:02:47.729703 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31eba492-bdf8-474f-a650-37fdffe1d1d0 None admin] Authenticating user token {{(pid=80377) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:02:47.792636 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG dbcounter [-] [80377] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80377) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 10:02:47.872482 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f702cff-38e1-4dcd-9423-2303e46319a6 None admin] Validating token access rules against request {{(pid=80378) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:02:47.921918 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG dbcounter [-] [80378] Writing DB stats keystone:SELECT=100 {{(pid=80378) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 10:02:47.944596 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21428e4d-6c43-4d4e-9e15-0ad2c83d21dc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '891005fa5ff74041b75747d16cc66798', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80378) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:02:47.945868 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21428e4d-6c43-4d4e-9e15-0ad2c83d21dc None admin] REQUEST_METHOD: `GET` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:02:47.946350 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21428e4d-6c43-4d4e-9e15-0ad2c83d21dc None admin] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:02:47.946698 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21428e4d-6c43-4d4e-9e15-0ad2c83d21dc None admin] PATH_INFO: `/v3/users/nova` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:02:47.946838 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31eba492-bdf8-474f-a650-37fdffe1d1d0 None admin] Validating token access rules against request {{(pid=80377) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:02:47.960855 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21428e4d-6c43-4d4e-9e15-0ad2c83d21dc None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:02:47.962708 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21428e4d-6c43-4d4e-9e15-0ad2c83d21dc None admin] RBAC: Authorization granted {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:02:47.980052 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application [None req-21428e4d-6c43-4d4e-9e15-0ad2c83d21dc None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Aug 02 10:02:47.980052 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 02 10:02:47.980052 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 02 10:02:47.980052 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 02 10:02:47.980052 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 02 10:02:47.980052 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 02 10:02:47.980052 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 02 10:02:47.980052 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 02 10:02:47.980052 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 02 10:02:47.980052 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 02 10:02:47.980052 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 02 10:02:47.980052 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 02 10:02:47.980052 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 02 10:02:47.980052 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 02 10:02:47.980052 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 02 10:02:47.980052 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 02 10:02:47.981163 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 02 10:02:47.981163 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 02 10:02:47.981163 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 02 10:02:47.981163 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 02 10:02:47.981163 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 02 10:02:47.981163 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 02 10:02:47.981163 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 02 10:02:47.981163 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 02 10:02:47.981163 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 02 10:02:47.981163 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application return self.get_or_create( Aug 02 10:02:47.981163 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 02 10:02:47.981163 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application with Lock( Aug 02 10:02:47.981163 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 02 10:02:47.981163 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application return self._enter() Aug 02 10:02:47.981163 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 02 10:02:47.981163 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 02 10:02:47.981163 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 02 10:02:47.981163 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application return self.creator() Aug 02 10:02:47.982203 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 02 10:02:47.982203 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application created_value = creator( Aug 02 10:02:47.982203 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 02 10:02:47.982203 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 02 10:02:47.982203 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 02 10:02:47.982203 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 02 10:02:47.982203 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 02 10:02:47.982203 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 02 10:02:47.982203 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Aug 02 10:02:47.982203 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application Aug 02 10:02:47.986418 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 372/751] 10.222.0.82 () {56 vars in 1093 bytes} [Tue Aug 2 10:02:47 2022] GET /identity/v3/users/nova => generated 82 bytes in 315 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 02 10:02:47.987430 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21428e4d-6c43-4d4e-9e15-0ad2c83d21dc None admin] Authenticating user token {{(pid=80378) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:02:48.027925 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d8c0747-652d-48fb-a75a-f7b252f2837b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '891005fa5ff74041b75747d16cc66798', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80377) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:02:48.028852 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d8c0747-652d-48fb-a75a-f7b252f2837b None admin] REQUEST_METHOD: `GET` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:02:48.029307 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d8c0747-652d-48fb-a75a-f7b252f2837b None admin] SCRIPT_NAME: `/identity` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:02:48.029722 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d8c0747-652d-48fb-a75a-f7b252f2837b None admin] PATH_INFO: `/v3/users/glance-swift` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:02:48.040557 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d8c0747-652d-48fb-a75a-f7b252f2837b None admin] RBAC: Authorizing `identity:get_user(user_id=glance-swift)` {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:02:48.043188 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d8c0747-652d-48fb-a75a-f7b252f2837b None admin] RBAC: Authorization granted {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:02:48.066661 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application [None req-9d8c0747-652d-48fb-a75a-f7b252f2837b None admin] Could not find user: glance-swift.: keystone.exception.UserNotFound: Could not find user: glance-swift. Aug 02 10:02:48.066661 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 02 10:02:48.066661 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 02 10:02:48.066661 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 02 10:02:48.066661 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 02 10:02:48.066661 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 02 10:02:48.066661 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 02 10:02:48.066661 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 02 10:02:48.066661 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 02 10:02:48.066661 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 02 10:02:48.066661 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 02 10:02:48.066661 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 02 10:02:48.066661 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 02 10:02:48.066661 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 02 10:02:48.066661 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 02 10:02:48.066661 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 02 10:02:48.067747 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 02 10:02:48.067747 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 02 10:02:48.067747 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 02 10:02:48.067747 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 02 10:02:48.067747 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 02 10:02:48.067747 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 02 10:02:48.067747 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 02 10:02:48.067747 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 02 10:02:48.067747 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 02 10:02:48.067747 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application return self.get_or_create( Aug 02 10:02:48.067747 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 02 10:02:48.067747 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application with Lock( Aug 02 10:02:48.067747 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 02 10:02:48.067747 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application return self._enter() Aug 02 10:02:48.067747 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 02 10:02:48.067747 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 02 10:02:48.067747 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 02 10:02:48.067747 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application return self.creator() Aug 02 10:02:48.068705 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 02 10:02:48.068705 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application created_value = creator( Aug 02 10:02:48.068705 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 02 10:02:48.068705 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 02 10:02:48.068705 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 02 10:02:48.068705 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 02 10:02:48.068705 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 02 10:02:48.068705 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 02 10:02:48.068705 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance-swift. Aug 02 10:02:48.068705 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application Aug 02 10:02:48.070572 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: [pid: 80377|app: 0|req: 380/752] 10.222.0.82 () {56 vars in 1117 bytes} [Tue Aug 2 10:02:47 2022] GET /identity/v3/users/glance-swift => generated 90 bytes in 340 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 02 10:02:48.070862 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df2aaaa9-3aab-4f7a-8b7a-15718f21d994 None None] REQUEST_METHOD: `GET` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:02:48.071247 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df2aaaa9-3aab-4f7a-8b7a-15718f21d994 None None] SCRIPT_NAME: `/identity` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:02:48.071737 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df2aaaa9-3aab-4f7a-8b7a-15718f21d994 None None] PATH_INFO: `/` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:02:48.077390 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: [pid: 80377|app: 0|req: 381/753] 10.222.0.82 () {54 vars in 926 bytes} [Tue Aug 2 10:02:48 2022] GET /identity => generated 270 bytes in 8 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 02 10:02:48.078990 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df2aaaa9-3aab-4f7a-8b7a-15718f21d994 None None] Authenticating user token {{(pid=80377) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:02:48.194987 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21428e4d-6c43-4d4e-9e15-0ad2c83d21dc None admin] Validating token access rules against request {{(pid=80378) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:02:48.200584 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG dbcounter [-] [80377] Writing DB stats keystone:SELECT=100 {{(pid=80377) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 10:02:48.227750 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df2aaaa9-3aab-4f7a-8b7a-15718f21d994 None None] Validating token access rules against request {{(pid=80377) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:02:48.248115 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fe843e1-6ee6-4a7d-a10c-d38cb61f66ef None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '891005fa5ff74041b75747d16cc66798', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80378) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:02:48.249250 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fe843e1-6ee6-4a7d-a10c-d38cb61f66ef None admin] REQUEST_METHOD: `GET` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:02:48.249961 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fe843e1-6ee6-4a7d-a10c-d38cb61f66ef None admin] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:02:48.250595 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fe843e1-6ee6-4a7d-a10c-d38cb61f66ef None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:02:48.252402 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1fe843e1-6ee6-4a7d-a10c-d38cb61f66ef None admin] RBAC: Adding query filter params (role.id=237966442f05411191105bf336bc14f3, scope.project.id=f1b38ebb804c41c69772c998aae72c41, user.id=bef1b2e714cb45c4a51eb6d3e15aad6d) {{(pid=80378) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 02 10:02:48.253001 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1fe843e1-6ee6-4a7d-a10c-d38cb61f66ef None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:02:48.256184 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1fe843e1-6ee6-4a7d-a10c-d38cb61f66ef None admin] RBAC: Authorization granted {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:02:48.273819 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 373/754] 10.222.0.82 () {56 vars in 1374 bytes} [Tue Aug 2 10:02:47 2022] GET /identity/v3/role_assignments?user.id=bef1b2e714cb45c4a51eb6d3e15aad6d&scope.project.id=f1b38ebb804c41c69772c998aae72c41&role.id=237966442f05411191105bf336bc14f3 => generated 259 bytes in 276 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 02 10:02:48.273819 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fe843e1-6ee6-4a7d-a10c-d38cb61f66ef None admin] Authenticating user token {{(pid=80378) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:02:48.309116 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-466bc7d9-dd06-4378-a024-d29723bfbe5b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '891005fa5ff74041b75747d16cc66798', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80377) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:02:48.310014 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-466bc7d9-dd06-4378-a024-d29723bfbe5b None admin] REQUEST_METHOD: `GET` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:02:48.310302 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-466bc7d9-dd06-4378-a024-d29723bfbe5b None admin] SCRIPT_NAME: `/identity` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:02:48.310580 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-466bc7d9-dd06-4378-a024-d29723bfbe5b None admin] PATH_INFO: `/v3/users` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:02:48.311530 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-466bc7d9-dd06-4378-a024-d29723bfbe5b None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=80377) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 02 10:02:48.311900 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-466bc7d9-dd06-4378-a024-d29723bfbe5b None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:02:48.313657 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-466bc7d9-dd06-4378-a024-d29723bfbe5b None admin] RBAC: Authorization granted {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:02:48.343535 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: [pid: 80377|app: 0|req: 382/755] 10.222.0.82 () {56 vars in 1133 bytes} [Tue Aug 2 10:02:48 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 259 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 02 10:02:48.343535 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99d9fc46-2516-4d05-a8d8-f8c497af27ef None None] REQUEST_METHOD: `POST` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:02:48.343535 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99d9fc46-2516-4d05-a8d8-f8c497af27ef None None] SCRIPT_NAME: `/identity` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:02:48.343535 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99d9fc46-2516-4d05-a8d8-f8c497af27ef None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:02:48.370065 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.auth.core [None req-99d9fc46-2516-4d05-a8d8-f8c497af27ef None None] MFA Rules not processed for user `891005fa5ff74041b75747d16cc66798`. Rule list: `[]` (Enabled: `True`). {{(pid=80377) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 02 10:02:48.371802 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.fernet_utils [None req-99d9fc46-2516-4d05-a8d8-f8c497af27ef None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80377) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 10:02:48.385645 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG dbcounter [-] [80378] Writing DB stats keystone:SELECT=100 {{(pid=80378) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 10:02:48.475950 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fe843e1-6ee6-4a7d-a10c-d38cb61f66ef None admin] Validating token access rules against request {{(pid=80378) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:02:48.484321 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: [pid: 80377|app: 0|req: 383/756] 10.222.0.82 () {58 vars in 1023 bytes} [Tue Aug 2 10:02:48 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 146 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 02 10:02:48.486214 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99d9fc46-2516-4d05-a8d8-f8c497af27ef None None] Authenticating user token {{(pid=80377) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:02:48.546954 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a6b23c6-e8ea-4dde-a28e-24b9816cd730 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '891005fa5ff74041b75747d16cc66798', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80378) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:02:48.550489 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a6b23c6-e8ea-4dde-a28e-24b9816cd730 None admin] REQUEST_METHOD: `GET` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:02:48.552526 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a6b23c6-e8ea-4dde-a28e-24b9816cd730 None admin] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:02:48.553259 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a6b23c6-e8ea-4dde-a28e-24b9816cd730 None admin] PATH_INFO: `/v3/users` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:02:48.555325 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a6b23c6-e8ea-4dde-a28e-24b9816cd730 None admin] RBAC: Adding query filter params (domain_id=default, name=glance-swift) {{(pid=80378) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 02 10:02:48.556184 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a6b23c6-e8ea-4dde-a28e-24b9816cd730 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:02:48.559212 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a6b23c6-e8ea-4dde-a28e-24b9816cd730 None admin] RBAC: Authorization granted {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:02:48.587239 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 374/757] 10.222.0.82 () {56 vars in 1149 bytes} [Tue Aug 2 10:02:48 2022] GET /identity/v3/users?domain_id=default&name=glance-swift => generated 384 bytes in 323 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 02 10:02:48.588666 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a6b23c6-e8ea-4dde-a28e-24b9816cd730 None admin] Authenticating user token {{(pid=80378) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:02:48.623653 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG dbcounter [-] [80377] Writing DB stats keystone:SELECT=100 {{(pid=80377) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 10:02:48.707416 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99d9fc46-2516-4d05-a8d8-f8c497af27ef None None] Validating token access rules against request {{(pid=80377) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:02:48.774872 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a6b23c6-e8ea-4dde-a28e-24b9816cd730 None admin] Validating token access rules against request {{(pid=80378) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:02:48.775549 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e5eca7e-a15c-463b-a218-51d249d51541 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '891005fa5ff74041b75747d16cc66798', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80377) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:02:48.776480 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e5eca7e-a15c-463b-a218-51d249d51541 None admin] REQUEST_METHOD: `GET` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:02:48.776913 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e5eca7e-a15c-463b-a218-51d249d51541 None admin] SCRIPT_NAME: `/identity` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:02:48.777348 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e5eca7e-a15c-463b-a218-51d249d51541 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:02:48.796113 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG dbcounter [-] [80378] Writing DB stats keystone:SELECT=100 {{(pid=80378) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 10:02:48.796674 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e5eca7e-a15c-463b-a218-51d249d51541 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:02:48.798748 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e5eca7e-a15c-463b-a218-51d249d51541 None admin] RBAC: Authorization granted {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:02:48.820347 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application [None req-0e5eca7e-a15c-463b-a218-51d249d51541 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 02 10:02:48.820347 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 02 10:02:48.820347 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 02 10:02:48.820347 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 02 10:02:48.820347 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 02 10:02:48.820347 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 02 10:02:48.820347 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 02 10:02:48.820347 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 02 10:02:48.820347 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 02 10:02:48.820347 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 02 10:02:48.820347 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 02 10:02:48.820347 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 02 10:02:48.820347 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 02 10:02:48.820347 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 02 10:02:48.820347 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 02 10:02:48.822163 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 02 10:02:48.822163 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 02 10:02:48.822163 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 02 10:02:48.822163 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 02 10:02:48.822163 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 02 10:02:48.822163 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 02 10:02:48.822163 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application return self.get_or_create( Aug 02 10:02:48.822163 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 02 10:02:48.822163 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application with Lock( Aug 02 10:02:48.822163 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 02 10:02:48.822163 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application return self._enter() Aug 02 10:02:48.822163 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 02 10:02:48.822163 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 02 10:02:48.822163 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 02 10:02:48.822163 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application return self.creator() Aug 02 10:02:48.822163 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 02 10:02:48.822163 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application created_value = creator( Aug 02 10:02:48.824620 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 02 10:02:48.824620 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 02 10:02:48.824620 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 02 10:02:48.824620 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application Aug 02 10:02:48.827658 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: [pid: 80377|app: 0|req: 384/758] 10.222.0.82 () {56 vars in 1108 bytes} [Tue Aug 2 10:02:48 2022] GET /identity/v3/domains/Default => generated 87 bytes in 342 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 02 10:02:48.828680 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e5eca7e-a15c-463b-a218-51d249d51541 None admin] Authenticating user token {{(pid=80377) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:02:48.858556 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8ec5b01-ad2a-4d6c-8bb2-b73c3b0ae142 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '891005fa5ff74041b75747d16cc66798', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80378) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:02:48.860303 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8ec5b01-ad2a-4d6c-8bb2-b73c3b0ae142 None admin] REQUEST_METHOD: `GET` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:02:48.861170 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8ec5b01-ad2a-4d6c-8bb2-b73c3b0ae142 None admin] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:02:48.861971 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8ec5b01-ad2a-4d6c-8bb2-b73c3b0ae142 None admin] PATH_INFO: `/v3/services/block-storage` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:02:48.863419 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8ec5b01-ad2a-4d6c-8bb2-b73c3b0ae142 None admin] RBAC: Authorizing `identity:get_service(service_id=block-storage)` {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:02:48.866472 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8ec5b01-ad2a-4d6c-8bb2-b73c3b0ae142 None admin] RBAC: Authorization granted {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:02:48.888645 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application [None req-a8ec5b01-ad2a-4d6c-8bb2-b73c3b0ae142 None admin] Could not find service: block-storage.: keystone.exception.ServiceNotFound: Could not find service: block-storage. Aug 02 10:02:48.888645 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 02 10:02:48.888645 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Aug 02 10:02:48.888645 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Aug 02 10:02:48.888645 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Aug 02 10:02:48.888645 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Aug 02 10:02:48.888645 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Aug 02 10:02:48.888645 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 02 10:02:48.888645 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: block-storage. Aug 02 10:02:48.888645 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application Aug 02 10:02:48.888645 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 02 10:02:48.888645 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application Aug 02 10:02:48.888645 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 02 10:02:48.888645 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 02 10:02:48.888645 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 02 10:02:48.888645 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 02 10:02:48.891309 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 02 10:02:48.891309 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 02 10:02:48.891309 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 02 10:02:48.891309 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 02 10:02:48.891309 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 02 10:02:48.891309 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 02 10:02:48.891309 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 02 10:02:48.891309 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Aug 02 10:02:48.891309 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application return self._get_service(service_id) Aug 02 10:02:48.891309 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Aug 02 10:02:48.891309 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Aug 02 10:02:48.891309 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 02 10:02:48.891309 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 02 10:02:48.891309 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 02 10:02:48.891309 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 02 10:02:48.891309 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 02 10:02:48.892966 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application return self.get_or_create( Aug 02 10:02:48.892966 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 02 10:02:48.892966 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application with Lock( Aug 02 10:02:48.892966 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 02 10:02:48.892966 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application return self._enter() Aug 02 10:02:48.892966 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 02 10:02:48.892966 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 02 10:02:48.892966 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 02 10:02:48.892966 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application return self.creator() Aug 02 10:02:48.892966 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 02 10:02:48.892966 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application created_value = creator( Aug 02 10:02:48.892966 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Aug 02 10:02:48.892966 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 02 10:02:48.892966 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: block-storage. Aug 02 10:02:48.892966 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application Aug 02 10:02:48.899694 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 375/759] 10.222.0.82 () {56 vars in 1129 bytes} [Tue Aug 2 10:02:48 2022] GET /identity/v3/services/block-storage => generated 94 bytes in 306 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 02 10:02:48.899694 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8ec5b01-ad2a-4d6c-8bb2-b73c3b0ae142 None admin] Authenticating user token {{(pid=80378) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:02:48.992607 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e5eca7e-a15c-463b-a218-51d249d51541 None admin] Validating token access rules against request {{(pid=80377) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:02:49.023215 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG dbcounter [-] [80377] Writing DB stats keystone:SELECT=100 {{(pid=80377) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 10:02:49.035610 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8ec5b01-ad2a-4d6c-8bb2-b73c3b0ae142 None admin] Validating token access rules against request {{(pid=80378) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:02:49.046799 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75c7bd15-c917-4fd8-8437-7bb42d960fca None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '891005fa5ff74041b75747d16cc66798', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80377) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:02:49.048527 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75c7bd15-c917-4fd8-8437-7bb42d960fca None admin] REQUEST_METHOD: `GET` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:02:49.049366 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75c7bd15-c917-4fd8-8437-7bb42d960fca None admin] SCRIPT_NAME: `/identity` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:02:49.050628 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75c7bd15-c917-4fd8-8437-7bb42d960fca None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:02:49.087200 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8154aca4-9652-4ce7-9f0e-e1ec9a6bd689 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '891005fa5ff74041b75747d16cc66798', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80378) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:02:49.088105 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8154aca4-9652-4ce7-9f0e-e1ec9a6bd689 None admin] REQUEST_METHOD: `GET` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:02:49.088548 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8154aca4-9652-4ce7-9f0e-e1ec9a6bd689 None admin] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:02:49.088974 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8154aca4-9652-4ce7-9f0e-e1ec9a6bd689 None admin] PATH_INFO: `/v3/domains` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:02:49.090453 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8154aca4-9652-4ce7-9f0e-e1ec9a6bd689 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80378) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 02 10:02:49.090453 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8154aca4-9652-4ce7-9f0e-e1ec9a6bd689 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:02:49.091964 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8154aca4-9652-4ce7-9f0e-e1ec9a6bd689 None admin] RBAC: Authorization granted {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:02:49.117385 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 376/760] 10.222.0.82 () {56 vars in 1109 bytes} [Tue Aug 2 10:02:48 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 223 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 02 10:02:49.118469 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8154aca4-9652-4ce7-9f0e-e1ec9a6bd689 None admin] Authenticating user token {{(pid=80378) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:02:49.158619 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG dbcounter [-] [80378] Writing DB stats keystone:SELECT=100 {{(pid=80378) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 10:02:49.181346 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75c7bd15-c917-4fd8-8437-7bb42d960fca None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:02:49.185548 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75c7bd15-c917-4fd8-8437-7bb42d960fca None admin] RBAC: Authorization granted {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:02:49.286750 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8154aca4-9652-4ce7-9f0e-e1ec9a6bd689 None admin] Validating token access rules against request {{(pid=80378) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:02:49.338998 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7cee514-c01a-430e-b4da-d545b442f326 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '891005fa5ff74041b75747d16cc66798', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80378) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:02:49.340210 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7cee514-c01a-430e-b4da-d545b442f326 None admin] REQUEST_METHOD: `GET` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:02:49.340764 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7cee514-c01a-430e-b4da-d545b442f326 None admin] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:02:49.341248 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7cee514-c01a-430e-b4da-d545b442f326 None admin] PATH_INFO: `/v3/services` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:02:49.342739 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7cee514-c01a-430e-b4da-d545b442f326 None admin] RBAC: Adding query filter params (name=block-storage) {{(pid=80378) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 02 10:02:49.343280 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7cee514-c01a-430e-b4da-d545b442f326 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:02:49.347096 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7cee514-c01a-430e-b4da-d545b442f326 None admin] RBAC: Authorization granted {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:02:49.355228 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 377/761] 10.222.0.82 () {56 vars in 1124 bytes} [Tue Aug 2 10:02:49 2022] GET /identity/v3/services?name=block-storage => generated 130 bytes in 237 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 02 10:02:49.356705 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7cee514-c01a-430e-b4da-d545b442f326 None admin] Authenticating user token {{(pid=80378) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:02:49.401499 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: [pid: 80377|app: 0|req: 385/762] 10.222.0.82 () {58 vars in 1282 bytes} [Tue Aug 2 10:02:48 2022] GET /identity/v3/auth/tokens => generated 1002 bytes in 573 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Aug 02 10:02:49.404225 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75c7bd15-c917-4fd8-8437-7bb42d960fca None admin] Authenticating user token {{(pid=80377) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:02:49.441391 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG dbcounter [-] [80377] Writing DB stats keystone:SELECT=100 {{(pid=80377) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 10:02:49.531058 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG dbcounter [-] [80378] Writing DB stats keystone:SELECT=100 {{(pid=80378) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 10:02:49.543629 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7cee514-c01a-430e-b4da-d545b442f326 None admin] Validating token access rules against request {{(pid=80378) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:02:49.559405 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75c7bd15-c917-4fd8-8437-7bb42d960fca None admin] Validating token access rules against request {{(pid=80377) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:02:49.601938 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c00dc352-9a06-4b9e-bd44-e19dcb5b3837 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '891005fa5ff74041b75747d16cc66798', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80378) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:02:49.602728 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c00dc352-9a06-4b9e-bd44-e19dcb5b3837 None admin] REQUEST_METHOD: `GET` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:02:49.603322 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c00dc352-9a06-4b9e-bd44-e19dcb5b3837 None admin] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:02:49.603786 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c00dc352-9a06-4b9e-bd44-e19dcb5b3837 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:02:49.614454 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c00dc352-9a06-4b9e-bd44-e19dcb5b3837 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:02:49.616665 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c00dc352-9a06-4b9e-bd44-e19dcb5b3837 None admin] RBAC: Authorization granted {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:02:49.627160 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90224733-8f9a-40a3-9663-fe75940f5485 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '891005fa5ff74041b75747d16cc66798', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80377) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:02:49.631923 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90224733-8f9a-40a3-9663-fe75940f5485 None admin] REQUEST_METHOD: `GET` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:02:49.632404 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90224733-8f9a-40a3-9663-fe75940f5485 None admin] SCRIPT_NAME: `/identity` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:02:49.632958 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90224733-8f9a-40a3-9663-fe75940f5485 None admin] PATH_INFO: `/v3/services` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:02:49.634078 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90224733-8f9a-40a3-9663-fe75940f5485 None admin] RBAC: Adding query filter params (type=block-storage) {{(pid=80377) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 02 10:02:49.634624 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90224733-8f9a-40a3-9663-fe75940f5485 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:02:49.636531 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90224733-8f9a-40a3-9663-fe75940f5485 None admin] RBAC: Authorization granted {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:02:49.641043 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application [None req-c00dc352-9a06-4b9e-bd44-e19dcb5b3837 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 02 10:02:49.641043 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 02 10:02:49.641043 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 02 10:02:49.641043 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 02 10:02:49.641043 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 02 10:02:49.641043 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 02 10:02:49.641043 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 02 10:02:49.641043 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 02 10:02:49.641043 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 02 10:02:49.641043 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 02 10:02:49.641043 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 02 10:02:49.641043 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 02 10:02:49.641043 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 02 10:02:49.641043 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 02 10:02:49.641043 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 02 10:02:49.643198 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 02 10:02:49.643198 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 02 10:02:49.643198 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 02 10:02:49.643198 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 02 10:02:49.643198 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 02 10:02:49.643198 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 02 10:02:49.643198 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application return self.get_or_create( Aug 02 10:02:49.643198 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 02 10:02:49.643198 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application with Lock( Aug 02 10:02:49.643198 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 02 10:02:49.643198 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application return self._enter() Aug 02 10:02:49.643198 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 02 10:02:49.643198 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 02 10:02:49.643198 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 02 10:02:49.643198 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application return self.creator() Aug 02 10:02:49.643198 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 02 10:02:49.643198 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application created_value = creator( Aug 02 10:02:49.644717 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 02 10:02:49.644717 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 02 10:02:49.644717 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 02 10:02:49.644717 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 02 10:02:49.644717 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 02 10:02:49.644717 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 02 10:02:49.644717 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 02 10:02:49.644717 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application Aug 02 10:02:49.644717 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 378/763] 10.222.0.82 () {56 vars in 1111 bytes} [Tue Aug 2 10:02:49 2022] GET /identity/v3/projects/service => generated 88 bytes in 288 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 02 10:02:49.645827 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c00dc352-9a06-4b9e-bd44-e19dcb5b3837 None admin] Authenticating user token {{(pid=80378) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:02:49.651029 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: [pid: 80377|app: 0|req: 386/764] 10.222.0.82 () {56 vars in 1124 bytes} [Tue Aug 2 10:02:49 2022] GET /identity/v3/services?type=block-storage => generated 367 bytes in 246 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 02 10:02:49.656398 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90224733-8f9a-40a3-9663-fe75940f5485 None admin] Authenticating user token {{(pid=80377) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:02:49.786234 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG dbcounter [-] [80377] Writing DB stats keystone:SELECT=100 {{(pid=80377) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 10:02:49.793804 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c00dc352-9a06-4b9e-bd44-e19dcb5b3837 None admin] Validating token access rules against request {{(pid=80378) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:02:49.811924 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90224733-8f9a-40a3-9663-fe75940f5485 None admin] Validating token access rules against request {{(pid=80377) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:02:49.858525 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4b22b92-c7d0-48cd-8b6e-43bbf746b5de None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '891005fa5ff74041b75747d16cc66798', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80378) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:02:49.859886 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4b22b92-c7d0-48cd-8b6e-43bbf746b5de None admin] REQUEST_METHOD: `GET` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:02:49.860385 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4b22b92-c7d0-48cd-8b6e-43bbf746b5de None admin] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:02:49.861303 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4b22b92-c7d0-48cd-8b6e-43bbf746b5de None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:02:49.862231 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bf1a09c-24a9-4908-a131-57090d4361bf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '891005fa5ff74041b75747d16cc66798', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80377) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:02:49.863126 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bf1a09c-24a9-4908-a131-57090d4361bf None admin] REQUEST_METHOD: `GET` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:02:49.863585 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bf1a09c-24a9-4908-a131-57090d4361bf None admin] SCRIPT_NAME: `/identity` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:02:49.864283 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bf1a09c-24a9-4908-a131-57090d4361bf None admin] PATH_INFO: `/v3/endpoints` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:02:49.865637 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1bf1a09c-24a9-4908-a131-57090d4361bf None admin] RBAC: Adding query filter params (interface=public, service_id=af5eca8015e743c9b7aaea0ec7170c56, region_id=RegionOne) {{(pid=80377) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 02 10:02:49.866197 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1bf1a09c-24a9-4908-a131-57090d4361bf None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:02:49.867812 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1bf1a09c-24a9-4908-a131-57090d4361bf None admin] RBAC: Authorization granted {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:02:49.877122 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4b22b92-c7d0-48cd-8b6e-43bbf746b5de None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:02:49.878889 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4b22b92-c7d0-48cd-8b6e-43bbf746b5de None admin] RBAC: Authorization granted {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:02:49.879354 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: [pid: 80377|app: 0|req: 387/765] 10.222.0.82 () {56 vars in 1251 bytes} [Tue Aug 2 10:02:49 2022] GET /identity/v3/endpoints?service_id=af5eca8015e743c9b7aaea0ec7170c56&interface=public®ion_id=RegionOne => generated 194 bytes in 224 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 02 10:02:49.880500 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bf1a09c-24a9-4908-a131-57090d4361bf None admin] Authenticating user token {{(pid=80377) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:02:49.904529 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application [None req-b4b22b92-c7d0-48cd-8b6e-43bbf746b5de None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 02 10:02:49.904529 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 02 10:02:49.904529 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 02 10:02:49.904529 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 02 10:02:49.904529 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 02 10:02:49.904529 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 02 10:02:49.904529 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 02 10:02:49.904529 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 02 10:02:49.904529 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 02 10:02:49.904529 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 02 10:02:49.904529 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 02 10:02:49.904529 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 02 10:02:49.904529 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 02 10:02:49.904529 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 02 10:02:49.904529 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 02 10:02:49.905523 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 02 10:02:49.905523 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 02 10:02:49.905523 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 02 10:02:49.905523 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 02 10:02:49.905523 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 02 10:02:49.905523 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 02 10:02:49.905523 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application return self.get_or_create( Aug 02 10:02:49.905523 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 02 10:02:49.905523 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application with Lock( Aug 02 10:02:49.905523 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 02 10:02:49.905523 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application return self._enter() Aug 02 10:02:49.905523 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 02 10:02:49.905523 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 02 10:02:49.905523 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 02 10:02:49.905523 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application return self.creator() Aug 02 10:02:49.905523 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 02 10:02:49.905523 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application created_value = creator( Aug 02 10:02:49.906553 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 02 10:02:49.906553 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 02 10:02:49.906553 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 02 10:02:49.906553 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application Aug 02 10:02:49.908779 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 379/766] 10.222.0.82 () {56 vars in 1108 bytes} [Tue Aug 2 10:02:49 2022] GET /identity/v3/domains/Default => generated 87 bytes in 264 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 02 10:02:49.909807 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4b22b92-c7d0-48cd-8b6e-43bbf746b5de None admin] Authenticating user token {{(pid=80378) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:02:49.927979 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG dbcounter [-] [80378] Writing DB stats keystone:SELECT=100 {{(pid=80378) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 10:02:50.026241 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bf1a09c-24a9-4908-a131-57090d4361bf None admin] Validating token access rules against request {{(pid=80377) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:02:50.046239 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4b22b92-c7d0-48cd-8b6e-43bbf746b5de None admin] Validating token access rules against request {{(pid=80378) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:02:50.075990 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dafeae98-37d2-4152-97c9-3583e9aa2afb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '891005fa5ff74041b75747d16cc66798', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80377) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:02:50.077278 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dafeae98-37d2-4152-97c9-3583e9aa2afb None admin] REQUEST_METHOD: `GET` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:02:50.078012 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dafeae98-37d2-4152-97c9-3583e9aa2afb None admin] SCRIPT_NAME: `/identity` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:02:50.078712 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dafeae98-37d2-4152-97c9-3583e9aa2afb None admin] PATH_INFO: `/v3/projects` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:02:50.080788 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dafeae98-37d2-4152-97c9-3583e9aa2afb None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80377) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 02 10:02:50.081445 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dafeae98-37d2-4152-97c9-3583e9aa2afb None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:02:50.084138 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dafeae98-37d2-4152-97c9-3583e9aa2afb None admin] RBAC: Authorization granted {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:02:50.085159 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73281f1f-de24-47dc-8c9b-cec8f0da8df4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '891005fa5ff74041b75747d16cc66798', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80378) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:02:50.085895 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73281f1f-de24-47dc-8c9b-cec8f0da8df4 None admin] REQUEST_METHOD: `GET` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:02:50.086261 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73281f1f-de24-47dc-8c9b-cec8f0da8df4 None admin] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:02:50.086636 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73281f1f-de24-47dc-8c9b-cec8f0da8df4 None admin] PATH_INFO: `/v3/services` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:02:50.087367 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73281f1f-de24-47dc-8c9b-cec8f0da8df4 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:02:50.089355 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73281f1f-de24-47dc-8c9b-cec8f0da8df4 None admin] RBAC: Authorization granted {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:02:50.097141 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 380/767] 10.222.0.82 () {56 vars in 1087 bytes} [Tue Aug 2 10:02:49 2022] GET /identity/v3/services => generated 772 bytes in 186 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 02 10:02:50.097141 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73281f1f-de24-47dc-8c9b-cec8f0da8df4 None admin] Authenticating user token {{(pid=80378) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:02:50.111969 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: [pid: 80377|app: 0|req: 388/768] 10.222.0.82 () {56 vars in 1148 bytes} [Tue Aug 2 10:02:49 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 232 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 02 10:02:50.121968 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dafeae98-37d2-4152-97c9-3583e9aa2afb None admin] Authenticating user token {{(pid=80377) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:02:50.152940 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG dbcounter [-] [80377] Writing DB stats keystone:SELECT=100 {{(pid=80377) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 10:02:50.205983 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG dbcounter [-] [80378] Writing DB stats keystone:SELECT=100 {{(pid=80378) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 10:02:50.250285 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73281f1f-de24-47dc-8c9b-cec8f0da8df4 None admin] Validating token access rules against request {{(pid=80378) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:02:50.306642 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e2508b0-c1b2-4e7e-950e-321e32dda501 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '891005fa5ff74041b75747d16cc66798', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80378) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:02:50.307350 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e2508b0-c1b2-4e7e-950e-321e32dda501 None admin] REQUEST_METHOD: `GET` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:02:50.307868 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e2508b0-c1b2-4e7e-950e-321e32dda501 None admin] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:02:50.308237 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e2508b0-c1b2-4e7e-950e-321e32dda501 None admin] PATH_INFO: `/v3/domains` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:02:50.309043 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e2508b0-c1b2-4e7e-950e-321e32dda501 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80378) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 02 10:02:50.310327 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e2508b0-c1b2-4e7e-950e-321e32dda501 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:02:50.311963 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e2508b0-c1b2-4e7e-950e-321e32dda501 None admin] RBAC: Authorization granted {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:02:50.330774 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 381/769] 10.222.0.82 () {56 vars in 1109 bytes} [Tue Aug 2 10:02:50 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 234 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 02 10:02:50.337836 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e2508b0-c1b2-4e7e-950e-321e32dda501 None admin] Authenticating user token {{(pid=80378) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:02:50.410818 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dafeae98-37d2-4152-97c9-3583e9aa2afb None admin] Validating token access rules against request {{(pid=80377) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:02:50.503471 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-294cf9ce-317d-4cc4-92df-169a203f9fb3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '891005fa5ff74041b75747d16cc66798', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80377) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:02:50.504586 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-294cf9ce-317d-4cc4-92df-169a203f9fb3 None admin] REQUEST_METHOD: `PUT` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:02:50.505090 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-294cf9ce-317d-4cc4-92df-169a203f9fb3 None admin] SCRIPT_NAME: `/identity` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:02:50.505578 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-294cf9ce-317d-4cc4-92df-169a203f9fb3 None admin] PATH_INFO: `/v3/projects/f1b38ebb804c41c69772c998aae72c41/users/88af518196674fe593ad88d30111fb7d/roles/237966442f05411191105bf336bc14f3` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:02:50.513043 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-294cf9ce-317d-4cc4-92df-169a203f9fb3 None admin] RBAC: Authorizing `identity:create_grant(project_id=f1b38ebb804c41c69772c998aae72c41, user_id=88af518196674fe593ad88d30111fb7d, role_id=237966442f05411191105bf336bc14f3)` {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:02:50.525151 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-294cf9ce-317d-4cc4-92df-169a203f9fb3 None admin] RBAC: Authorization granted {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:02:50.569949 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: [pid: 80377|app: 0|req: 389/770] 10.222.0.82 () {58 vars in 1439 bytes} [Tue Aug 2 10:02:50 2022] PUT /identity/v3/projects/f1b38ebb804c41c69772c998aae72c41/users/88af518196674fe593ad88d30111fb7d/roles/237966442f05411191105bf336bc14f3 => generated 0 bytes in 452 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 02 10:02:50.571065 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e2508b0-c1b2-4e7e-950e-321e32dda501 None admin] Validating token access rules against request {{(pid=80378) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:02:50.647050 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG dbcounter [-] [80378] Writing DB stats keystone:SELECT=100 {{(pid=80378) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 10:02:50.655409 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41cd0dc6-1af9-4ec7-b1a5-47308059abed None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '891005fa5ff74041b75747d16cc66798', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80378) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:02:50.657163 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41cd0dc6-1af9-4ec7-b1a5-47308059abed None admin] REQUEST_METHOD: `GET` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:02:50.658596 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41cd0dc6-1af9-4ec7-b1a5-47308059abed None admin] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:02:50.659092 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41cd0dc6-1af9-4ec7-b1a5-47308059abed None admin] PATH_INFO: `/v3/projects/service` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:02:50.674019 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41cd0dc6-1af9-4ec7-b1a5-47308059abed None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:02:50.677184 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41cd0dc6-1af9-4ec7-b1a5-47308059abed None admin] RBAC: Authorization granted {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:02:50.688225 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46cc436e-a8c2-4889-98ac-66b4f8373224 None None] REQUEST_METHOD: `GET` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:02:50.690430 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46cc436e-a8c2-4889-98ac-66b4f8373224 None None] SCRIPT_NAME: `/identity` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:02:50.691201 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46cc436e-a8c2-4889-98ac-66b4f8373224 None None] PATH_INFO: `/` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:02:50.701919 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: [pid: 80377|app: 0|req: 390/771] 10.222.0.82 () {54 vars in 926 bytes} [Tue Aug 2 10:02:50 2022] GET /identity => generated 270 bytes in 8 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 02 10:02:50.704309 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application [None req-41cd0dc6-1af9-4ec7-b1a5-47308059abed None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 02 10:02:50.704309 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 02 10:02:50.704309 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 02 10:02:50.704309 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 02 10:02:50.704309 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 02 10:02:50.704309 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 02 10:02:50.704309 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 02 10:02:50.704309 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 02 10:02:50.704309 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 02 10:02:50.704309 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 02 10:02:50.704309 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 02 10:02:50.704309 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 02 10:02:50.704309 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 02 10:02:50.704309 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 02 10:02:50.704309 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 02 10:02:50.706057 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 02 10:02:50.706057 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 02 10:02:50.706057 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 02 10:02:50.706057 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 02 10:02:50.706057 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 02 10:02:50.706057 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 02 10:02:50.706057 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application return self.get_or_create( Aug 02 10:02:50.706057 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 02 10:02:50.706057 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application with Lock( Aug 02 10:02:50.706057 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 02 10:02:50.706057 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application return self._enter() Aug 02 10:02:50.706057 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 02 10:02:50.706057 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 02 10:02:50.706057 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 02 10:02:50.706057 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application return self.creator() Aug 02 10:02:50.706057 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 02 10:02:50.706057 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application created_value = creator( Aug 02 10:02:50.707475 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 02 10:02:50.707475 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 02 10:02:50.707475 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 02 10:02:50.707475 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 02 10:02:50.707475 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 02 10:02:50.707475 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 02 10:02:50.707475 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 02 10:02:50.707475 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application Aug 02 10:02:50.707475 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 382/772] 10.222.0.82 () {56 vars in 1111 bytes} [Tue Aug 2 10:02:50 2022] GET /identity/v3/projects/service => generated 88 bytes in 370 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 02 10:02:50.708991 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa69bca0-5d23-42ec-a946-f322436a8563 None None] REQUEST_METHOD: `POST` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:02:50.709480 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa69bca0-5d23-42ec-a946-f322436a8563 None None] SCRIPT_NAME: `/identity` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:02:50.713388 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa69bca0-5d23-42ec-a946-f322436a8563 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:02:50.714985 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41cd0dc6-1af9-4ec7-b1a5-47308059abed None admin] Authenticating user token {{(pid=80378) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:02:50.766780 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.auth.core [None req-aa69bca0-5d23-42ec-a946-f322436a8563 None None] MFA Rules not processed for user `891005fa5ff74041b75747d16cc66798`. Rule list: `[]` (Enabled: `True`). {{(pid=80377) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 02 10:02:50.769128 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.fernet_utils [None req-aa69bca0-5d23-42ec-a946-f322436a8563 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80377) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 10:02:50.854978 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41cd0dc6-1af9-4ec7-b1a5-47308059abed None admin] Validating token access rules against request {{(pid=80378) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:02:50.940576 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG dbcounter [-] [80377] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80377) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 10:02:50.948962 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ff79717-0fb0-4792-a786-d87c6580adba None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '891005fa5ff74041b75747d16cc66798', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80378) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:02:50.949606 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ff79717-0fb0-4792-a786-d87c6580adba None admin] REQUEST_METHOD: `GET` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:02:50.949953 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ff79717-0fb0-4792-a786-d87c6580adba None admin] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:02:50.950278 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ff79717-0fb0-4792-a786-d87c6580adba None admin] PATH_INFO: `/v3/projects` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:02:50.951083 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ff79717-0fb0-4792-a786-d87c6580adba None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80378) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 02 10:02:50.951395 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ff79717-0fb0-4792-a786-d87c6580adba None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:02:50.952885 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ff79717-0fb0-4792-a786-d87c6580adba None admin] RBAC: Authorization granted {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:02:50.961914 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: [pid: 80377|app: 0|req: 391/773] 10.222.0.82 () {58 vars in 1023 bytes} [Tue Aug 2 10:02:50 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 254 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 02 10:02:50.981747 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 383/774] 10.222.0.82 () {56 vars in 1148 bytes} [Tue Aug 2 10:02:50 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 267 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 02 10:02:50.988985 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa69bca0-5d23-42ec-a946-f322436a8563 None None] Authenticating user token {{(pid=80377) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:02:51.004371 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ff79717-0fb0-4792-a786-d87c6580adba None admin] Authenticating user token {{(pid=80378) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:02:51.109487 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG dbcounter [-] [80378] Writing DB stats keystone:SELECT=100 {{(pid=80378) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 10:02:51.190148 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ff79717-0fb0-4792-a786-d87c6580adba None admin] Validating token access rules against request {{(pid=80378) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:02:51.190781 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa69bca0-5d23-42ec-a946-f322436a8563 None None] Validating token access rules against request {{(pid=80377) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:02:51.247603 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-360f562a-1175-46c6-9378-4566643fa64a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '891005fa5ff74041b75747d16cc66798', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80378) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:02:51.248618 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-360f562a-1175-46c6-9378-4566643fa64a None admin] REQUEST_METHOD: `GET` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:02:51.249078 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-360f562a-1175-46c6-9378-4566643fa64a None admin] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:02:51.249548 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-360f562a-1175-46c6-9378-4566643fa64a None admin] PATH_INFO: `/v3/services/network` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:02:51.250744 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-360f562a-1175-46c6-9378-4566643fa64a None admin] RBAC: Authorizing `identity:get_service(service_id=network)` {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:02:51.253143 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-360f562a-1175-46c6-9378-4566643fa64a None admin] RBAC: Authorization granted {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:02:51.259728 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0cd381f-3be9-49c4-a216-3d49afc03eb2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '891005fa5ff74041b75747d16cc66798', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80377) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:02:51.261121 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0cd381f-3be9-49c4-a216-3d49afc03eb2 None admin] REQUEST_METHOD: `GET` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:02:51.261960 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0cd381f-3be9-49c4-a216-3d49afc03eb2 None admin] SCRIPT_NAME: `/identity` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:02:51.262516 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0cd381f-3be9-49c4-a216-3d49afc03eb2 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:02:51.263595 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0cd381f-3be9-49c4-a216-3d49afc03eb2 None admin] RBAC: Adding query filter params (role.id=02003a0be60f47f2a8dfea17053af52c, scope.project.id=f1b38ebb804c41c69772c998aae72c41, user.id=bd8dd72086db48b8aaac2ed0058872df) {{(pid=80377) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 02 10:02:51.264206 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0cd381f-3be9-49c4-a216-3d49afc03eb2 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:02:51.266656 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application [None req-360f562a-1175-46c6-9378-4566643fa64a None admin] Could not find service: network.: keystone.exception.ServiceNotFound: Could not find service: network. Aug 02 10:02:51.266656 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 02 10:02:51.266656 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Aug 02 10:02:51.266656 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Aug 02 10:02:51.266656 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Aug 02 10:02:51.266656 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Aug 02 10:02:51.266656 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Aug 02 10:02:51.266656 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 02 10:02:51.266656 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: network. Aug 02 10:02:51.266656 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application Aug 02 10:02:51.266656 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 02 10:02:51.266656 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application Aug 02 10:02:51.266656 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 02 10:02:51.266656 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 02 10:02:51.266656 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 02 10:02:51.266656 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 02 10:02:51.266656 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 02 10:02:51.268514 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 02 10:02:51.268514 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 02 10:02:51.268514 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 02 10:02:51.268514 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 02 10:02:51.268514 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 02 10:02:51.268514 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 02 10:02:51.268514 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Aug 02 10:02:51.268514 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application return self._get_service(service_id) Aug 02 10:02:51.268514 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Aug 02 10:02:51.268514 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Aug 02 10:02:51.268514 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 02 10:02:51.268514 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 02 10:02:51.268514 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 02 10:02:51.268514 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 02 10:02:51.268514 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 02 10:02:51.268514 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application return self.get_or_create( Aug 02 10:02:51.270996 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 02 10:02:51.270996 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application with Lock( Aug 02 10:02:51.270996 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 02 10:02:51.270996 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application return self._enter() Aug 02 10:02:51.270996 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 02 10:02:51.270996 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 02 10:02:51.270996 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 02 10:02:51.270996 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application return self.creator() Aug 02 10:02:51.270996 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 02 10:02:51.270996 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application created_value = creator( Aug 02 10:02:51.270996 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Aug 02 10:02:51.270996 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 02 10:02:51.270996 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: network. Aug 02 10:02:51.270996 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application Aug 02 10:02:51.273038 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 384/775] 10.222.0.82 () {56 vars in 1111 bytes} [Tue Aug 2 10:02:50 2022] GET /identity/v3/services/network => generated 88 bytes in 275 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 02 10:02:51.273496 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0cd381f-3be9-49c4-a216-3d49afc03eb2 None admin] RBAC: Authorization granted {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:02:51.275107 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55b125f5-3de4-44e3-b093-f41e89b799e8 None None] REQUEST_METHOD: `GET` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:02:51.275457 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55b125f5-3de4-44e3-b093-f41e89b799e8 None None] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:02:51.275734 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55b125f5-3de4-44e3-b093-f41e89b799e8 None None] PATH_INFO: `/` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:02:51.277617 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 385/776] 10.222.0.82 () {54 vars in 926 bytes} [Tue Aug 2 10:02:51 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 02 10:02:51.278466 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55b125f5-3de4-44e3-b093-f41e89b799e8 None None] Authenticating user token {{(pid=80378) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:02:51.286126 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: [pid: 80377|app: 0|req: 392/777] 10.222.0.82 () {56 vars in 1374 bytes} [Tue Aug 2 10:02:50 2022] GET /identity/v3/role_assignments?user.id=bd8dd72086db48b8aaac2ed0058872df&scope.project.id=f1b38ebb804c41c69772c998aae72c41&role.id=02003a0be60f47f2a8dfea17053af52c => generated 259 bytes in 298 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 02 10:02:51.288510 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47a415b1-7201-4289-bc44-9e6024a001df None None] REQUEST_METHOD: `POST` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:02:51.289222 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47a415b1-7201-4289-bc44-9e6024a001df None None] SCRIPT_NAME: `/identity` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:02:51.289723 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47a415b1-7201-4289-bc44-9e6024a001df None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:02:51.313522 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.auth.core [None req-47a415b1-7201-4289-bc44-9e6024a001df None None] MFA Rules not processed for user `891005fa5ff74041b75747d16cc66798`. Rule list: `[]` (Enabled: `True`). {{(pid=80377) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 02 10:02:51.315228 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.fernet_utils [None req-47a415b1-7201-4289-bc44-9e6024a001df None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80377) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 10:02:51.398503 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG dbcounter [-] [80377] Writing DB stats keystone:SELECT=100 {{(pid=80377) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 10:02:51.407524 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: [pid: 80377|app: 0|req: 393/778] 10.222.0.82 () {58 vars in 1023 bytes} [Tue Aug 2 10:02:51 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 120 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 02 10:02:51.438242 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47a415b1-7201-4289-bc44-9e6024a001df None None] Authenticating user token {{(pid=80377) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:02:51.557980 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55b125f5-3de4-44e3-b093-f41e89b799e8 None None] Validating token access rules against request {{(pid=80378) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:02:51.639971 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG dbcounter [-] [80378] Writing DB stats keystone:SELECT=100 {{(pid=80378) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 10:02:51.697244 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47a415b1-7201-4289-bc44-9e6024a001df None None] Validating token access rules against request {{(pid=80377) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:02:51.723358 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3630e423-c2ad-46b0-be11-d6ccf22d6fa3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '891005fa5ff74041b75747d16cc66798', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80378) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:02:51.723998 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3630e423-c2ad-46b0-be11-d6ccf22d6fa3 None admin] REQUEST_METHOD: `GET` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:02:51.724283 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3630e423-c2ad-46b0-be11-d6ccf22d6fa3 None admin] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:02:51.724602 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3630e423-c2ad-46b0-be11-d6ccf22d6fa3 None admin] PATH_INFO: `/v3/services` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:02:51.725565 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3630e423-c2ad-46b0-be11-d6ccf22d6fa3 None admin] RBAC: Adding query filter params (name=network) {{(pid=80378) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 02 10:02:51.725565 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3630e423-c2ad-46b0-be11-d6ccf22d6fa3 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:02:51.726967 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3630e423-c2ad-46b0-be11-d6ccf22d6fa3 None admin] RBAC: Authorization granted {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:02:51.744436 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 386/779] 10.222.0.82 () {56 vars in 1112 bytes} [Tue Aug 2 10:02:51 2022] GET /identity/v3/services?name=network => generated 124 bytes in 466 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 02 10:02:51.753338 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3630e423-c2ad-46b0-be11-d6ccf22d6fa3 None admin] Authenticating user token {{(pid=80378) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:02:51.783252 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00bfacbd-63c8-4e0e-8476-92c10ff70c2b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '891005fa5ff74041b75747d16cc66798', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80377) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:02:51.785265 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00bfacbd-63c8-4e0e-8476-92c10ff70c2b None admin] REQUEST_METHOD: `GET` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:02:51.786114 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00bfacbd-63c8-4e0e-8476-92c10ff70c2b None admin] SCRIPT_NAME: `/identity` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:02:51.786888 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00bfacbd-63c8-4e0e-8476-92c10ff70c2b None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:02:51.812993 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00bfacbd-63c8-4e0e-8476-92c10ff70c2b None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 02 10:02:51.826344 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application [None req-00bfacbd-63c8-4e0e-8476-92c10ff70c2b None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 02 10:02:51.826344 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 02 10:02:51.826344 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 02 10:02:51.826344 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 02 10:02:51.826344 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 02 10:02:51.826344 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 02 10:02:51.826344 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 02 10:02:51.826344 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 02 10:02:51.826344 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 02 10:02:51.826344 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 02 10:02:51.826344 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 02 10:02:51.826344 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 02 10:02:51.826344 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 02 10:02:51.826344 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 02 10:02:51.826344 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 02 10:02:51.826344 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 02 10:02:51.828176 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 02 10:02:51.828176 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 02 10:02:51.828176 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 02 10:02:51.828176 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 02 10:02:51.828176 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 02 10:02:51.828176 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 02 10:02:51.828176 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 02 10:02:51.828176 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 02 10:02:51.828176 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 02 10:02:51.828176 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application return self.get_or_create( Aug 02 10:02:51.828176 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 02 10:02:51.828176 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application with Lock( Aug 02 10:02:51.828176 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 02 10:02:51.828176 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application return self._enter() Aug 02 10:02:51.828176 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 02 10:02:51.828176 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 02 10:02:51.831912 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 02 10:02:51.831912 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application return self.creator() Aug 02 10:02:51.831912 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 02 10:02:51.831912 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application created_value = creator( Aug 02 10:02:51.831912 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 02 10:02:51.831912 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 02 10:02:51.831912 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 02 10:02:51.831912 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 02 10:02:51.831912 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 02 10:02:51.831912 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 02 10:02:51.831912 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 02 10:02:51.831912 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application Aug 02 10:02:51.837652 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: [pid: 80377|app: 0|req: 394/780] 10.222.0.82 () {56 vars in 1096 bytes} [Tue Aug 2 10:02:51 2022] GET /identity/v3/roles/admin => generated 83 bytes in 400 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 02 10:02:51.848731 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00bfacbd-63c8-4e0e-8476-92c10ff70c2b None admin] Authenticating user token {{(pid=80377) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:02:52.004172 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG dbcounter [-] [80377] Writing DB stats keystone:SELECT=100 {{(pid=80377) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 10:02:52.081894 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3630e423-c2ad-46b0-be11-d6ccf22d6fa3 None admin] Validating token access rules against request {{(pid=80378) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:02:52.099284 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00bfacbd-63c8-4e0e-8476-92c10ff70c2b None admin] Validating token access rules against request {{(pid=80377) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:02:52.155421 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82936d65-67ae-470f-974c-83e013d9c415 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '891005fa5ff74041b75747d16cc66798', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80378) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:02:52.155685 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c617d8c-c5df-423d-b328-13686a217bea None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '891005fa5ff74041b75747d16cc66798', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80377) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:02:52.156450 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c617d8c-c5df-423d-b328-13686a217bea None admin] REQUEST_METHOD: `GET` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:02:52.156589 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82936d65-67ae-470f-974c-83e013d9c415 None admin] REQUEST_METHOD: `GET` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:02:52.156766 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c617d8c-c5df-423d-b328-13686a217bea None admin] SCRIPT_NAME: `/identity` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:02:52.156952 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82936d65-67ae-470f-974c-83e013d9c415 None admin] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:02:52.157085 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c617d8c-c5df-423d-b328-13686a217bea None admin] PATH_INFO: `/v3/roles` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:02:52.157440 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82936d65-67ae-470f-974c-83e013d9c415 None admin] PATH_INFO: `/v3/services` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:02:52.158866 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c617d8c-c5df-423d-b328-13686a217bea None admin] RBAC: Adding query filter params (name=admin) {{(pid=80377) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 02 10:02:52.159399 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c617d8c-c5df-423d-b328-13686a217bea None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:02:52.159607 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82936d65-67ae-470f-974c-83e013d9c415 None admin] RBAC: Adding query filter params (type=network) {{(pid=80378) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 02 10:02:52.160043 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82936d65-67ae-470f-974c-83e013d9c415 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:02:52.161492 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c617d8c-c5df-423d-b328-13686a217bea None admin] RBAC: Authorization granted {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:02:52.169079 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82936d65-67ae-470f-974c-83e013d9c415 None admin] RBAC: Authorization granted {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:02:52.175364 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 387/781] 10.222.0.82 () {56 vars in 1112 bytes} [Tue Aug 2 10:02:51 2022] GET /identity/v3/services?type=network => generated 350 bytes in 423 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 02 10:02:52.177966 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: [pid: 80377|app: 0|req: 395/782] 10.222.0.82 () {56 vars in 1099 bytes} [Tue Aug 2 10:02:51 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 331 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 02 10:02:52.184258 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82936d65-67ae-470f-974c-83e013d9c415 None admin] Authenticating user token {{(pid=80378) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:02:52.184487 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c617d8c-c5df-423d-b328-13686a217bea None admin] Authenticating user token {{(pid=80377) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:02:52.339408 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG dbcounter [-] [80378] Writing DB stats keystone:SELECT=100 {{(pid=80378) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 10:02:52.358936 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c617d8c-c5df-423d-b328-13686a217bea None admin] Validating token access rules against request {{(pid=80377) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:02:52.386232 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG dbcounter [-] [80377] Writing DB stats keystone:SELECT=100 {{(pid=80377) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 10:02:52.413567 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f83f8619-dae2-4290-bf17-b31b9e34eba2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '891005fa5ff74041b75747d16cc66798', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80377) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:02:52.414477 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f83f8619-dae2-4290-bf17-b31b9e34eba2 None admin] REQUEST_METHOD: `GET` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:02:52.414800 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f83f8619-dae2-4290-bf17-b31b9e34eba2 None admin] SCRIPT_NAME: `/identity` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:02:52.415118 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f83f8619-dae2-4290-bf17-b31b9e34eba2 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:02:52.429310 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f83f8619-dae2-4290-bf17-b31b9e34eba2 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:02:52.431084 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f83f8619-dae2-4290-bf17-b31b9e34eba2 None admin] RBAC: Authorization granted {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:02:52.451342 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application [None req-f83f8619-dae2-4290-bf17-b31b9e34eba2 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 02 10:02:52.451342 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 02 10:02:52.451342 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 02 10:02:52.451342 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 02 10:02:52.451342 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 02 10:02:52.451342 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 02 10:02:52.451342 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 02 10:02:52.451342 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 02 10:02:52.451342 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 02 10:02:52.451342 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 02 10:02:52.451342 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 02 10:02:52.451342 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 02 10:02:52.451342 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 02 10:02:52.451342 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 02 10:02:52.451342 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 02 10:02:52.452566 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 02 10:02:52.452566 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 02 10:02:52.452566 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 02 10:02:52.452566 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 02 10:02:52.452566 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 02 10:02:52.452566 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 02 10:02:52.452566 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application return self.get_or_create( Aug 02 10:02:52.452566 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 02 10:02:52.452566 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application with Lock( Aug 02 10:02:52.452566 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 02 10:02:52.452566 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application return self._enter() Aug 02 10:02:52.452566 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 02 10:02:52.452566 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 02 10:02:52.452566 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 02 10:02:52.452566 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application return self.creator() Aug 02 10:02:52.452566 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 02 10:02:52.452566 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application created_value = creator( Aug 02 10:02:52.453733 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 02 10:02:52.453733 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 02 10:02:52.453733 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 02 10:02:52.453733 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application Aug 02 10:02:52.457957 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: [pid: 80377|app: 0|req: 396/783] 10.222.0.82 () {56 vars in 1108 bytes} [Tue Aug 2 10:02:52 2022] GET /identity/v3/domains/Default => generated 87 bytes in 269 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 02 10:02:52.462799 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f83f8619-dae2-4290-bf17-b31b9e34eba2 None admin] Authenticating user token {{(pid=80377) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:02:52.505068 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82936d65-67ae-470f-974c-83e013d9c415 None admin] Validating token access rules against request {{(pid=80378) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:02:52.621031 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5790959-42e8-491d-a59d-31060e0d1df3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '891005fa5ff74041b75747d16cc66798', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80378) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:02:52.625562 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5790959-42e8-491d-a59d-31060e0d1df3 None admin] REQUEST_METHOD: `GET` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:02:52.626411 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5790959-42e8-491d-a59d-31060e0d1df3 None admin] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:02:52.627299 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5790959-42e8-491d-a59d-31060e0d1df3 None admin] PATH_INFO: `/v3/endpoints` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:02:52.628948 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5790959-42e8-491d-a59d-31060e0d1df3 None admin] RBAC: Adding query filter params (interface=public, service_id=73143ff1df46438193298ca5009bcb8f, region_id=RegionOne) {{(pid=80378) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 02 10:02:52.629765 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5790959-42e8-491d-a59d-31060e0d1df3 None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:02:52.632967 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5790959-42e8-491d-a59d-31060e0d1df3 None admin] RBAC: Authorization granted {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:02:52.656841 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 388/784] 10.222.0.82 () {56 vars in 1251 bytes} [Tue Aug 2 10:02:52 2022] GET /identity/v3/endpoints?service_id=73143ff1df46438193298ca5009bcb8f&interface=public®ion_id=RegionOne => generated 194 bytes in 473 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 02 10:02:52.664406 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5790959-42e8-491d-a59d-31060e0d1df3 None admin] Authenticating user token {{(pid=80378) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:02:52.675826 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f83f8619-dae2-4290-bf17-b31b9e34eba2 None admin] Validating token access rules against request {{(pid=80377) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:02:52.748194 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6871e137-e06f-4578-9801-639f58a2832d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '891005fa5ff74041b75747d16cc66798', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80377) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:02:52.749317 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6871e137-e06f-4578-9801-639f58a2832d None admin] REQUEST_METHOD: `GET` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:02:52.749840 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6871e137-e06f-4578-9801-639f58a2832d None admin] SCRIPT_NAME: `/identity` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:02:52.750345 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6871e137-e06f-4578-9801-639f58a2832d None admin] PATH_INFO: `/v3/domains` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:02:52.752395 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6871e137-e06f-4578-9801-639f58a2832d None admin] RBAC: Adding query filter params (name=Default) {{(pid=80377) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 02 10:02:52.753187 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6871e137-e06f-4578-9801-639f58a2832d None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:02:52.755828 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6871e137-e06f-4578-9801-639f58a2832d None admin] RBAC: Authorization granted {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:02:52.790261 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: [pid: 80377|app: 0|req: 397/785] 10.222.0.82 () {56 vars in 1109 bytes} [Tue Aug 2 10:02:52 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 329 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 02 10:02:52.800621 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6871e137-e06f-4578-9801-639f58a2832d None admin] Authenticating user token {{(pid=80377) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:02:52.916361 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5790959-42e8-491d-a59d-31060e0d1df3 None admin] Validating token access rules against request {{(pid=80378) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:02:52.920325 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG dbcounter [-] [80377] Writing DB stats keystone:SELECT=100 {{(pid=80377) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 10:02:52.957300 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG dbcounter [-] [80378] Writing DB stats keystone:SELECT=100 {{(pid=80378) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 10:02:52.972557 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2206c384-26c5-45b3-83b5-450b19bb5162 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '891005fa5ff74041b75747d16cc66798', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80378) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:02:52.973647 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2206c384-26c5-45b3-83b5-450b19bb5162 None admin] REQUEST_METHOD: `GET` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:02:52.974189 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2206c384-26c5-45b3-83b5-450b19bb5162 None admin] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:02:52.974709 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2206c384-26c5-45b3-83b5-450b19bb5162 None admin] PATH_INFO: `/v3/services` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:02:52.975641 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2206c384-26c5-45b3-83b5-450b19bb5162 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:02:52.977450 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2206c384-26c5-45b3-83b5-450b19bb5162 None admin] RBAC: Authorization granted {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:02:52.986646 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 389/786] 10.222.0.82 () {56 vars in 1087 bytes} [Tue Aug 2 10:02:52 2022] GET /identity/v3/services => generated 772 bytes in 323 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 02 10:02:53.112725 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6871e137-e06f-4578-9801-639f58a2832d None admin] Validating token access rules against request {{(pid=80377) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:02:53.180287 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-556b952a-483a-4bf1-9bc4-1fa6754b69b2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '891005fa5ff74041b75747d16cc66798', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80377) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:02:53.180885 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-556b952a-483a-4bf1-9bc4-1fa6754b69b2 None admin] REQUEST_METHOD: `GET` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:02:53.181171 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-556b952a-483a-4bf1-9bc4-1fa6754b69b2 None admin] SCRIPT_NAME: `/identity` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:02:53.181449 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-556b952a-483a-4bf1-9bc4-1fa6754b69b2 None admin] PATH_INFO: `/v3/users/swift` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:02:53.191134 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-556b952a-483a-4bf1-9bc4-1fa6754b69b2 None admin] RBAC: Authorizing `identity:get_user(user_id=swift)` {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:02:53.192678 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-556b952a-483a-4bf1-9bc4-1fa6754b69b2 None admin] RBAC: Authorization granted {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:02:53.209941 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application [None req-556b952a-483a-4bf1-9bc4-1fa6754b69b2 None admin] Could not find user: swift.: keystone.exception.UserNotFound: Could not find user: swift. Aug 02 10:02:53.209941 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 02 10:02:53.209941 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 02 10:02:53.209941 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 02 10:02:53.209941 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 02 10:02:53.209941 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 02 10:02:53.209941 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 02 10:02:53.209941 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 02 10:02:53.209941 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 02 10:02:53.209941 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 02 10:02:53.209941 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 02 10:02:53.209941 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 02 10:02:53.209941 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 02 10:02:53.209941 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 02 10:02:53.209941 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 02 10:02:53.209941 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 02 10:02:53.211073 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 02 10:02:53.211073 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 02 10:02:53.211073 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 02 10:02:53.211073 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 02 10:02:53.211073 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 02 10:02:53.211073 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 02 10:02:53.211073 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 02 10:02:53.211073 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 02 10:02:53.211073 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 02 10:02:53.211073 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application return self.get_or_create( Aug 02 10:02:53.211073 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 02 10:02:53.211073 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application with Lock( Aug 02 10:02:53.211073 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 02 10:02:53.211073 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application return self._enter() Aug 02 10:02:53.211073 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 02 10:02:53.211073 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 02 10:02:53.211073 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 02 10:02:53.211073 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application return self.creator() Aug 02 10:02:53.212135 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 02 10:02:53.212135 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application created_value = creator( Aug 02 10:02:53.212135 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 02 10:02:53.212135 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 02 10:02:53.212135 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 02 10:02:53.212135 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 02 10:02:53.212135 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 02 10:02:53.212135 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 02 10:02:53.212135 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: swift. Aug 02 10:02:53.212135 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application Aug 02 10:02:53.212135 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: [pid: 80377|app: 0|req: 398/787] 10.222.0.82 () {56 vars in 1096 bytes} [Tue Aug 2 10:02:52 2022] GET /identity/v3/users/swift => generated 83 bytes in 412 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 02 10:02:53.223352 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2206c384-26c5-45b3-83b5-450b19bb5162 None admin] Authenticating user token {{(pid=80378) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:02:53.301026 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e734ea49-c5a2-4dd1-a291-2f27273371b0 None None] REQUEST_METHOD: `GET` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:02:53.301443 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e734ea49-c5a2-4dd1-a291-2f27273371b0 None None] SCRIPT_NAME: `/identity` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:02:53.301912 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e734ea49-c5a2-4dd1-a291-2f27273371b0 None None] PATH_INFO: `/` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:02:53.304409 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: [pid: 80377|app: 0|req: 399/788] 10.222.0.82 () {54 vars in 926 bytes} [Tue Aug 2 10:02:53 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 02 10:02:53.310331 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9a33c79-59d7-4d29-81b8-77f56be2d991 None None] REQUEST_METHOD: `POST` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:02:53.310945 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9a33c79-59d7-4d29-81b8-77f56be2d991 None None] SCRIPT_NAME: `/identity` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:02:53.311670 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9a33c79-59d7-4d29-81b8-77f56be2d991 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:02:53.335554 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.auth.core [None req-d9a33c79-59d7-4d29-81b8-77f56be2d991 None None] MFA Rules not processed for user `891005fa5ff74041b75747d16cc66798`. Rule list: `[]` (Enabled: `True`). {{(pid=80377) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 02 10:02:53.337621 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.fernet_utils [None req-d9a33c79-59d7-4d29-81b8-77f56be2d991 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80377) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 10:02:53.461748 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: [pid: 80377|app: 0|req: 400/789] 10.222.0.82 () {58 vars in 1023 bytes} [Tue Aug 2 10:02:53 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 152 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Aug 02 10:02:53.489825 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2206c384-26c5-45b3-83b5-450b19bb5162 None admin] Validating token access rules against request {{(pid=80378) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:02:53.522781 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9a33c79-59d7-4d29-81b8-77f56be2d991 None None] Authenticating user token {{(pid=80377) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:02:53.558341 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG dbcounter [-] [80377] Writing DB stats keystone:SELECT=100 {{(pid=80377) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 10:02:53.575078 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27700ab6-6eed-4019-91ad-2b4925127267 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '891005fa5ff74041b75747d16cc66798', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80378) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:02:53.575731 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27700ab6-6eed-4019-91ad-2b4925127267 None admin] REQUEST_METHOD: `GET` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:02:53.576012 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27700ab6-6eed-4019-91ad-2b4925127267 None admin] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:02:53.576355 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27700ab6-6eed-4019-91ad-2b4925127267 None admin] PATH_INFO: `/v3/users` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:02:53.577177 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27700ab6-6eed-4019-91ad-2b4925127267 None admin] RBAC: Adding query filter params (domain_id=default, name=swift) {{(pid=80378) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 02 10:02:53.577487 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27700ab6-6eed-4019-91ad-2b4925127267 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:02:53.578980 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27700ab6-6eed-4019-91ad-2b4925127267 None admin] RBAC: Authorization granted {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:02:53.628622 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 390/790] 10.222.0.82 () {56 vars in 1135 bytes} [Tue Aug 2 10:02:53 2022] GET /identity/v3/users?domain_id=default&name=swift => generated 370 bytes in 406 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 02 10:02:53.638354 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27700ab6-6eed-4019-91ad-2b4925127267 None admin] Authenticating user token {{(pid=80378) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:02:53.690276 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG dbcounter [-] [80378] Writing DB stats keystone:SELECT=100 {{(pid=80378) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 10:02:53.776326 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9a33c79-59d7-4d29-81b8-77f56be2d991 None None] Validating token access rules against request {{(pid=80377) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:02:53.786535 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27700ab6-6eed-4019-91ad-2b4925127267 None admin] Validating token access rules against request {{(pid=80378) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:02:53.825040 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-147b7934-3688-46fe-9b11-e659dee0649a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '891005fa5ff74041b75747d16cc66798', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80377) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:02:53.825808 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-147b7934-3688-46fe-9b11-e659dee0649a None admin] REQUEST_METHOD: `GET` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:02:53.826109 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-147b7934-3688-46fe-9b11-e659dee0649a None admin] SCRIPT_NAME: `/identity` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:02:53.826605 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-147b7934-3688-46fe-9b11-e659dee0649a None admin] PATH_INFO: `/v3/services/block-storage` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:02:53.827541 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-147b7934-3688-46fe-9b11-e659dee0649a None admin] RBAC: Authorizing `identity:get_service(service_id=block-storage)` {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:02:53.829252 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-147b7934-3688-46fe-9b11-e659dee0649a None admin] RBAC: Authorization granted {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:02:53.838296 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3a562c4-9226-4980-a93e-7692b0703836 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '891005fa5ff74041b75747d16cc66798', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80378) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:02:53.839300 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3a562c4-9226-4980-a93e-7692b0703836 None admin] REQUEST_METHOD: `GET` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:02:53.839778 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3a562c4-9226-4980-a93e-7692b0703836 None admin] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:02:53.840064 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3a562c4-9226-4980-a93e-7692b0703836 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:02:53.844364 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application [None req-147b7934-3688-46fe-9b11-e659dee0649a None admin] Could not find service: block-storage.: keystone.exception.ServiceNotFound: Could not find service: block-storage. Aug 02 10:02:53.844364 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 02 10:02:53.844364 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Aug 02 10:02:53.844364 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Aug 02 10:02:53.844364 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Aug 02 10:02:53.844364 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Aug 02 10:02:53.844364 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Aug 02 10:02:53.844364 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 02 10:02:53.844364 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: block-storage. Aug 02 10:02:53.844364 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application Aug 02 10:02:53.844364 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 02 10:02:53.844364 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application Aug 02 10:02:53.844364 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 02 10:02:53.844364 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 02 10:02:53.844364 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 02 10:02:53.844364 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 02 10:02:53.846112 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 02 10:02:53.846112 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 02 10:02:53.846112 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 02 10:02:53.846112 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 02 10:02:53.846112 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 02 10:02:53.846112 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 02 10:02:53.846112 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 02 10:02:53.846112 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Aug 02 10:02:53.846112 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application return self._get_service(service_id) Aug 02 10:02:53.846112 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Aug 02 10:02:53.846112 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Aug 02 10:02:53.846112 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 02 10:02:53.846112 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 02 10:02:53.846112 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 02 10:02:53.846112 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 02 10:02:53.846112 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 02 10:02:53.847564 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application return self.get_or_create( Aug 02 10:02:53.847564 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 02 10:02:53.847564 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application with Lock( Aug 02 10:02:53.847564 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 02 10:02:53.847564 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application return self._enter() Aug 02 10:02:53.847564 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 02 10:02:53.847564 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 02 10:02:53.847564 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 02 10:02:53.847564 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application return self.creator() Aug 02 10:02:53.847564 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 02 10:02:53.847564 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application created_value = creator( Aug 02 10:02:53.847564 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Aug 02 10:02:53.847564 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 02 10:02:53.847564 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: block-storage. Aug 02 10:02:53.847564 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application Aug 02 10:02:53.852397 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: [pid: 80377|app: 0|req: 401/791] 10.222.0.82 () {56 vars in 1129 bytes} [Tue Aug 2 10:02:53 2022] GET /identity/v3/services/block-storage => generated 94 bytes in 330 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 02 10:02:53.853906 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8de5052-a8ec-4eac-8d1a-ce2537828d2f None None] REQUEST_METHOD: `GET` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:02:53.854196 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8de5052-a8ec-4eac-8d1a-ce2537828d2f None None] SCRIPT_NAME: `/identity` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:02:53.854475 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8de5052-a8ec-4eac-8d1a-ce2537828d2f None None] PATH_INFO: `/` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:02:53.855929 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3a562c4-9226-4980-a93e-7692b0703836 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:02:53.858487 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3a562c4-9226-4980-a93e-7692b0703836 None admin] RBAC: Authorization granted {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:02:53.860642 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: [pid: 80377|app: 0|req: 402/792] 10.222.0.82 () {54 vars in 926 bytes} [Tue Aug 2 10:02:53 2022] GET /identity => generated 270 bytes in 7 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 02 10:02:53.861678 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8de5052-a8ec-4eac-8d1a-ce2537828d2f None None] Authenticating user token {{(pid=80377) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:02:53.885220 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application [None req-a3a562c4-9226-4980-a93e-7692b0703836 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 02 10:02:53.885220 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 02 10:02:53.885220 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 02 10:02:53.885220 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 02 10:02:53.885220 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 02 10:02:53.885220 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 02 10:02:53.885220 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 02 10:02:53.885220 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 02 10:02:53.885220 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 02 10:02:53.885220 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 02 10:02:53.885220 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 02 10:02:53.885220 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 02 10:02:53.885220 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 02 10:02:53.885220 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 02 10:02:53.885220 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 02 10:02:53.886270 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 02 10:02:53.886270 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 02 10:02:53.886270 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 02 10:02:53.886270 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 02 10:02:53.886270 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 02 10:02:53.886270 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 02 10:02:53.886270 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application return self.get_or_create( Aug 02 10:02:53.886270 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 02 10:02:53.886270 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application with Lock( Aug 02 10:02:53.886270 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 02 10:02:53.886270 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application return self._enter() Aug 02 10:02:53.886270 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 02 10:02:53.886270 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 02 10:02:53.886270 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 02 10:02:53.886270 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application return self.creator() Aug 02 10:02:53.886270 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 02 10:02:53.886270 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application created_value = creator( Aug 02 10:02:53.887155 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 02 10:02:53.887155 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 02 10:02:53.887155 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 02 10:02:53.887155 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application Aug 02 10:02:53.888072 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 391/793] 10.222.0.82 () {56 vars in 1108 bytes} [Tue Aug 2 10:02:53 2022] GET /identity/v3/domains/Default => generated 87 bytes in 250 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 02 10:02:53.897723 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-362883c6-e9f1-4f44-a756-09bba577ebc3 None None] REQUEST_METHOD: `POST` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:02:53.897723 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-362883c6-e9f1-4f44-a756-09bba577ebc3 None None] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:02:53.897723 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-362883c6-e9f1-4f44-a756-09bba577ebc3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:02:53.925539 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.auth.core [None req-362883c6-e9f1-4f44-a756-09bba577ebc3 None None] MFA Rules not processed for user `891005fa5ff74041b75747d16cc66798`. Rule list: `[]` (Enabled: `True`). {{(pid=80378) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 02 10:02:53.927620 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.fernet_utils [None req-362883c6-e9f1-4f44-a756-09bba577ebc3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80378) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 10:02:54.008156 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG dbcounter [-] [80377] Writing DB stats keystone:SELECT=100 {{(pid=80377) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 10:02:54.043065 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8de5052-a8ec-4eac-8d1a-ce2537828d2f None None] Validating token access rules against request {{(pid=80377) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:02:54.062359 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 392/794] 10.222.0.82 () {58 vars in 1023 bytes} [Tue Aug 2 10:02:53 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 174 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 02 10:02:54.064264 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-362883c6-e9f1-4f44-a756-09bba577ebc3 None None] Authenticating user token {{(pid=80378) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:02:54.099178 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG dbcounter [-] [80378] Writing DB stats keystone:SELECT=100 {{(pid=80378) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 10:02:54.100846 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52f14fd4-22f0-4749-9d51-4cb49f065c0e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '891005fa5ff74041b75747d16cc66798', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80377) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:02:54.103431 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52f14fd4-22f0-4749-9d51-4cb49f065c0e None admin] REQUEST_METHOD: `GET` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:02:54.103938 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52f14fd4-22f0-4749-9d51-4cb49f065c0e None admin] SCRIPT_NAME: `/identity` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:02:54.104397 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52f14fd4-22f0-4749-9d51-4cb49f065c0e None admin] PATH_INFO: `/v3/services` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:02:54.105484 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52f14fd4-22f0-4749-9d51-4cb49f065c0e None admin] RBAC: Adding query filter params (name=block-storage) {{(pid=80377) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 02 10:02:54.106164 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52f14fd4-22f0-4749-9d51-4cb49f065c0e None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:02:54.108388 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52f14fd4-22f0-4749-9d51-4cb49f065c0e None admin] RBAC: Authorization granted {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:02:54.121477 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: [pid: 80377|app: 0|req: 403/795] 10.222.0.82 () {56 vars in 1124 bytes} [Tue Aug 2 10:02:53 2022] GET /identity/v3/services?name=block-storage => generated 130 bytes in 260 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 02 10:02:54.123372 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52f14fd4-22f0-4749-9d51-4cb49f065c0e None admin] Authenticating user token {{(pid=80377) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:02:54.274050 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-362883c6-e9f1-4f44-a756-09bba577ebc3 None None] Validating token access rules against request {{(pid=80378) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:02:54.302493 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52f14fd4-22f0-4749-9d51-4cb49f065c0e None admin] Validating token access rules against request {{(pid=80377) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:02:54.338587 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a1ad3bf-0f40-47dc-89fd-584245ced3fb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '891005fa5ff74041b75747d16cc66798', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80378) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:02:54.339957 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a1ad3bf-0f40-47dc-89fd-584245ced3fb None admin] REQUEST_METHOD: `GET` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:02:54.340199 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a1ad3bf-0f40-47dc-89fd-584245ced3fb None admin] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:02:54.340538 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a1ad3bf-0f40-47dc-89fd-584245ced3fb None admin] PATH_INFO: `/v3/domains` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:02:54.341336 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a1ad3bf-0f40-47dc-89fd-584245ced3fb None admin] RBAC: Adding query filter params (name=Default) {{(pid=80378) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 02 10:02:54.341660 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a1ad3bf-0f40-47dc-89fd-584245ced3fb None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:02:54.343159 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a1ad3bf-0f40-47dc-89fd-584245ced3fb None admin] RBAC: Authorization granted {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:02:54.354729 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-804de12f-24b9-4e7c-93ec-00d7886219ac None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '891005fa5ff74041b75747d16cc66798', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80377) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:02:54.356303 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-804de12f-24b9-4e7c-93ec-00d7886219ac None admin] REQUEST_METHOD: `GET` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:02:54.357098 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-804de12f-24b9-4e7c-93ec-00d7886219ac None admin] SCRIPT_NAME: `/identity` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:02:54.358698 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-804de12f-24b9-4e7c-93ec-00d7886219ac None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:02:54.370529 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 393/796] 10.222.0.82 () {56 vars in 1109 bytes} [Tue Aug 2 10:02:54 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 303 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 02 10:02:54.373708 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a1ad3bf-0f40-47dc-89fd-584245ced3fb None admin] Authenticating user token {{(pid=80378) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:02:54.387474 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-804de12f-24b9-4e7c-93ec-00d7886219ac None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 02 10:02:54.399207 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application [None req-804de12f-24b9-4e7c-93ec-00d7886219ac None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 02 10:02:54.399207 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 02 10:02:54.399207 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 02 10:02:54.399207 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 02 10:02:54.399207 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 02 10:02:54.399207 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 02 10:02:54.399207 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 02 10:02:54.399207 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 02 10:02:54.399207 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 02 10:02:54.399207 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 02 10:02:54.399207 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 02 10:02:54.399207 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 02 10:02:54.399207 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 02 10:02:54.399207 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 02 10:02:54.399207 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 02 10:02:54.399207 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 02 10:02:54.400527 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 02 10:02:54.400527 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 02 10:02:54.400527 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 02 10:02:54.400527 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 02 10:02:54.400527 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 02 10:02:54.400527 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 02 10:02:54.400527 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 02 10:02:54.400527 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 02 10:02:54.400527 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 02 10:02:54.400527 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application return self.get_or_create( Aug 02 10:02:54.400527 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 02 10:02:54.400527 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application with Lock( Aug 02 10:02:54.400527 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 02 10:02:54.400527 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application return self._enter() Aug 02 10:02:54.400527 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 02 10:02:54.400527 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 02 10:02:54.402115 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 02 10:02:54.402115 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application return self.creator() Aug 02 10:02:54.402115 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 02 10:02:54.402115 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application created_value = creator( Aug 02 10:02:54.402115 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 02 10:02:54.402115 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 02 10:02:54.402115 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 02 10:02:54.402115 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 02 10:02:54.402115 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 02 10:02:54.402115 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 02 10:02:54.402115 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 02 10:02:54.402115 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application Aug 02 10:02:54.405750 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: [pid: 80377|app: 0|req: 404/797] 10.222.0.82 () {56 vars in 1096 bytes} [Tue Aug 2 10:02:54 2022] GET /identity/v3/roles/admin => generated 83 bytes in 283 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 02 10:02:54.406847 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-804de12f-24b9-4e7c-93ec-00d7886219ac None admin] Authenticating user token {{(pid=80377) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:02:54.412783 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG dbcounter [-] [80377] Writing DB stats keystone:SELECT=100 {{(pid=80377) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 10:02:54.485163 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG dbcounter [-] [80378] Writing DB stats keystone:SELECT=100 {{(pid=80378) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 10:02:54.529602 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a1ad3bf-0f40-47dc-89fd-584245ced3fb None admin] Validating token access rules against request {{(pid=80378) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:02:54.568336 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-804de12f-24b9-4e7c-93ec-00d7886219ac None admin] Validating token access rules against request {{(pid=80377) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:02:54.578652 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e48673c7-a128-4dc3-8b53-925e06af42f9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '891005fa5ff74041b75747d16cc66798', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80378) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:02:54.579736 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e48673c7-a128-4dc3-8b53-925e06af42f9 None admin] REQUEST_METHOD: `GET` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:02:54.580200 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e48673c7-a128-4dc3-8b53-925e06af42f9 None admin] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:02:54.580626 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e48673c7-a128-4dc3-8b53-925e06af42f9 None admin] PATH_INFO: `/v3/services` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:02:54.581845 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e48673c7-a128-4dc3-8b53-925e06af42f9 None admin] RBAC: Adding query filter params (type=block-storage) {{(pid=80378) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 02 10:02:54.582340 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e48673c7-a128-4dc3-8b53-925e06af42f9 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:02:54.584754 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e48673c7-a128-4dc3-8b53-925e06af42f9 None admin] RBAC: Authorization granted {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:02:54.600178 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 394/798] 10.222.0.82 () {56 vars in 1124 bytes} [Tue Aug 2 10:02:54 2022] GET /identity/v3/services?type=block-storage => generated 367 bytes in 221 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 02 10:02:54.600178 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3365676c-41ed-4c86-8583-6c662d73168c None None] REQUEST_METHOD: `GET` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:02:54.600178 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3365676c-41ed-4c86-8583-6c662d73168c None None] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:02:54.600178 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3365676c-41ed-4c86-8583-6c662d73168c None None] PATH_INFO: `/` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:02:54.602046 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 395/799] 10.222.0.82 () {54 vars in 926 bytes} [Tue Aug 2 10:02:54 2022] GET /identity => generated 270 bytes in 8 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 02 10:02:54.603720 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3365676c-41ed-4c86-8583-6c662d73168c None None] Authenticating user token {{(pid=80378) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:02:54.628231 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-468463cb-e293-4a70-97b3-ad0989f60495 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '891005fa5ff74041b75747d16cc66798', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80377) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:02:54.628885 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-468463cb-e293-4a70-97b3-ad0989f60495 None admin] REQUEST_METHOD: `GET` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:02:54.629159 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-468463cb-e293-4a70-97b3-ad0989f60495 None admin] SCRIPT_NAME: `/identity` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:02:54.629443 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-468463cb-e293-4a70-97b3-ad0989f60495 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:02:54.637730 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-468463cb-e293-4a70-97b3-ad0989f60495 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:02:54.639242 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-468463cb-e293-4a70-97b3-ad0989f60495 None admin] RBAC: Authorization granted {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:02:54.656938 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application [None req-468463cb-e293-4a70-97b3-ad0989f60495 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 02 10:02:54.656938 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 02 10:02:54.656938 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 02 10:02:54.656938 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 02 10:02:54.656938 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 02 10:02:54.656938 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 02 10:02:54.656938 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 02 10:02:54.656938 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 02 10:02:54.656938 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 02 10:02:54.656938 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 02 10:02:54.656938 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 02 10:02:54.656938 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 02 10:02:54.656938 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 02 10:02:54.656938 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 02 10:02:54.656938 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 02 10:02:54.658036 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 02 10:02:54.658036 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 02 10:02:54.658036 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 02 10:02:54.658036 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 02 10:02:54.658036 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 02 10:02:54.658036 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 02 10:02:54.658036 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application return self.get_or_create( Aug 02 10:02:54.658036 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 02 10:02:54.658036 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application with Lock( Aug 02 10:02:54.658036 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 02 10:02:54.658036 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application return self._enter() Aug 02 10:02:54.658036 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 02 10:02:54.658036 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 02 10:02:54.658036 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 02 10:02:54.658036 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application return self.creator() Aug 02 10:02:54.658036 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 02 10:02:54.658036 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application created_value = creator( Aug 02 10:02:54.659504 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 02 10:02:54.659504 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 02 10:02:54.659504 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 02 10:02:54.659504 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 02 10:02:54.659504 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 02 10:02:54.659504 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 02 10:02:54.659504 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 02 10:02:54.659504 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application Aug 02 10:02:54.659504 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: [pid: 80377|app: 0|req: 405/800] 10.222.0.82 () {56 vars in 1111 bytes} [Tue Aug 2 10:02:54 2022] GET /identity/v3/projects/service => generated 88 bytes in 252 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 02 10:02:54.660187 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-468463cb-e293-4a70-97b3-ad0989f60495 None admin] Authenticating user token {{(pid=80377) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:02:54.747431 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG dbcounter [-] [80377] Writing DB stats keystone:SELECT=100 {{(pid=80377) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 10:02:54.777231 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3365676c-41ed-4c86-8583-6c662d73168c None None] Validating token access rules against request {{(pid=80378) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:02:54.818028 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-468463cb-e293-4a70-97b3-ad0989f60495 None admin] Validating token access rules against request {{(pid=80377) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:02:54.831106 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b323bed6-3246-486e-9e78-c9ee368a9937 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '891005fa5ff74041b75747d16cc66798', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80378) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:02:54.831811 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b323bed6-3246-486e-9e78-c9ee368a9937 None admin] REQUEST_METHOD: `GET` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:02:54.832194 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b323bed6-3246-486e-9e78-c9ee368a9937 None admin] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:02:54.832542 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b323bed6-3246-486e-9e78-c9ee368a9937 None admin] PATH_INFO: `/v3/roles` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:02:54.833327 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b323bed6-3246-486e-9e78-c9ee368a9937 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80378) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 02 10:02:54.833638 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b323bed6-3246-486e-9e78-c9ee368a9937 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:02:54.835240 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b323bed6-3246-486e-9e78-c9ee368a9937 None admin] RBAC: Authorization granted {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:02:54.838491 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG dbcounter [-] [80378] Writing DB stats keystone:SELECT=100 {{(pid=80378) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 10:02:54.855461 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 396/801] 10.222.0.82 () {56 vars in 1099 bytes} [Tue Aug 2 10:02:54 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 253 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 02 10:02:54.858138 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c50894e-458a-4df6-8c0e-f33cf826b4d9 None None] REQUEST_METHOD: `POST` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:02:54.858469 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c50894e-458a-4df6-8c0e-f33cf826b4d9 None None] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:02:54.858804 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c50894e-458a-4df6-8c0e-f33cf826b4d9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:02:54.883593 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.auth.core [None req-4c50894e-458a-4df6-8c0e-f33cf826b4d9 None None] MFA Rules not processed for user `891005fa5ff74041b75747d16cc66798`. Rule list: `[]` (Enabled: `True`). {{(pid=80378) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 02 10:02:54.885270 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.fernet_utils [None req-4c50894e-458a-4df6-8c0e-f33cf826b4d9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80378) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 10:02:54.888257 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32d3b2c8-c172-4de9-97ca-59dc5539d2b0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '891005fa5ff74041b75747d16cc66798', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80377) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:02:54.889329 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32d3b2c8-c172-4de9-97ca-59dc5539d2b0 None admin] REQUEST_METHOD: `POST` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:02:54.889329 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32d3b2c8-c172-4de9-97ca-59dc5539d2b0 None admin] SCRIPT_NAME: `/identity` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:02:54.889527 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32d3b2c8-c172-4de9-97ca-59dc5539d2b0 None admin] PATH_INFO: `/v3/endpoints` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:02:54.890438 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32d3b2c8-c172-4de9-97ca-59dc5539d2b0 None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:02:54.892104 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32d3b2c8-c172-4de9-97ca-59dc5539d2b0 None admin] RBAC: Authorization granted {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:02:54.931272 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: [pid: 80377|app: 0|req: 406/802] 10.222.0.82 () {60 vars in 1144 bytes} [Tue Aug 2 10:02:54 2022] POST /identity/v3/endpoints => generated 346 bytes in 265 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 02 10:02:54.931272 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32d3b2c8-c172-4de9-97ca-59dc5539d2b0 None admin] Authenticating user token {{(pid=80377) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:02:55.022048 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 397/803] 10.222.0.82 () {58 vars in 1023 bytes} [Tue Aug 2 10:02:54 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 166 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 02 10:02:55.023172 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c50894e-458a-4df6-8c0e-f33cf826b4d9 None None] Authenticating user token {{(pid=80378) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:02:55.128493 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32d3b2c8-c172-4de9-97ca-59dc5539d2b0 None admin] Validating token access rules against request {{(pid=80377) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:02:55.151857 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG dbcounter [-] [80377] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80377) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 10:02:55.170283 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85a337fe-4bd9-4a37-882e-4d66c0814d00 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '891005fa5ff74041b75747d16cc66798', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80377) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:02:55.170915 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85a337fe-4bd9-4a37-882e-4d66c0814d00 None admin] REQUEST_METHOD: `GET` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:02:55.171195 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85a337fe-4bd9-4a37-882e-4d66c0814d00 None admin] SCRIPT_NAME: `/identity` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:02:55.171467 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85a337fe-4bd9-4a37-882e-4d66c0814d00 None admin] PATH_INFO: `/v3/projects` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:02:55.172420 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85a337fe-4bd9-4a37-882e-4d66c0814d00 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80377) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 02 10:02:55.172783 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85a337fe-4bd9-4a37-882e-4d66c0814d00 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:02:55.174920 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85a337fe-4bd9-4a37-882e-4d66c0814d00 None admin] RBAC: Authorization granted {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:02:55.191498 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: [pid: 80377|app: 0|req: 407/804] 10.222.0.82 () {56 vars in 1148 bytes} [Tue Aug 2 10:02:54 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 267 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 02 10:02:55.192169 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85a337fe-4bd9-4a37-882e-4d66c0814d00 None admin] Authenticating user token {{(pid=80377) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:02:55.200595 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c50894e-458a-4df6-8c0e-f33cf826b4d9 None None] Validating token access rules against request {{(pid=80378) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:02:55.232683 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG dbcounter [-] [80378] Writing DB stats keystone:SELECT=100 {{(pid=80378) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 10:02:55.250899 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0577d4dd-c9b1-41f3-b58e-ae23cb94ab81 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '891005fa5ff74041b75747d16cc66798', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80378) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:02:55.252511 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0577d4dd-c9b1-41f3-b58e-ae23cb94ab81 None admin] REQUEST_METHOD: `GET` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:02:55.253098 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0577d4dd-c9b1-41f3-b58e-ae23cb94ab81 None admin] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:02:55.253735 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0577d4dd-c9b1-41f3-b58e-ae23cb94ab81 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:02:55.273947 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0577d4dd-c9b1-41f3-b58e-ae23cb94ab81 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:02:55.276358 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0577d4dd-c9b1-41f3-b58e-ae23cb94ab81 None admin] RBAC: Authorization granted {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:02:55.305174 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application [None req-0577d4dd-c9b1-41f3-b58e-ae23cb94ab81 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 02 10:02:55.305174 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 02 10:02:55.305174 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 02 10:02:55.305174 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 02 10:02:55.305174 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 02 10:02:55.305174 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 02 10:02:55.305174 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 02 10:02:55.305174 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 02 10:02:55.305174 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 02 10:02:55.305174 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 02 10:02:55.305174 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 02 10:02:55.305174 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 02 10:02:55.305174 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 02 10:02:55.305174 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 02 10:02:55.305174 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 02 10:02:55.306201 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 02 10:02:55.306201 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 02 10:02:55.306201 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 02 10:02:55.306201 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 02 10:02:55.306201 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 02 10:02:55.306201 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 02 10:02:55.306201 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application return self.get_or_create( Aug 02 10:02:55.306201 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 02 10:02:55.306201 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application with Lock( Aug 02 10:02:55.306201 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 02 10:02:55.306201 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application return self._enter() Aug 02 10:02:55.306201 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 02 10:02:55.306201 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 02 10:02:55.306201 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 02 10:02:55.306201 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application return self.creator() Aug 02 10:02:55.306201 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 02 10:02:55.306201 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application created_value = creator( Aug 02 10:02:55.307134 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 02 10:02:55.307134 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 02 10:02:55.307134 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 02 10:02:55.307134 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application Aug 02 10:02:55.310346 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 398/805] 10.222.0.82 () {56 vars in 1108 bytes} [Tue Aug 2 10:02:55 2022] GET /identity/v3/domains/Default => generated 87 bytes in 288 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 02 10:02:55.311780 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0577d4dd-c9b1-41f3-b58e-ae23cb94ab81 None admin] Authenticating user token {{(pid=80378) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:02:55.333390 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85a337fe-4bd9-4a37-882e-4d66c0814d00 None admin] Validating token access rules against request {{(pid=80377) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:02:55.396522 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c091936a-2072-4880-bb2f-73c9ad41a872 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '891005fa5ff74041b75747d16cc66798', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80377) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:02:55.398008 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c091936a-2072-4880-bb2f-73c9ad41a872 None admin] REQUEST_METHOD: `GET` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:02:55.398943 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c091936a-2072-4880-bb2f-73c9ad41a872 None admin] SCRIPT_NAME: `/identity` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:02:55.399768 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c091936a-2072-4880-bb2f-73c9ad41a872 None admin] PATH_INFO: `/v3/roles/service` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:02:55.426395 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c091936a-2072-4880-bb2f-73c9ad41a872 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 02 10:02:55.439956 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application [None req-c091936a-2072-4880-bb2f-73c9ad41a872 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Aug 02 10:02:55.439956 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 02 10:02:55.439956 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 02 10:02:55.439956 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 02 10:02:55.439956 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 02 10:02:55.439956 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 02 10:02:55.439956 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 02 10:02:55.439956 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 02 10:02:55.439956 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 02 10:02:55.439956 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 02 10:02:55.439956 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 02 10:02:55.439956 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 02 10:02:55.439956 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 02 10:02:55.439956 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 02 10:02:55.439956 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 02 10:02:55.439956 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 02 10:02:55.440981 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 02 10:02:55.440981 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 02 10:02:55.440981 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 02 10:02:55.440981 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 02 10:02:55.440981 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 02 10:02:55.440981 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 02 10:02:55.440981 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 02 10:02:55.440981 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 02 10:02:55.440981 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 02 10:02:55.440981 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application return self.get_or_create( Aug 02 10:02:55.440981 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 02 10:02:55.440981 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application with Lock( Aug 02 10:02:55.440981 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 02 10:02:55.440981 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application return self._enter() Aug 02 10:02:55.440981 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 02 10:02:55.440981 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 02 10:02:55.442471 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 02 10:02:55.442471 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application return self.creator() Aug 02 10:02:55.442471 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 02 10:02:55.442471 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application created_value = creator( Aug 02 10:02:55.442471 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 02 10:02:55.442471 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 02 10:02:55.442471 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 02 10:02:55.442471 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 02 10:02:55.442471 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 02 10:02:55.442471 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 02 10:02:55.442471 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Aug 02 10:02:55.442471 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application Aug 02 10:02:55.450023 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: [pid: 80377|app: 0|req: 408/806] 10.222.0.82 () {56 vars in 1102 bytes} [Tue Aug 2 10:02:55 2022] GET /identity/v3/roles/service => generated 85 bytes in 252 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 02 10:02:55.450023 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c091936a-2072-4880-bb2f-73c9ad41a872 None admin] Authenticating user token {{(pid=80377) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:02:55.456366 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0577d4dd-c9b1-41f3-b58e-ae23cb94ab81 None admin] Validating token access rules against request {{(pid=80378) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:02:55.524396 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-159685cf-bf12-4a54-9cc5-84c54a79ad90 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '891005fa5ff74041b75747d16cc66798', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80378) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:02:55.526300 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG dbcounter [-] [80377] Writing DB stats keystone:SELECT=100 {{(pid=80377) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 10:02:55.529408 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-159685cf-bf12-4a54-9cc5-84c54a79ad90 None admin] REQUEST_METHOD: `PUT` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:02:55.530682 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-159685cf-bf12-4a54-9cc5-84c54a79ad90 None admin] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:02:55.531188 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-159685cf-bf12-4a54-9cc5-84c54a79ad90 None admin] PATH_INFO: `/v3/projects/f1b38ebb804c41c69772c998aae72c41/users/bef1b2e714cb45c4a51eb6d3e15aad6d/roles/237966442f05411191105bf336bc14f3` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:02:55.534766 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-159685cf-bf12-4a54-9cc5-84c54a79ad90 None admin] RBAC: Authorizing `identity:create_grant(project_id=f1b38ebb804c41c69772c998aae72c41, user_id=bef1b2e714cb45c4a51eb6d3e15aad6d, role_id=237966442f05411191105bf336bc14f3)` {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:02:55.536953 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-159685cf-bf12-4a54-9cc5-84c54a79ad90 None admin] RBAC: Authorization granted {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:02:55.565828 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 399/807] 10.222.0.82 () {58 vars in 1439 bytes} [Tue Aug 2 10:02:55 2022] PUT /identity/v3/projects/f1b38ebb804c41c69772c998aae72c41/users/bef1b2e714cb45c4a51eb6d3e15aad6d/roles/237966442f05411191105bf336bc14f3 => generated 0 bytes in 248 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 02 10:02:55.565828 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-159685cf-bf12-4a54-9cc5-84c54a79ad90 None admin] Authenticating user token {{(pid=80378) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:02:55.655276 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG dbcounter [-] [80378] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80378) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 10:02:55.675428 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c091936a-2072-4880-bb2f-73c9ad41a872 None admin] Validating token access rules against request {{(pid=80377) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:02:55.754853 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c8450a8-e12f-4778-9d3c-4111920e3251 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '891005fa5ff74041b75747d16cc66798', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80377) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:02:55.756178 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c8450a8-e12f-4778-9d3c-4111920e3251 None admin] REQUEST_METHOD: `GET` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:02:55.756724 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c8450a8-e12f-4778-9d3c-4111920e3251 None admin] SCRIPT_NAME: `/identity` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:02:55.757312 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c8450a8-e12f-4778-9d3c-4111920e3251 None admin] PATH_INFO: `/v3/domains` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:02:55.758756 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c8450a8-e12f-4778-9d3c-4111920e3251 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80377) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 02 10:02:55.759405 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c8450a8-e12f-4778-9d3c-4111920e3251 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:02:55.761447 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c8450a8-e12f-4778-9d3c-4111920e3251 None admin] RBAC: Authorization granted {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:02:55.791274 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-159685cf-bf12-4a54-9cc5-84c54a79ad90 None admin] Validating token access rules against request {{(pid=80378) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:02:55.798920 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: [pid: 80377|app: 0|req: 409/808] 10.222.0.82 () {56 vars in 1109 bytes} [Tue Aug 2 10:02:55 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 354 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 02 10:02:55.810486 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c8450a8-e12f-4778-9d3c-4111920e3251 None admin] Authenticating user token {{(pid=80377) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:02:55.868332 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e5cc6bd-729e-4910-98d9-00b7c71424d7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '891005fa5ff74041b75747d16cc66798', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80378) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:02:55.869316 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e5cc6bd-729e-4910-98d9-00b7c71424d7 None admin] REQUEST_METHOD: `GET` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:02:55.869800 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e5cc6bd-729e-4910-98d9-00b7c71424d7 None admin] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:02:55.870266 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e5cc6bd-729e-4910-98d9-00b7c71424d7 None admin] PATH_INFO: `/v3/roles` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:02:55.871542 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e5cc6bd-729e-4910-98d9-00b7c71424d7 None admin] RBAC: Adding query filter params (name=service) {{(pid=80378) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 02 10:02:55.872080 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e5cc6bd-729e-4910-98d9-00b7c71424d7 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:02:55.884926 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e5cc6bd-729e-4910-98d9-00b7c71424d7 None admin] RBAC: Authorization granted {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:02:55.902788 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 400/809] 10.222.0.82 () {56 vars in 1103 bytes} [Tue Aug 2 10:02:55 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 343 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 02 10:02:55.910602 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e5cc6bd-729e-4910-98d9-00b7c71424d7 None admin] Authenticating user token {{(pid=80378) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:02:56.018490 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG dbcounter [-] [80377] Writing DB stats keystone:SELECT=100 {{(pid=80377) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 10:02:56.024068 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c8450a8-e12f-4778-9d3c-4111920e3251 None admin] Validating token access rules against request {{(pid=80377) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:02:56.071659 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87ab494b-1368-43ce-9894-3e076fd9bf11 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '891005fa5ff74041b75747d16cc66798', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80377) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:02:56.072615 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87ab494b-1368-43ce-9894-3e076fd9bf11 None admin] REQUEST_METHOD: `GET` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:02:56.072935 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87ab494b-1368-43ce-9894-3e076fd9bf11 None admin] SCRIPT_NAME: `/identity` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:02:56.073398 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87ab494b-1368-43ce-9894-3e076fd9bf11 None admin] PATH_INFO: `/v3/users/nova` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:02:56.082564 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG dbcounter [-] [80378] Writing DB stats keystone:SELECT=100 {{(pid=80378) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 10:02:56.088931 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e5cc6bd-729e-4910-98d9-00b7c71424d7 None admin] Validating token access rules against request {{(pid=80378) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:02:56.091204 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87ab494b-1368-43ce-9894-3e076fd9bf11 None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:02:56.093776 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87ab494b-1368-43ce-9894-3e076fd9bf11 None admin] RBAC: Authorization granted {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:02:56.109179 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application [None req-87ab494b-1368-43ce-9894-3e076fd9bf11 None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Aug 02 10:02:56.109179 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 02 10:02:56.109179 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 02 10:02:56.109179 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 02 10:02:56.109179 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 02 10:02:56.109179 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 02 10:02:56.109179 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 02 10:02:56.109179 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 02 10:02:56.109179 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 02 10:02:56.109179 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 02 10:02:56.109179 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 02 10:02:56.109179 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 02 10:02:56.109179 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 02 10:02:56.109179 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 02 10:02:56.109179 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 02 10:02:56.109179 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 02 10:02:56.110464 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 02 10:02:56.110464 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 02 10:02:56.110464 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 02 10:02:56.110464 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 02 10:02:56.110464 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 02 10:02:56.110464 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 02 10:02:56.110464 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 02 10:02:56.110464 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 02 10:02:56.110464 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 02 10:02:56.110464 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application return self.get_or_create( Aug 02 10:02:56.110464 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 02 10:02:56.110464 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application with Lock( Aug 02 10:02:56.110464 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 02 10:02:56.110464 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application return self._enter() Aug 02 10:02:56.110464 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 02 10:02:56.110464 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 02 10:02:56.110464 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 02 10:02:56.110464 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application return self.creator() Aug 02 10:02:56.112360 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 02 10:02:56.112360 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application created_value = creator( Aug 02 10:02:56.112360 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 02 10:02:56.112360 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 02 10:02:56.112360 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 02 10:02:56.112360 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 02 10:02:56.112360 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 02 10:02:56.112360 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 02 10:02:56.112360 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Aug 02 10:02:56.112360 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application Aug 02 10:02:56.116465 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: [pid: 80377|app: 0|req: 410/810] 10.222.0.82 () {56 vars in 1093 bytes} [Tue Aug 2 10:02:55 2022] GET /identity/v3/users/nova => generated 82 bytes in 307 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 02 10:02:56.118135 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90fd2279-4a12-48d2-96d1-4d7120e037cf None None] REQUEST_METHOD: `GET` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:02:56.118431 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90fd2279-4a12-48d2-96d1-4d7120e037cf None None] SCRIPT_NAME: `/identity` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:02:56.118829 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90fd2279-4a12-48d2-96d1-4d7120e037cf None None] PATH_INFO: `/` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:02:56.125458 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: [pid: 80377|app: 0|req: 411/811] 10.222.0.82 () {54 vars in 926 bytes} [Tue Aug 2 10:02:56 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 02 10:02:56.125458 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90fd2279-4a12-48d2-96d1-4d7120e037cf None None] Authenticating user token {{(pid=80377) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:02:56.156391 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b14e851-b082-47b9-be7a-80e181e8a39a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '891005fa5ff74041b75747d16cc66798', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80378) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:02:56.157559 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b14e851-b082-47b9-be7a-80e181e8a39a None admin] REQUEST_METHOD: `GET` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:02:56.158018 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b14e851-b082-47b9-be7a-80e181e8a39a None admin] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:02:56.158431 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b14e851-b082-47b9-be7a-80e181e8a39a None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:02:56.182008 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b14e851-b082-47b9-be7a-80e181e8a39a None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:02:56.182008 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b14e851-b082-47b9-be7a-80e181e8a39a None admin] RBAC: Authorization granted {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:02:56.207761 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application [None req-0b14e851-b082-47b9-be7a-80e181e8a39a None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 02 10:02:56.207761 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 02 10:02:56.207761 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 02 10:02:56.207761 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 02 10:02:56.207761 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 02 10:02:56.207761 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 02 10:02:56.207761 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 02 10:02:56.207761 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 02 10:02:56.207761 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 02 10:02:56.207761 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 02 10:02:56.207761 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 02 10:02:56.207761 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 02 10:02:56.207761 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 02 10:02:56.207761 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 02 10:02:56.207761 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 02 10:02:56.209880 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 02 10:02:56.209880 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 02 10:02:56.209880 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 02 10:02:56.209880 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 02 10:02:56.209880 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 02 10:02:56.209880 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 02 10:02:56.209880 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application return self.get_or_create( Aug 02 10:02:56.209880 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 02 10:02:56.209880 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application with Lock( Aug 02 10:02:56.209880 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 02 10:02:56.209880 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application return self._enter() Aug 02 10:02:56.209880 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 02 10:02:56.209880 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 02 10:02:56.209880 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 02 10:02:56.209880 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application return self.creator() Aug 02 10:02:56.209880 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 02 10:02:56.209880 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application created_value = creator( Aug 02 10:02:56.212021 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 02 10:02:56.212021 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 02 10:02:56.212021 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 02 10:02:56.212021 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application Aug 02 10:02:56.216493 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 401/812] 10.222.0.82 () {56 vars in 1108 bytes} [Tue Aug 2 10:02:55 2022] GET /identity/v3/domains/Default => generated 87 bytes in 307 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 02 10:02:56.218346 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb7f4562-3ea4-4831-a387-65f85a8003e5 None None] REQUEST_METHOD: `POST` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:02:56.218860 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb7f4562-3ea4-4831-a387-65f85a8003e5 None None] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:02:56.219744 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb7f4562-3ea4-4831-a387-65f85a8003e5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:02:56.245860 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.auth.core [None req-cb7f4562-3ea4-4831-a387-65f85a8003e5 None None] MFA Rules not processed for user `891005fa5ff74041b75747d16cc66798`. Rule list: `[]` (Enabled: `True`). {{(pid=80378) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 02 10:02:56.248488 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.fernet_utils [None req-cb7f4562-3ea4-4831-a387-65f85a8003e5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80378) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 10:02:56.315484 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90fd2279-4a12-48d2-96d1-4d7120e037cf None None] Validating token access rules against request {{(pid=80377) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:02:56.374439 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e954626-fb31-45b7-b20b-b0e4d7324796 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '891005fa5ff74041b75747d16cc66798', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80377) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:02:56.376248 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e954626-fb31-45b7-b20b-b0e4d7324796 None admin] REQUEST_METHOD: `GET` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:02:56.377418 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e954626-fb31-45b7-b20b-b0e4d7324796 None admin] SCRIPT_NAME: `/identity` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:02:56.377921 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e954626-fb31-45b7-b20b-b0e4d7324796 None admin] PATH_INFO: `/v3/users` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:02:56.379013 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e954626-fb31-45b7-b20b-b0e4d7324796 None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=80377) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 02 10:02:56.379498 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e954626-fb31-45b7-b20b-b0e4d7324796 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:02:56.381353 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e954626-fb31-45b7-b20b-b0e4d7324796 None admin] RBAC: Authorization granted {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:02:56.407673 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 402/813] 10.222.0.82 () {58 vars in 1023 bytes} [Tue Aug 2 10:02:56 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 189 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 02 10:02:56.409359 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb7f4562-3ea4-4831-a387-65f85a8003e5 None None] Authenticating user token {{(pid=80378) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:02:56.413999 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: [pid: 80377|app: 0|req: 412/814] 10.222.0.82 () {56 vars in 1133 bytes} [Tue Aug 2 10:02:56 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 292 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 02 10:02:56.425857 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e954626-fb31-45b7-b20b-b0e4d7324796 None admin] Authenticating user token {{(pid=80377) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:02:56.456768 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG dbcounter [-] [80377] Writing DB stats keystone:SELECT=100 {{(pid=80377) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 10:02:56.561507 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG dbcounter [-] [80378] Writing DB stats keystone:SELECT=100 {{(pid=80378) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 10:02:56.591067 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e954626-fb31-45b7-b20b-b0e4d7324796 None admin] Validating token access rules against request {{(pid=80377) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:02:56.598730 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb7f4562-3ea4-4831-a387-65f85a8003e5 None None] Validating token access rules against request {{(pid=80378) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:02:56.636318 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22847d13-d868-451a-b6bc-97d3fb1590c0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '891005fa5ff74041b75747d16cc66798', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80377) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:02:56.636964 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22847d13-d868-451a-b6bc-97d3fb1590c0 None admin] REQUEST_METHOD: `GET` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:02:56.637244 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22847d13-d868-451a-b6bc-97d3fb1590c0 None admin] SCRIPT_NAME: `/identity` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:02:56.637875 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22847d13-d868-451a-b6bc-97d3fb1590c0 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:02:56.655598 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54d75980-40ac-4d97-b4b6-e3b92c4c436c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '891005fa5ff74041b75747d16cc66798', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80378) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:02:56.656312 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54d75980-40ac-4d97-b4b6-e3b92c4c436c None admin] REQUEST_METHOD: `GET` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:02:56.656597 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54d75980-40ac-4d97-b4b6-e3b92c4c436c None admin] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:02:56.656928 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54d75980-40ac-4d97-b4b6-e3b92c4c436c None admin] PATH_INFO: `/v3/domains` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:02:56.657768 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54d75980-40ac-4d97-b4b6-e3b92c4c436c None admin] RBAC: Adding query filter params (name=Default) {{(pid=80378) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 02 10:02:56.658120 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54d75980-40ac-4d97-b4b6-e3b92c4c436c None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:02:56.660536 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54d75980-40ac-4d97-b4b6-e3b92c4c436c None admin] RBAC: Authorization granted {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:02:56.678604 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 403/815] 10.222.0.82 () {56 vars in 1109 bytes} [Tue Aug 2 10:02:56 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 270 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 02 10:02:56.679577 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54d75980-40ac-4d97-b4b6-e3b92c4c436c None admin] Authenticating user token {{(pid=80378) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:02:56.758333 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22847d13-d868-451a-b6bc-97d3fb1590c0 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:02:56.760890 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22847d13-d868-451a-b6bc-97d3fb1590c0 None admin] RBAC: Authorization granted {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:02:56.793351 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG dbcounter [-] [80377] Writing DB stats keystone:SELECT=100 {{(pid=80377) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 10:02:56.891938 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54d75980-40ac-4d97-b4b6-e3b92c4c436c None admin] Validating token access rules against request {{(pid=80378) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:02:56.955128 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: [pid: 80377|app: 0|req: 413/816] 10.222.0.82 () {58 vars in 1282 bytes} [Tue Aug 2 10:02:56 2022] GET /identity/v3/auth/tokens => generated 1002 bytes in 531 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Aug 02 10:02:56.955128 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22847d13-d868-451a-b6bc-97d3fb1590c0 None admin] Authenticating user token {{(pid=80377) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:02:56.960830 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7bd8cbe-4b29-4825-b1cd-8321ebbf1b84 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '891005fa5ff74041b75747d16cc66798', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80378) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:02:56.961498 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7bd8cbe-4b29-4825-b1cd-8321ebbf1b84 None admin] REQUEST_METHOD: `GET` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:02:56.961957 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7bd8cbe-4b29-4825-b1cd-8321ebbf1b84 None admin] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:02:56.962370 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7bd8cbe-4b29-4825-b1cd-8321ebbf1b84 None admin] PATH_INFO: `/v3/services/network` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:02:56.963302 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7bd8cbe-4b29-4825-b1cd-8321ebbf1b84 None admin] RBAC: Authorizing `identity:get_service(service_id=network)` {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:02:56.964946 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7bd8cbe-4b29-4825-b1cd-8321ebbf1b84 None admin] RBAC: Authorization granted {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:02:56.979934 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application [None req-a7bd8cbe-4b29-4825-b1cd-8321ebbf1b84 None admin] Could not find service: network.: keystone.exception.ServiceNotFound: Could not find service: network. Aug 02 10:02:56.979934 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 02 10:02:56.979934 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Aug 02 10:02:56.979934 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Aug 02 10:02:56.979934 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Aug 02 10:02:56.979934 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Aug 02 10:02:56.979934 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Aug 02 10:02:56.979934 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 02 10:02:56.979934 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: network. Aug 02 10:02:56.979934 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application Aug 02 10:02:56.979934 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 02 10:02:56.979934 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application Aug 02 10:02:56.979934 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 02 10:02:56.979934 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 02 10:02:56.979934 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 02 10:02:56.979934 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 02 10:02:56.979934 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 02 10:02:56.981023 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 02 10:02:56.981023 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 02 10:02:56.981023 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 02 10:02:56.981023 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 02 10:02:56.981023 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 02 10:02:56.981023 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 02 10:02:56.981023 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Aug 02 10:02:56.981023 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application return self._get_service(service_id) Aug 02 10:02:56.981023 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Aug 02 10:02:56.981023 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Aug 02 10:02:56.981023 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 02 10:02:56.981023 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 02 10:02:56.981023 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 02 10:02:56.981023 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 02 10:02:56.981023 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 02 10:02:56.981023 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application return self.get_or_create( Aug 02 10:02:56.982041 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 02 10:02:56.982041 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application with Lock( Aug 02 10:02:56.982041 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 02 10:02:56.982041 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application return self._enter() Aug 02 10:02:56.982041 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 02 10:02:56.982041 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 02 10:02:56.982041 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 02 10:02:56.982041 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application return self.creator() Aug 02 10:02:56.982041 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 02 10:02:56.982041 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application created_value = creator( Aug 02 10:02:56.982041 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Aug 02 10:02:56.982041 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 02 10:02:56.982041 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: network. Aug 02 10:02:56.982041 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application Aug 02 10:02:56.986509 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 404/817] 10.222.0.82 () {56 vars in 1111 bytes} [Tue Aug 2 10:02:56 2022] GET /identity/v3/services/network => generated 88 bytes in 302 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 02 10:02:56.988121 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7bd8cbe-4b29-4825-b1cd-8321ebbf1b84 None admin] Authenticating user token {{(pid=80378) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:02:56.998498 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG dbcounter [-] [80378] Writing DB stats keystone:SELECT=100 {{(pid=80378) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 10:02:57.122567 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22847d13-d868-451a-b6bc-97d3fb1590c0 None admin] Validating token access rules against request {{(pid=80377) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:02:57.150668 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7bd8cbe-4b29-4825-b1cd-8321ebbf1b84 None admin] Validating token access rules against request {{(pid=80378) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:02:57.151983 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG dbcounter [-] [80377] Writing DB stats keystone:SELECT=100 {{(pid=80377) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 10:02:57.180237 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f3e2ee3-0b21-40f9-8f5e-8dbe1efe45e0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '891005fa5ff74041b75747d16cc66798', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80377) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:02:57.181698 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f3e2ee3-0b21-40f9-8f5e-8dbe1efe45e0 None admin] REQUEST_METHOD: `GET` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:02:57.182503 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f3e2ee3-0b21-40f9-8f5e-8dbe1efe45e0 None admin] SCRIPT_NAME: `/identity` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:02:57.183226 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f3e2ee3-0b21-40f9-8f5e-8dbe1efe45e0 None admin] PATH_INFO: `/v3/users/glance-swift` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:02:57.195136 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f3e2ee3-0b21-40f9-8f5e-8dbe1efe45e0 None admin] RBAC: Authorizing `identity:get_user(user_id=glance-swift)` {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:02:57.198167 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f3e2ee3-0b21-40f9-8f5e-8dbe1efe45e0 None admin] RBAC: Authorization granted {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:02:57.201664 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10ff62d9-7f61-4e5b-a030-c4f2b096b291 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '891005fa5ff74041b75747d16cc66798', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80378) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:02:57.202546 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10ff62d9-7f61-4e5b-a030-c4f2b096b291 None admin] REQUEST_METHOD: `GET` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:02:57.202968 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10ff62d9-7f61-4e5b-a030-c4f2b096b291 None admin] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:02:57.203363 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10ff62d9-7f61-4e5b-a030-c4f2b096b291 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:02:57.216633 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application [None req-1f3e2ee3-0b21-40f9-8f5e-8dbe1efe45e0 None admin] Could not find user: glance-swift.: keystone.exception.UserNotFound: Could not find user: glance-swift. Aug 02 10:02:57.216633 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 02 10:02:57.216633 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 02 10:02:57.216633 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 02 10:02:57.216633 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 02 10:02:57.216633 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 02 10:02:57.216633 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 02 10:02:57.216633 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 02 10:02:57.216633 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 02 10:02:57.216633 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 02 10:02:57.216633 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 02 10:02:57.216633 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 02 10:02:57.216633 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 02 10:02:57.216633 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 02 10:02:57.216633 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 02 10:02:57.216633 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 02 10:02:57.217675 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 02 10:02:57.217675 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 02 10:02:57.217675 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 02 10:02:57.217675 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 02 10:02:57.217675 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 02 10:02:57.217675 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 02 10:02:57.217675 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 02 10:02:57.217675 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 02 10:02:57.217675 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 02 10:02:57.217675 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application return self.get_or_create( Aug 02 10:02:57.217675 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 02 10:02:57.217675 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application with Lock( Aug 02 10:02:57.217675 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 02 10:02:57.217675 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application return self._enter() Aug 02 10:02:57.217675 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 02 10:02:57.217675 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 02 10:02:57.217675 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 02 10:02:57.217675 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application return self.creator() Aug 02 10:02:57.218950 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 02 10:02:57.218950 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application created_value = creator( Aug 02 10:02:57.218950 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 02 10:02:57.218950 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 02 10:02:57.218950 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 02 10:02:57.218950 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 02 10:02:57.218950 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 02 10:02:57.218950 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 02 10:02:57.218950 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance-swift. Aug 02 10:02:57.218950 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application Aug 02 10:02:57.221594 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: [pid: 80377|app: 0|req: 414/818] 10.222.0.82 () {56 vars in 1117 bytes} [Tue Aug 2 10:02:56 2022] GET /identity/v3/users/glance-swift => generated 90 bytes in 267 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 02 10:02:57.222057 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10ff62d9-7f61-4e5b-a030-c4f2b096b291 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:02:57.222959 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f3e2ee3-0b21-40f9-8f5e-8dbe1efe45e0 None admin] Authenticating user token {{(pid=80377) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:02:57.224581 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10ff62d9-7f61-4e5b-a030-c4f2b096b291 None admin] RBAC: Authorization granted {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:02:57.250793 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application [None req-10ff62d9-7f61-4e5b-a030-c4f2b096b291 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 02 10:02:57.250793 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 02 10:02:57.250793 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 02 10:02:57.250793 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 02 10:02:57.250793 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 02 10:02:57.250793 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 02 10:02:57.250793 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 02 10:02:57.250793 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 02 10:02:57.250793 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 02 10:02:57.250793 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 02 10:02:57.250793 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 02 10:02:57.250793 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 02 10:02:57.250793 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 02 10:02:57.250793 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 02 10:02:57.250793 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 02 10:02:57.253153 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 02 10:02:57.253153 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 02 10:02:57.253153 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 02 10:02:57.253153 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 02 10:02:57.253153 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 02 10:02:57.253153 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 02 10:02:57.253153 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application return self.get_or_create( Aug 02 10:02:57.253153 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 02 10:02:57.253153 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application with Lock( Aug 02 10:02:57.253153 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 02 10:02:57.253153 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application return self._enter() Aug 02 10:02:57.253153 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 02 10:02:57.253153 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 02 10:02:57.253153 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 02 10:02:57.253153 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application return self.creator() Aug 02 10:02:57.253153 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 02 10:02:57.253153 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application created_value = creator( Aug 02 10:02:57.255775 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 02 10:02:57.255775 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 02 10:02:57.255775 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 02 10:02:57.255775 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application Aug 02 10:02:57.258227 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 405/819] 10.222.0.82 () {56 vars in 1108 bytes} [Tue Aug 2 10:02:56 2022] GET /identity/v3/domains/Default => generated 87 bytes in 266 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 02 10:02:57.259905 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10ff62d9-7f61-4e5b-a030-c4f2b096b291 None admin] Authenticating user token {{(pid=80378) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:02:57.343679 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG dbcounter [-] [80378] Writing DB stats keystone:SELECT=100 {{(pid=80378) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 10:02:57.405501 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f3e2ee3-0b21-40f9-8f5e-8dbe1efe45e0 None admin] Validating token access rules against request {{(pid=80377) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:02:57.431516 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10ff62d9-7f61-4e5b-a030-c4f2b096b291 None admin] Validating token access rules against request {{(pid=80378) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:02:57.464659 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b30775f-ca50-484e-bfc5-7cdd1a5089f4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '891005fa5ff74041b75747d16cc66798', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80377) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:02:57.465341 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b30775f-ca50-484e-bfc5-7cdd1a5089f4 None admin] REQUEST_METHOD: `GET` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:02:57.465818 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b30775f-ca50-484e-bfc5-7cdd1a5089f4 None admin] SCRIPT_NAME: `/identity` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:02:57.466063 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b30775f-ca50-484e-bfc5-7cdd1a5089f4 None admin] PATH_INFO: `/v3/services` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:02:57.467009 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b30775f-ca50-484e-bfc5-7cdd1a5089f4 None admin] RBAC: Adding query filter params (name=network) {{(pid=80377) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 02 10:02:57.467322 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b30775f-ca50-484e-bfc5-7cdd1a5089f4 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:02:57.469009 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b30775f-ca50-484e-bfc5-7cdd1a5089f4 None admin] RBAC: Authorization granted {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:02:57.483918 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: [pid: 80377|app: 0|req: 415/820] 10.222.0.82 () {56 vars in 1112 bytes} [Tue Aug 2 10:02:57 2022] GET /identity/v3/services?name=network => generated 124 bytes in 255 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 02 10:02:57.485194 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b30775f-ca50-484e-bfc5-7cdd1a5089f4 None admin] Authenticating user token {{(pid=80377) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:02:57.486578 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c5c0acc-19f9-478d-9dcc-c64b89366d13 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '891005fa5ff74041b75747d16cc66798', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80378) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:02:57.487476 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c5c0acc-19f9-478d-9dcc-c64b89366d13 None admin] REQUEST_METHOD: `GET` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:02:57.488013 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c5c0acc-19f9-478d-9dcc-c64b89366d13 None admin] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:02:57.488489 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c5c0acc-19f9-478d-9dcc-c64b89366d13 None admin] PATH_INFO: `/v3/users` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:02:57.489576 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c5c0acc-19f9-478d-9dcc-c64b89366d13 None admin] RBAC: Adding query filter params (domain_id=default, name=glance-swift) {{(pid=80378) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 02 10:02:57.490486 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c5c0acc-19f9-478d-9dcc-c64b89366d13 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:02:57.493610 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c5c0acc-19f9-478d-9dcc-c64b89366d13 None admin] RBAC: Authorization granted {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:02:57.537180 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 406/821] 10.222.0.82 () {56 vars in 1149 bytes} [Tue Aug 2 10:02:57 2022] GET /identity/v3/users?domain_id=default&name=glance-swift => generated 384 bytes in 278 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 02 10:02:57.538821 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c5c0acc-19f9-478d-9dcc-c64b89366d13 None admin] Authenticating user token {{(pid=80378) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:02:57.560345 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG dbcounter [-] [80377] Writing DB stats keystone:SELECT=100 {{(pid=80377) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 10:02:57.661043 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b30775f-ca50-484e-bfc5-7cdd1a5089f4 None admin] Validating token access rules against request {{(pid=80377) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:02:57.715507 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c5c0acc-19f9-478d-9dcc-c64b89366d13 None admin] Validating token access rules against request {{(pid=80378) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:02:57.715881 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-089329d5-59c5-4cde-b933-18d4cadd6ac3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '891005fa5ff74041b75747d16cc66798', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80377) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:02:57.716819 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-089329d5-59c5-4cde-b933-18d4cadd6ac3 None admin] REQUEST_METHOD: `GET` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:02:57.717707 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-089329d5-59c5-4cde-b933-18d4cadd6ac3 None admin] SCRIPT_NAME: `/identity` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:02:57.718213 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-089329d5-59c5-4cde-b933-18d4cadd6ac3 None admin] PATH_INFO: `/v3/domains` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:02:57.719280 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-089329d5-59c5-4cde-b933-18d4cadd6ac3 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80377) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 02 10:02:57.719994 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-089329d5-59c5-4cde-b933-18d4cadd6ac3 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:02:57.724010 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-089329d5-59c5-4cde-b933-18d4cadd6ac3 None admin] RBAC: Authorization granted {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:02:57.742850 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG dbcounter [-] [80378] Writing DB stats keystone:SELECT=100 {{(pid=80378) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 10:02:57.749944 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: [pid: 80377|app: 0|req: 416/822] 10.222.0.82 () {56 vars in 1109 bytes} [Tue Aug 2 10:02:57 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 265 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 02 10:02:57.752649 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-089329d5-59c5-4cde-b933-18d4cadd6ac3 None admin] Authenticating user token {{(pid=80377) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:02:57.776199 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c80a180e-e5e0-4cc5-8449-c6a0faf067bb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '891005fa5ff74041b75747d16cc66798', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80378) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:02:57.777168 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c80a180e-e5e0-4cc5-8449-c6a0faf067bb None admin] REQUEST_METHOD: `GET` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:02:57.777754 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c80a180e-e5e0-4cc5-8449-c6a0faf067bb None admin] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:02:57.778265 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c80a180e-e5e0-4cc5-8449-c6a0faf067bb None admin] PATH_INFO: `/v3/services` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:02:57.779728 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c80a180e-e5e0-4cc5-8449-c6a0faf067bb None admin] RBAC: Adding query filter params (type=network) {{(pid=80378) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 02 10:02:57.780234 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c80a180e-e5e0-4cc5-8449-c6a0faf067bb None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:02:57.782943 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c80a180e-e5e0-4cc5-8449-c6a0faf067bb None admin] RBAC: Authorization granted {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:02:57.792296 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 407/823] 10.222.0.82 () {56 vars in 1112 bytes} [Tue Aug 2 10:02:57 2022] GET /identity/v3/services?type=network => generated 350 bytes in 254 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 02 10:02:57.793770 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c80a180e-e5e0-4cc5-8449-c6a0faf067bb None admin] Authenticating user token {{(pid=80378) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:02:57.935079 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-089329d5-59c5-4cde-b933-18d4cadd6ac3 None admin] Validating token access rules against request {{(pid=80377) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:02:57.937407 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG dbcounter [-] [80377] Writing DB stats keystone:SELECT=100 {{(pid=80377) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 10:02:57.995053 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97aae7f3-1ed0-461e-a9fb-1f695c220b55 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '891005fa5ff74041b75747d16cc66798', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80377) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:02:57.996115 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97aae7f3-1ed0-461e-a9fb-1f695c220b55 None admin] REQUEST_METHOD: `GET` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:02:57.996683 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97aae7f3-1ed0-461e-a9fb-1f695c220b55 None admin] SCRIPT_NAME: `/identity` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:02:57.997153 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97aae7f3-1ed0-461e-a9fb-1f695c220b55 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:02:58.012940 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97aae7f3-1ed0-461e-a9fb-1f695c220b55 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:02:58.016191 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97aae7f3-1ed0-461e-a9fb-1f695c220b55 None admin] RBAC: Authorization granted {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:02:58.036384 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application [None req-97aae7f3-1ed0-461e-a9fb-1f695c220b55 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 02 10:02:58.036384 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 02 10:02:58.036384 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 02 10:02:58.036384 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 02 10:02:58.036384 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 02 10:02:58.036384 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 02 10:02:58.036384 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 02 10:02:58.036384 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 02 10:02:58.036384 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 02 10:02:58.036384 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 02 10:02:58.036384 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 02 10:02:58.036384 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 02 10:02:58.036384 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 02 10:02:58.036384 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 02 10:02:58.036384 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 02 10:02:58.038435 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 02 10:02:58.038435 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 02 10:02:58.038435 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 02 10:02:58.038435 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 02 10:02:58.038435 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 02 10:02:58.038435 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 02 10:02:58.038435 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application return self.get_or_create( Aug 02 10:02:58.038435 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 02 10:02:58.038435 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application with Lock( Aug 02 10:02:58.038435 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 02 10:02:58.038435 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application return self._enter() Aug 02 10:02:58.038435 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 02 10:02:58.038435 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 02 10:02:58.038435 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 02 10:02:58.038435 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application return self.creator() Aug 02 10:02:58.038435 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 02 10:02:58.038435 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application created_value = creator( Aug 02 10:02:58.040204 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 02 10:02:58.040204 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 02 10:02:58.040204 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 02 10:02:58.040204 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application Aug 02 10:02:58.043285 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: [pid: 80377|app: 0|req: 417/824] 10.222.0.82 () {56 vars in 1108 bytes} [Tue Aug 2 10:02:57 2022] GET /identity/v3/domains/Default => generated 87 bytes in 293 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 02 10:02:58.044592 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97aae7f3-1ed0-461e-a9fb-1f695c220b55 None admin] Authenticating user token {{(pid=80377) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:02:58.060564 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c80a180e-e5e0-4cc5-8449-c6a0faf067bb None admin] Validating token access rules against request {{(pid=80378) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:02:58.140781 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b614dd99-4fed-4ade-80d5-56a7a8e28998 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '891005fa5ff74041b75747d16cc66798', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80378) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:02:58.141866 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b614dd99-4fed-4ade-80d5-56a7a8e28998 None admin] REQUEST_METHOD: `GET` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:02:58.142894 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b614dd99-4fed-4ade-80d5-56a7a8e28998 None admin] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:02:58.143390 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b614dd99-4fed-4ade-80d5-56a7a8e28998 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:02:58.154440 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b614dd99-4fed-4ade-80d5-56a7a8e28998 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:02:58.156232 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b614dd99-4fed-4ade-80d5-56a7a8e28998 None admin] RBAC: Authorization granted {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:02:58.177755 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application [None req-b614dd99-4fed-4ade-80d5-56a7a8e28998 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 02 10:02:58.177755 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 02 10:02:58.177755 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 02 10:02:58.177755 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 02 10:02:58.177755 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 02 10:02:58.177755 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 02 10:02:58.177755 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 02 10:02:58.177755 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 02 10:02:58.177755 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 02 10:02:58.177755 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 02 10:02:58.177755 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 02 10:02:58.177755 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 02 10:02:58.177755 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 02 10:02:58.177755 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 02 10:02:58.177755 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 02 10:02:58.178922 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 02 10:02:58.178922 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 02 10:02:58.178922 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 02 10:02:58.178922 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 02 10:02:58.178922 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 02 10:02:58.178922 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 02 10:02:58.178922 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application return self.get_or_create( Aug 02 10:02:58.178922 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 02 10:02:58.178922 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application with Lock( Aug 02 10:02:58.178922 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 02 10:02:58.178922 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application return self._enter() Aug 02 10:02:58.178922 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 02 10:02:58.178922 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 02 10:02:58.178922 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 02 10:02:58.178922 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application return self.creator() Aug 02 10:02:58.178922 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 02 10:02:58.178922 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application created_value = creator( Aug 02 10:02:58.180321 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 02 10:02:58.180321 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 02 10:02:58.180321 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 02 10:02:58.180321 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 02 10:02:58.180321 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 02 10:02:58.180321 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 02 10:02:58.180321 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 02 10:02:58.180321 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application Aug 02 10:02:58.181722 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 408/825] 10.222.0.82 () {56 vars in 1111 bytes} [Tue Aug 2 10:02:57 2022] GET /identity/v3/projects/service => generated 88 bytes in 388 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 02 10:02:58.183663 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b614dd99-4fed-4ade-80d5-56a7a8e28998 None admin] Authenticating user token {{(pid=80378) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:02:58.214585 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97aae7f3-1ed0-461e-a9fb-1f695c220b55 None admin] Validating token access rules against request {{(pid=80377) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:02:58.259688 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG dbcounter [-] [80378] Writing DB stats keystone:SELECT=100 {{(pid=80378) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 10:02:58.307726 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66d26509-5613-4edc-a3a7-d0e8877fbeaf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '891005fa5ff74041b75747d16cc66798', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80377) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:02:58.308756 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66d26509-5613-4edc-a3a7-d0e8877fbeaf None admin] REQUEST_METHOD: `POST` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:02:58.309263 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66d26509-5613-4edc-a3a7-d0e8877fbeaf None admin] SCRIPT_NAME: `/identity` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:02:58.309759 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66d26509-5613-4edc-a3a7-d0e8877fbeaf None admin] PATH_INFO: `/v3/endpoints` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:02:58.311123 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66d26509-5613-4edc-a3a7-d0e8877fbeaf None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:02:58.313683 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66d26509-5613-4edc-a3a7-d0e8877fbeaf None admin] RBAC: Authorization granted {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:02:58.350260 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: [pid: 80377|app: 0|req: 418/826] 10.222.0.82 () {60 vars in 1144 bytes} [Tue Aug 2 10:02:58 2022] POST /identity/v3/endpoints => generated 337 bytes in 306 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 02 10:02:58.352395 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73199a1a-2fc3-4f23-bb24-43338b821e7f None None] REQUEST_METHOD: `GET` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:02:58.352918 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73199a1a-2fc3-4f23-bb24-43338b821e7f None None] SCRIPT_NAME: `/identity` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:02:58.353397 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73199a1a-2fc3-4f23-bb24-43338b821e7f None None] PATH_INFO: `/` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:02:58.358303 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: [pid: 80377|app: 0|req: 419/827] 10.222.0.82 () {54 vars in 926 bytes} [Tue Aug 2 10:02:58 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 02 10:02:58.358303 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73199a1a-2fc3-4f23-bb24-43338b821e7f None None] Authenticating user token {{(pid=80377) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:02:58.374745 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b614dd99-4fed-4ade-80d5-56a7a8e28998 None admin] Validating token access rules against request {{(pid=80378) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:02:58.387835 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG dbcounter [-] [80377] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80377) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 10:02:58.448999 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4633a37e-e48c-4c6c-ac24-e79d7e7e4c90 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '891005fa5ff74041b75747d16cc66798', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80378) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:02:58.449892 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4633a37e-e48c-4c6c-ac24-e79d7e7e4c90 None admin] REQUEST_METHOD: `GET` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:02:58.450182 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4633a37e-e48c-4c6c-ac24-e79d7e7e4c90 None admin] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:02:58.450446 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4633a37e-e48c-4c6c-ac24-e79d7e7e4c90 None admin] PATH_INFO: `/v3/domains` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:02:58.451301 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4633a37e-e48c-4c6c-ac24-e79d7e7e4c90 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80378) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 02 10:02:58.451612 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4633a37e-e48c-4c6c-ac24-e79d7e7e4c90 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:02:58.453227 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4633a37e-e48c-4c6c-ac24-e79d7e7e4c90 None admin] RBAC: Authorization granted {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:02:58.471058 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 409/828] 10.222.0.82 () {56 vars in 1109 bytes} [Tue Aug 2 10:02:58 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 289 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 02 10:02:58.472334 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fe39cef-89bf-4a6e-b9b8-f282fc41755d None None] REQUEST_METHOD: `POST` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:02:58.472618 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fe39cef-89bf-4a6e-b9b8-f282fc41755d None None] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:02:58.472890 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fe39cef-89bf-4a6e-b9b8-f282fc41755d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:02:58.494277 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.auth.core [None req-3fe39cef-89bf-4a6e-b9b8-f282fc41755d None None] MFA Rules not processed for user `891005fa5ff74041b75747d16cc66798`. Rule list: `[]` (Enabled: `True`). {{(pid=80378) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 02 10:02:58.495527 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.fernet_utils [None req-3fe39cef-89bf-4a6e-b9b8-f282fc41755d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80378) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 10:02:58.578257 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73199a1a-2fc3-4f23-bb24-43338b821e7f None None] Validating token access rules against request {{(pid=80377) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:02:58.594723 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 410/829] 10.222.0.82 () {58 vars in 1023 bytes} [Tue Aug 2 10:02:58 2022] POST /identity/v3/auth/tokens => generated 1161 bytes in 123 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 02 10:02:58.596604 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de06f7fb-c0e1-4b36-b0d7-27cf5a730113 None None] REQUEST_METHOD: `GET` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:02:58.596889 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de06f7fb-c0e1-4b36-b0d7-27cf5a730113 None None] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:02:58.597165 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de06f7fb-c0e1-4b36-b0d7-27cf5a730113 None None] PATH_INFO: `/` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:02:58.598686 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 411/830] 10.222.0.82 () {54 vars in 926 bytes} [Tue Aug 2 10:02:58 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 02 10:02:58.604347 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de06f7fb-c0e1-4b36-b0d7-27cf5a730113 None None] Authenticating user token {{(pid=80378) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:02:58.645516 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG dbcounter [-] [80378] Writing DB stats keystone:SELECT=100 {{(pid=80378) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 10:02:58.649992 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f4de435-b6af-4858-98fd-f5973cc1692c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '891005fa5ff74041b75747d16cc66798', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80377) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:02:58.650965 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f4de435-b6af-4858-98fd-f5973cc1692c None admin] REQUEST_METHOD: `GET` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:02:58.651474 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f4de435-b6af-4858-98fd-f5973cc1692c None admin] SCRIPT_NAME: `/identity` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:02:58.651865 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f4de435-b6af-4858-98fd-f5973cc1692c None admin] PATH_INFO: `/v3/projects` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:02:58.653195 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f4de435-b6af-4858-98fd-f5973cc1692c None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80377) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 02 10:02:58.653722 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f4de435-b6af-4858-98fd-f5973cc1692c None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:02:58.655600 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f4de435-b6af-4858-98fd-f5973cc1692c None admin] RBAC: Authorization granted {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:02:58.680935 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: [pid: 80377|app: 0|req: 420/831] 10.222.0.82 () {56 vars in 1148 bytes} [Tue Aug 2 10:02:58 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 318 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 02 10:02:58.680935 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e91b95eb-38a0-46ea-8ece-57c90ff9e0d1 None None] REQUEST_METHOD: `POST` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:02:58.680935 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e91b95eb-38a0-46ea-8ece-57c90ff9e0d1 None None] SCRIPT_NAME: `/identity` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:02:58.680935 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e91b95eb-38a0-46ea-8ece-57c90ff9e0d1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:02:58.699433 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.auth.core [None req-e91b95eb-38a0-46ea-8ece-57c90ff9e0d1 None None] MFA Rules not processed for user `891005fa5ff74041b75747d16cc66798`. Rule list: `[]` (Enabled: `True`). {{(pid=80377) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 02 10:02:58.700751 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.fernet_utils [None req-e91b95eb-38a0-46ea-8ece-57c90ff9e0d1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80377) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 10:02:58.761674 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de06f7fb-c0e1-4b36-b0d7-27cf5a730113 None None] Validating token access rules against request {{(pid=80378) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:02:58.796585 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG dbcounter [-] [80377] Writing DB stats keystone:SELECT=100 {{(pid=80377) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 10:02:58.802934 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: [pid: 80377|app: 0|req: 421/832] 10.222.0.82 () {58 vars in 1023 bytes} [Tue Aug 2 10:02:58 2022] POST /identity/v3/auth/tokens => generated 1161 bytes in 127 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 02 10:02:58.804612 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e91b95eb-38a0-46ea-8ece-57c90ff9e0d1 None None] Authenticating user token {{(pid=80377) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:02:58.808168 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8da83ce-534d-4565-b352-ca9e4d40e4b6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '891005fa5ff74041b75747d16cc66798', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80378) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:02:58.808827 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8da83ce-534d-4565-b352-ca9e4d40e4b6 None admin] REQUEST_METHOD: `GET` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:02:58.809103 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8da83ce-534d-4565-b352-ca9e4d40e4b6 None admin] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:02:58.809375 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8da83ce-534d-4565-b352-ca9e4d40e4b6 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:02:58.818787 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8da83ce-534d-4565-b352-ca9e4d40e4b6 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:02:58.820643 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8da83ce-534d-4565-b352-ca9e4d40e4b6 None admin] RBAC: Authorization granted {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:02:58.843588 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application [None req-e8da83ce-534d-4565-b352-ca9e4d40e4b6 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 02 10:02:58.843588 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 02 10:02:58.843588 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 02 10:02:58.843588 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 02 10:02:58.843588 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 02 10:02:58.843588 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 02 10:02:58.843588 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 02 10:02:58.843588 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 02 10:02:58.843588 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 02 10:02:58.843588 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 02 10:02:58.843588 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 02 10:02:58.843588 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 02 10:02:58.843588 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 02 10:02:58.843588 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 02 10:02:58.843588 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 02 10:02:58.844879 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 02 10:02:58.844879 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 02 10:02:58.844879 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 02 10:02:58.844879 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 02 10:02:58.844879 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 02 10:02:58.844879 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 02 10:02:58.844879 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application return self.get_or_create( Aug 02 10:02:58.844879 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 02 10:02:58.844879 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application with Lock( Aug 02 10:02:58.844879 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 02 10:02:58.844879 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application return self._enter() Aug 02 10:02:58.844879 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 02 10:02:58.844879 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 02 10:02:58.844879 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 02 10:02:58.844879 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application return self.creator() Aug 02 10:02:58.844879 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 02 10:02:58.844879 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application created_value = creator( Aug 02 10:02:58.846023 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 02 10:02:58.846023 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 02 10:02:58.846023 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 02 10:02:58.846023 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 02 10:02:58.846023 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 02 10:02:58.846023 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 02 10:02:58.846023 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 02 10:02:58.846023 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application Aug 02 10:02:58.848006 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 412/833] 10.222.0.82 () {56 vars in 1111 bytes} [Tue Aug 2 10:02:58 2022] GET /identity/v3/projects/service => generated 88 bytes in 249 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 02 10:02:58.849064 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8da83ce-534d-4565-b352-ca9e4d40e4b6 None admin] Authenticating user token {{(pid=80378) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:02:58.957739 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e91b95eb-38a0-46ea-8ece-57c90ff9e0d1 None None] Validating token access rules against request {{(pid=80377) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:02:58.979353 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG dbcounter [-] [80378] Writing DB stats keystone:SELECT=100 {{(pid=80378) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 10:02:59.005532 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8da83ce-534d-4565-b352-ca9e4d40e4b6 None admin] Validating token access rules against request {{(pid=80378) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:02:59.006407 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68286dc0-1b05-4dcd-bbb4-a4ae89c39caf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '891005fa5ff74041b75747d16cc66798', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80377) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:02:59.007653 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68286dc0-1b05-4dcd-bbb4-a4ae89c39caf None admin] REQUEST_METHOD: `GET` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:02:59.008180 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68286dc0-1b05-4dcd-bbb4-a4ae89c39caf None admin] SCRIPT_NAME: `/identity` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:02:59.008761 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68286dc0-1b05-4dcd-bbb4-a4ae89c39caf None admin] PATH_INFO: `/v3/services/volumev3` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:02:59.009710 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68286dc0-1b05-4dcd-bbb4-a4ae89c39caf None admin] RBAC: Authorizing `identity:get_service(service_id=volumev3)` {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:02:59.012443 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68286dc0-1b05-4dcd-bbb4-a4ae89c39caf None admin] RBAC: Authorization granted {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:02:59.028915 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application [None req-68286dc0-1b05-4dcd-bbb4-a4ae89c39caf None admin] Could not find service: volumev3.: keystone.exception.ServiceNotFound: Could not find service: volumev3. Aug 02 10:02:59.028915 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 02 10:02:59.028915 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Aug 02 10:02:59.028915 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Aug 02 10:02:59.028915 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Aug 02 10:02:59.028915 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Aug 02 10:02:59.028915 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Aug 02 10:02:59.028915 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 02 10:02:59.028915 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: volumev3. Aug 02 10:02:59.028915 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application Aug 02 10:02:59.028915 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 02 10:02:59.028915 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application Aug 02 10:02:59.028915 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 02 10:02:59.028915 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 02 10:02:59.028915 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 02 10:02:59.028915 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 02 10:02:59.028915 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 02 10:02:59.030777 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 02 10:02:59.030777 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 02 10:02:59.030777 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 02 10:02:59.030777 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 02 10:02:59.030777 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 02 10:02:59.030777 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 02 10:02:59.030777 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Aug 02 10:02:59.030777 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application return self._get_service(service_id) Aug 02 10:02:59.030777 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Aug 02 10:02:59.030777 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Aug 02 10:02:59.030777 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 02 10:02:59.030777 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 02 10:02:59.030777 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 02 10:02:59.030777 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 02 10:02:59.030777 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 02 10:02:59.030777 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application return self.get_or_create( Aug 02 10:02:59.031718 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 02 10:02:59.031718 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application with Lock( Aug 02 10:02:59.031718 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 02 10:02:59.031718 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application return self._enter() Aug 02 10:02:59.031718 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 02 10:02:59.031718 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 02 10:02:59.031718 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 02 10:02:59.031718 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application return self.creator() Aug 02 10:02:59.031718 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 02 10:02:59.031718 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application created_value = creator( Aug 02 10:02:59.031718 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Aug 02 10:02:59.031718 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 02 10:02:59.031718 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: volumev3. Aug 02 10:02:59.031718 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application Aug 02 10:02:59.034261 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: [pid: 80377|app: 0|req: 422/834] 10.222.0.82 () {56 vars in 1114 bytes} [Tue Aug 2 10:02:58 2022] GET /identity/v3/services/volumev3 => generated 89 bytes in 230 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 02 10:02:59.035763 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68286dc0-1b05-4dcd-bbb4-a4ae89c39caf None admin] Authenticating user token {{(pid=80377) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:02:59.062435 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c6a2f4d-6ffe-48d7-b9ca-216b41833e3a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '891005fa5ff74041b75747d16cc66798', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80378) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:02:59.063386 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c6a2f4d-6ffe-48d7-b9ca-216b41833e3a None admin] REQUEST_METHOD: `GET` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:02:59.063822 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c6a2f4d-6ffe-48d7-b9ca-216b41833e3a None admin] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:02:59.064279 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c6a2f4d-6ffe-48d7-b9ca-216b41833e3a None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:02:59.065778 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c6a2f4d-6ffe-48d7-b9ca-216b41833e3a None admin] RBAC: Adding query filter params (role.id=237966442f05411191105bf336bc14f3, scope.project.id=f1b38ebb804c41c69772c998aae72c41, user.id=88af518196674fe593ad88d30111fb7d) {{(pid=80378) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 02 10:02:59.066249 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c6a2f4d-6ffe-48d7-b9ca-216b41833e3a None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:02:59.068787 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c6a2f4d-6ffe-48d7-b9ca-216b41833e3a None admin] RBAC: Authorization granted {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:02:59.077250 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 413/835] 10.222.0.82 () {56 vars in 1374 bytes} [Tue Aug 2 10:02:58 2022] GET /identity/v3/role_assignments?user.id=88af518196674fe593ad88d30111fb7d&scope.project.id=f1b38ebb804c41c69772c998aae72c41&role.id=237966442f05411191105bf336bc14f3 => generated 608 bytes in 228 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 02 10:02:59.078627 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c6a2f4d-6ffe-48d7-b9ca-216b41833e3a None admin] Authenticating user token {{(pid=80378) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:02:59.137072 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG dbcounter [-] [80377] Writing DB stats keystone:SELECT=100 {{(pid=80377) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 10:02:59.211511 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68286dc0-1b05-4dcd-bbb4-a4ae89c39caf None admin] Validating token access rules against request {{(pid=80377) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:02:59.234309 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c6a2f4d-6ffe-48d7-b9ca-216b41833e3a None admin] Validating token access rules against request {{(pid=80378) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:02:59.273354 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4dbf96cc-8863-46bd-a735-a5a29ec641de None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '891005fa5ff74041b75747d16cc66798', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80377) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:02:59.274900 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4dbf96cc-8863-46bd-a735-a5a29ec641de None admin] REQUEST_METHOD: `GET` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:02:59.274900 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4dbf96cc-8863-46bd-a735-a5a29ec641de None admin] SCRIPT_NAME: `/identity` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:02:59.274900 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4dbf96cc-8863-46bd-a735-a5a29ec641de None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:02:59.291480 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35b167d7-5e08-4854-8b06-a2ee66e216b8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '891005fa5ff74041b75747d16cc66798', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80378) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:02:59.291480 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35b167d7-5e08-4854-8b06-a2ee66e216b8 None admin] REQUEST_METHOD: `GET` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:02:59.294118 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35b167d7-5e08-4854-8b06-a2ee66e216b8 None admin] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:02:59.294118 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35b167d7-5e08-4854-8b06-a2ee66e216b8 None admin] PATH_INFO: `/v3/projects` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:02:59.294418 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4dbf96cc-8863-46bd-a735-a5a29ec641de None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 02 10:02:59.294563 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35b167d7-5e08-4854-8b06-a2ee66e216b8 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80378) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 02 10:02:59.294563 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35b167d7-5e08-4854-8b06-a2ee66e216b8 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:02:59.295661 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35b167d7-5e08-4854-8b06-a2ee66e216b8 None admin] RBAC: Authorization granted {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:02:59.303491 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application [None req-4dbf96cc-8863-46bd-a735-a5a29ec641de None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 02 10:02:59.303491 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 02 10:02:59.303491 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 02 10:02:59.303491 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 02 10:02:59.303491 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 02 10:02:59.303491 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 02 10:02:59.303491 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 02 10:02:59.303491 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 02 10:02:59.303491 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 02 10:02:59.303491 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 02 10:02:59.303491 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 02 10:02:59.303491 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 02 10:02:59.303491 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 02 10:02:59.303491 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 02 10:02:59.303491 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 02 10:02:59.303491 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 02 10:02:59.304667 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 02 10:02:59.304667 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 02 10:02:59.304667 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 02 10:02:59.304667 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 02 10:02:59.304667 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 02 10:02:59.304667 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 02 10:02:59.304667 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 02 10:02:59.304667 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 02 10:02:59.304667 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 02 10:02:59.304667 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application return self.get_or_create( Aug 02 10:02:59.304667 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 02 10:02:59.304667 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application with Lock( Aug 02 10:02:59.304667 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 02 10:02:59.304667 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application return self._enter() Aug 02 10:02:59.304667 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 02 10:02:59.304667 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 02 10:02:59.305671 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 02 10:02:59.305671 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application return self.creator() Aug 02 10:02:59.305671 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 02 10:02:59.305671 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application created_value = creator( Aug 02 10:02:59.305671 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 02 10:02:59.305671 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 02 10:02:59.305671 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 02 10:02:59.305671 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 02 10:02:59.305671 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 02 10:02:59.305671 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 02 10:02:59.305671 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 02 10:02:59.305671 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application Aug 02 10:02:59.306377 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: [pid: 80377|app: 0|req: 423/836] 10.222.0.82 () {56 vars in 1096 bytes} [Tue Aug 2 10:02:59 2022] GET /identity/v3/roles/admin => generated 83 bytes in 271 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 02 10:02:59.308114 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4dbf96cc-8863-46bd-a735-a5a29ec641de None admin] Authenticating user token {{(pid=80377) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:02:59.315538 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 414/837] 10.222.0.82 () {56 vars in 1148 bytes} [Tue Aug 2 10:02:59 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 238 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 02 10:02:59.316686 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35b167d7-5e08-4854-8b06-a2ee66e216b8 None admin] Authenticating user token {{(pid=80378) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:02:59.335179 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG dbcounter [-] [80378] Writing DB stats keystone:SELECT=100 {{(pid=80378) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 10:02:59.481678 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35b167d7-5e08-4854-8b06-a2ee66e216b8 None admin] Validating token access rules against request {{(pid=80378) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:02:59.486584 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4dbf96cc-8863-46bd-a735-a5a29ec641de None admin] Validating token access rules against request {{(pid=80377) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:02:59.535779 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG dbcounter [-] [80377] Writing DB stats keystone:SELECT=100 {{(pid=80377) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 10:02:59.540487 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffd888d1-9b3b-484e-b617-cbd54fb60fb3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '891005fa5ff74041b75747d16cc66798', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80378) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:02:59.541476 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffd888d1-9b3b-484e-b617-cbd54fb60fb3 None admin] REQUEST_METHOD: `GET` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:02:59.541814 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffd888d1-9b3b-484e-b617-cbd54fb60fb3 None admin] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:02:59.542111 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffd888d1-9b3b-484e-b617-cbd54fb60fb3 None admin] PATH_INFO: `/v3/roles` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:02:59.543037 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ffd888d1-9b3b-484e-b617-cbd54fb60fb3 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80378) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 02 10:02:59.543542 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ffd888d1-9b3b-484e-b617-cbd54fb60fb3 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:02:59.545763 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ffd888d1-9b3b-484e-b617-cbd54fb60fb3 None admin] RBAC: Authorization granted {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:02:59.554831 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-732512a6-17f4-4d3a-b5c1-caf6aa6a0b39 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '891005fa5ff74041b75747d16cc66798', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80377) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:02:59.555812 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-732512a6-17f4-4d3a-b5c1-caf6aa6a0b39 None admin] REQUEST_METHOD: `GET` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:02:59.556289 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-732512a6-17f4-4d3a-b5c1-caf6aa6a0b39 None admin] SCRIPT_NAME: `/identity` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:02:59.556762 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-732512a6-17f4-4d3a-b5c1-caf6aa6a0b39 None admin] PATH_INFO: `/v3/services` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:02:59.557909 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-732512a6-17f4-4d3a-b5c1-caf6aa6a0b39 None admin] RBAC: Adding query filter params (name=volumev3) {{(pid=80377) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 02 10:02:59.558427 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-732512a6-17f4-4d3a-b5c1-caf6aa6a0b39 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:02:59.560856 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-732512a6-17f4-4d3a-b5c1-caf6aa6a0b39 None admin] RBAC: Authorization granted {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:02:59.569670 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 415/838] 10.222.0.82 () {56 vars in 1099 bytes} [Tue Aug 2 10:02:59 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 254 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 02 10:02:59.570356 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: [pid: 80377|app: 0|req: 424/839] 10.222.0.82 () {56 vars in 1114 bytes} [Tue Aug 2 10:02:59 2022] GET /identity/v3/services?name=volumev3 => generated 125 bytes in 263 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 02 10:02:59.571638 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-732512a6-17f4-4d3a-b5c1-caf6aa6a0b39 None admin] Authenticating user token {{(pid=80377) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:02:59.571798 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffd888d1-9b3b-484e-b617-cbd54fb60fb3 None admin] Authenticating user token {{(pid=80378) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:02:59.689903 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG dbcounter [-] [80378] Writing DB stats keystone:SELECT=100 {{(pid=80378) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 10:02:59.741670 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffd888d1-9b3b-484e-b617-cbd54fb60fb3 None admin] Validating token access rules against request {{(pid=80378) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:02:59.748328 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-732512a6-17f4-4d3a-b5c1-caf6aa6a0b39 None admin] Validating token access rules against request {{(pid=80377) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:02:59.797226 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34018040-c422-45dc-a1d3-434bd69445e4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '891005fa5ff74041b75747d16cc66798', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80378) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:02:59.797853 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34018040-c422-45dc-a1d3-434bd69445e4 None admin] REQUEST_METHOD: `PUT` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:02:59.798161 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34018040-c422-45dc-a1d3-434bd69445e4 None admin] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:02:59.798487 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34018040-c422-45dc-a1d3-434bd69445e4 None admin] PATH_INFO: `/v3/projects/f1b38ebb804c41c69772c998aae72c41/users/bd8dd72086db48b8aaac2ed0058872df/roles/02003a0be60f47f2a8dfea17053af52c` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:02:59.808791 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9644b02b-1c44-423f-8c00-181416587257 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '891005fa5ff74041b75747d16cc66798', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80377) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:02:59.809754 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9644b02b-1c44-423f-8c00-181416587257 None admin] REQUEST_METHOD: `GET` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:02:59.810245 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9644b02b-1c44-423f-8c00-181416587257 None admin] SCRIPT_NAME: `/identity` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:02:59.810526 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9644b02b-1c44-423f-8c00-181416587257 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:02:59.820098 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34018040-c422-45dc-a1d3-434bd69445e4 None admin] RBAC: Authorizing `identity:create_grant(project_id=f1b38ebb804c41c69772c998aae72c41, user_id=bd8dd72086db48b8aaac2ed0058872df, role_id=02003a0be60f47f2a8dfea17053af52c)` {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:02:59.822322 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34018040-c422-45dc-a1d3-434bd69445e4 None admin] RBAC: Authorization granted {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:02:59.825495 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9644b02b-1c44-423f-8c00-181416587257 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:02:59.826992 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9644b02b-1c44-423f-8c00-181416587257 None admin] RBAC: Authorization granted {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:02:59.849935 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 416/840] 10.222.0.82 () {58 vars in 1439 bytes} [Tue Aug 2 10:02:59 2022] PUT /identity/v3/projects/f1b38ebb804c41c69772c998aae72c41/users/bd8dd72086db48b8aaac2ed0058872df/roles/02003a0be60f47f2a8dfea17053af52c => generated 0 bytes in 271 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 02 10:02:59.849935 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34018040-c422-45dc-a1d3-434bd69445e4 None admin] Authenticating user token {{(pid=80378) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:02:59.850233 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application [None req-9644b02b-1c44-423f-8c00-181416587257 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 02 10:02:59.850233 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 02 10:02:59.850233 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 02 10:02:59.850233 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 02 10:02:59.850233 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 02 10:02:59.850233 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 02 10:02:59.850233 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 02 10:02:59.850233 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 02 10:02:59.850233 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 02 10:02:59.850233 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 02 10:02:59.850233 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 02 10:02:59.850233 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 02 10:02:59.850233 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 02 10:02:59.850233 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 02 10:02:59.850233 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 02 10:02:59.851035 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 02 10:02:59.851035 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 02 10:02:59.851035 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 02 10:02:59.851035 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 02 10:02:59.851035 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 02 10:02:59.851035 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 02 10:02:59.851035 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application return self.get_or_create( Aug 02 10:02:59.851035 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 02 10:02:59.851035 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application with Lock( Aug 02 10:02:59.851035 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 02 10:02:59.851035 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application return self._enter() Aug 02 10:02:59.851035 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 02 10:02:59.851035 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 02 10:02:59.851035 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 02 10:02:59.851035 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application return self.creator() Aug 02 10:02:59.851035 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 02 10:02:59.851035 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application created_value = creator( Aug 02 10:02:59.852003 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 02 10:02:59.852003 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 02 10:02:59.852003 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 02 10:02:59.852003 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application Aug 02 10:02:59.852003 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: [pid: 80377|app: 0|req: 425/841] 10.222.0.82 () {56 vars in 1108 bytes} [Tue Aug 2 10:02:59 2022] GET /identity/v3/domains/Default => generated 87 bytes in 277 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 02 10:02:59.858021 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9644b02b-1c44-423f-8c00-181416587257 None admin] Authenticating user token {{(pid=80377) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:02:59.934723 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG dbcounter [-] [80377] Writing DB stats keystone:SELECT=100 {{(pid=80377) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 10:03:00.043878 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9644b02b-1c44-423f-8c00-181416587257 None admin] Validating token access rules against request {{(pid=80377) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:03:00.045917 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34018040-c422-45dc-a1d3-434bd69445e4 None admin] Validating token access rules against request {{(pid=80378) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:03:00.084196 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG dbcounter [-] [80378] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80378) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 10:03:00.096096 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fd8311e-e3ef-4a4a-91ef-219f004ffaca None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '891005fa5ff74041b75747d16cc66798', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80377) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:03:00.096747 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fd8311e-e3ef-4a4a-91ef-219f004ffaca None admin] REQUEST_METHOD: `GET` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:03:00.097029 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fd8311e-e3ef-4a4a-91ef-219f004ffaca None admin] SCRIPT_NAME: `/identity` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:03:00.097319 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fd8311e-e3ef-4a4a-91ef-219f004ffaca None admin] PATH_INFO: `/v3/domains` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:03:00.098057 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0fd8311e-e3ef-4a4a-91ef-219f004ffaca None admin] RBAC: Adding query filter params (name=Default) {{(pid=80377) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 02 10:03:00.098367 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0fd8311e-e3ef-4a4a-91ef-219f004ffaca None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:03:00.099842 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0fd8311e-e3ef-4a4a-91ef-219f004ffaca None admin] RBAC: Authorization granted {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:03:00.114757 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: [pid: 80377|app: 0|req: 426/842] 10.222.0.82 () {56 vars in 1109 bytes} [Tue Aug 2 10:02:59 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 257 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 02 10:03:00.115732 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c8b1333-61c2-4d05-944d-562e1d7e6377 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '891005fa5ff74041b75747d16cc66798', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80378) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:03:00.116377 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c8b1333-61c2-4d05-944d-562e1d7e6377 None admin] REQUEST_METHOD: `GET` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:03:00.116697 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c8b1333-61c2-4d05-944d-562e1d7e6377 None admin] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:03:00.116971 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c8b1333-61c2-4d05-944d-562e1d7e6377 None admin] PATH_INFO: `/v3/services` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:03:00.117707 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c8b1333-61c2-4d05-944d-562e1d7e6377 None admin] RBAC: Adding query filter params (type=volumev3) {{(pid=80378) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 02 10:03:00.118100 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c8b1333-61c2-4d05-944d-562e1d7e6377 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:03:00.121077 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c8b1333-61c2-4d05-944d-562e1d7e6377 None admin] RBAC: Authorization granted {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:03:00.125927 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 417/843] 10.222.0.82 () {56 vars in 1114 bytes} [Tue Aug 2 10:02:59 2022] GET /identity/v3/services?type=volumev3 => generated 125 bytes in 284 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 02 10:03:00.127062 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fd8311e-e3ef-4a4a-91ef-219f004ffaca None admin] Authenticating user token {{(pid=80377) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:03:00.276721 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG dbcounter [-] [80377] Writing DB stats keystone:SELECT=100 {{(pid=80377) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 10:03:00.280880 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fd8311e-e3ef-4a4a-91ef-219f004ffaca None admin] Validating token access rules against request {{(pid=80377) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:03:00.337489 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e528da2-5445-4550-9662-b71144d49d44 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '891005fa5ff74041b75747d16cc66798', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80377) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:03:00.338481 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e528da2-5445-4550-9662-b71144d49d44 None admin] REQUEST_METHOD: `GET` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:03:00.338893 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e528da2-5445-4550-9662-b71144d49d44 None admin] SCRIPT_NAME: `/identity` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:03:00.339315 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e528da2-5445-4550-9662-b71144d49d44 None admin] PATH_INFO: `/v3/users/swift` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:03:00.354759 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e528da2-5445-4550-9662-b71144d49d44 None admin] RBAC: Authorizing `identity:get_user(user_id=swift)` {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:03:00.357076 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e528da2-5445-4550-9662-b71144d49d44 None admin] RBAC: Authorization granted {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:03:00.378916 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application [None req-1e528da2-5445-4550-9662-b71144d49d44 None admin] Could not find user: swift.: keystone.exception.UserNotFound: Could not find user: swift. Aug 02 10:03:00.378916 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 02 10:03:00.378916 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 02 10:03:00.378916 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 02 10:03:00.378916 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 02 10:03:00.378916 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 02 10:03:00.378916 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 02 10:03:00.378916 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 02 10:03:00.378916 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 02 10:03:00.378916 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 02 10:03:00.378916 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 02 10:03:00.378916 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 02 10:03:00.378916 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 02 10:03:00.378916 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 02 10:03:00.378916 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 02 10:03:00.378916 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 02 10:03:00.380845 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 02 10:03:00.380845 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 02 10:03:00.380845 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 02 10:03:00.380845 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 02 10:03:00.380845 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 02 10:03:00.380845 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 02 10:03:00.380845 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 02 10:03:00.380845 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 02 10:03:00.380845 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 02 10:03:00.380845 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application return self.get_or_create( Aug 02 10:03:00.380845 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 02 10:03:00.380845 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application with Lock( Aug 02 10:03:00.380845 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 02 10:03:00.380845 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application return self._enter() Aug 02 10:03:00.380845 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 02 10:03:00.380845 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 02 10:03:00.380845 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 02 10:03:00.380845 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application return self.creator() Aug 02 10:03:00.383925 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 02 10:03:00.383925 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application created_value = creator( Aug 02 10:03:00.383925 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 02 10:03:00.383925 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 02 10:03:00.383925 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 02 10:03:00.383925 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 02 10:03:00.383925 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 02 10:03:00.383925 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 02 10:03:00.383925 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: swift. Aug 02 10:03:00.383925 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application Aug 02 10:03:00.383925 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: [pid: 80377|app: 0|req: 427/844] 10.222.0.82 () {56 vars in 1096 bytes} [Tue Aug 2 10:03:00 2022] GET /identity/v3/users/swift => generated 83 bytes in 258 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 02 10:03:00.391115 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c8b1333-61c2-4d05-944d-562e1d7e6377 None admin] Authenticating user token {{(pid=80378) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:03:00.559959 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c8b1333-61c2-4d05-944d-562e1d7e6377 None admin] Validating token access rules against request {{(pid=80378) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:03:00.606788 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85ca5f2a-7ecc-467d-86a5-8f4c2ca25843 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '891005fa5ff74041b75747d16cc66798', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80378) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:03:00.607332 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85ca5f2a-7ecc-467d-86a5-8f4c2ca25843 None admin] REQUEST_METHOD: `GET` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:03:00.607619 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85ca5f2a-7ecc-467d-86a5-8f4c2ca25843 None admin] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:03:00.607896 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85ca5f2a-7ecc-467d-86a5-8f4c2ca25843 None admin] PATH_INFO: `/v3/users` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:03:00.608708 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85ca5f2a-7ecc-467d-86a5-8f4c2ca25843 None admin] RBAC: Adding query filter params (domain_id=default, name=swift) {{(pid=80378) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 02 10:03:00.609018 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85ca5f2a-7ecc-467d-86a5-8f4c2ca25843 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:03:00.611570 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85ca5f2a-7ecc-467d-86a5-8f4c2ca25843 None admin] RBAC: Authorization granted {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:03:00.633973 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 418/845] 10.222.0.82 () {56 vars in 1135 bytes} [Tue Aug 2 10:03:00 2022] GET /identity/v3/users?domain_id=default&name=swift => generated 370 bytes in 243 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 02 10:03:00.640215 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e528da2-5445-4550-9662-b71144d49d44 None admin] Authenticating user token {{(pid=80377) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:03:00.771125 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e528da2-5445-4550-9662-b71144d49d44 None admin] Validating token access rules against request {{(pid=80377) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:03:00.813774 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efd101fa-ae2b-4efc-be72-00853c418b7a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '891005fa5ff74041b75747d16cc66798', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80377) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:03:00.815405 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efd101fa-ae2b-4efc-be72-00853c418b7a None admin] REQUEST_METHOD: `GET` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:03:00.815869 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efd101fa-ae2b-4efc-be72-00853c418b7a None admin] SCRIPT_NAME: `/identity` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:03:00.816373 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efd101fa-ae2b-4efc-be72-00853c418b7a None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:03:00.860692 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG dbcounter [-] [80377] Writing DB stats keystone:SELECT=100 {{(pid=80377) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 10:03:00.911490 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-efd101fa-ae2b-4efc-be72-00853c418b7a None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:03:00.914078 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-efd101fa-ae2b-4efc-be72-00853c418b7a None admin] RBAC: Authorization granted {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:03:01.077446 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: [pid: 80377|app: 0|req: 428/846] 10.222.0.82 () {58 vars in 1282 bytes} [Tue Aug 2 10:03:00 2022] GET /identity/v3/auth/tokens => generated 1161 bytes in 438 msecs (HTTP/1.1 200) 6 headers in 359 bytes (2 switches on core 0) Aug 02 10:03:01.086953 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85ca5f2a-7ecc-467d-86a5-8f4c2ca25843 None admin] Authenticating user token {{(pid=80378) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:03:01.169021 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG dbcounter [-] [80378] Writing DB stats keystone:SELECT=100 {{(pid=80378) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 10:03:01.265300 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85ca5f2a-7ecc-467d-86a5-8f4c2ca25843 None admin] Validating token access rules against request {{(pid=80378) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:03:01.315957 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae8a7a81-e21d-4417-80f6-18fb44cef4ab None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '891005fa5ff74041b75747d16cc66798', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80378) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:03:01.316620 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae8a7a81-e21d-4417-80f6-18fb44cef4ab None admin] REQUEST_METHOD: `GET` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:03:01.316909 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae8a7a81-e21d-4417-80f6-18fb44cef4ab None admin] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:03:01.317179 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae8a7a81-e21d-4417-80f6-18fb44cef4ab None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:03:01.335816 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae8a7a81-e21d-4417-80f6-18fb44cef4ab None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:03:01.337716 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae8a7a81-e21d-4417-80f6-18fb44cef4ab None admin] RBAC: Authorization granted {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:03:01.368034 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application [None req-ae8a7a81-e21d-4417-80f6-18fb44cef4ab None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 02 10:03:01.368034 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 02 10:03:01.368034 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 02 10:03:01.368034 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 02 10:03:01.368034 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 02 10:03:01.368034 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 02 10:03:01.368034 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 02 10:03:01.368034 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 02 10:03:01.368034 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 02 10:03:01.368034 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 02 10:03:01.368034 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 02 10:03:01.368034 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 02 10:03:01.368034 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 02 10:03:01.368034 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 02 10:03:01.368034 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 02 10:03:01.369063 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 02 10:03:01.369063 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 02 10:03:01.369063 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 02 10:03:01.369063 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 02 10:03:01.369063 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 02 10:03:01.369063 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 02 10:03:01.369063 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application return self.get_or_create( Aug 02 10:03:01.369063 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 02 10:03:01.369063 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application with Lock( Aug 02 10:03:01.369063 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 02 10:03:01.369063 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application return self._enter() Aug 02 10:03:01.369063 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 02 10:03:01.369063 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 02 10:03:01.369063 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 02 10:03:01.369063 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application return self.creator() Aug 02 10:03:01.369063 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 02 10:03:01.369063 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application created_value = creator( Aug 02 10:03:01.370175 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 02 10:03:01.370175 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 02 10:03:01.370175 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 02 10:03:01.370175 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application Aug 02 10:03:01.370175 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 419/847] 10.222.0.82 () {56 vars in 1108 bytes} [Tue Aug 2 10:03:01 2022] GET /identity/v3/domains/Default => generated 87 bytes in 284 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 02 10:03:01.378901 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efd101fa-ae2b-4efc-be72-00853c418b7a None admin] Authenticating user token {{(pid=80377) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:03:01.454719 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG dbcounter [-] [80377] Writing DB stats keystone:SELECT=100 {{(pid=80377) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 10:03:01.505574 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efd101fa-ae2b-4efc-be72-00853c418b7a None admin] Validating token access rules against request {{(pid=80377) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:03:01.549865 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6326bf0d-5e3b-4ca7-8830-1cb343b7503c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '891005fa5ff74041b75747d16cc66798', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80377) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:03:01.550710 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6326bf0d-5e3b-4ca7-8830-1cb343b7503c None admin] REQUEST_METHOD: `GET` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:03:01.551058 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6326bf0d-5e3b-4ca7-8830-1cb343b7503c None admin] SCRIPT_NAME: `/identity` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:03:01.551359 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6326bf0d-5e3b-4ca7-8830-1cb343b7503c None admin] PATH_INFO: `/v3/domains` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:03:01.552098 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6326bf0d-5e3b-4ca7-8830-1cb343b7503c None admin] RBAC: Adding query filter params (name=Default) {{(pid=80377) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 02 10:03:01.552526 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6326bf0d-5e3b-4ca7-8830-1cb343b7503c None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:03:01.554176 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6326bf0d-5e3b-4ca7-8830-1cb343b7503c None admin] RBAC: Authorization granted {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:03:01.569872 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: [pid: 80377|app: 0|req: 429/848] 10.222.0.82 () {56 vars in 1109 bytes} [Tue Aug 2 10:03:01 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 192 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 02 10:03:01.576616 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae8a7a81-e21d-4417-80f6-18fb44cef4ab None admin] Authenticating user token {{(pid=80378) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:03:01.714803 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG dbcounter [-] [80378] Writing DB stats keystone:SELECT=100 {{(pid=80378) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 10:03:01.730630 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae8a7a81-e21d-4417-80f6-18fb44cef4ab None admin] Validating token access rules against request {{(pid=80378) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:03:01.747451 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3640da6d-7411-43a3-8071-1805bcea58ad None None] REQUEST_METHOD: `GET` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:03:01.748135 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3640da6d-7411-43a3-8071-1805bcea58ad None None] SCRIPT_NAME: `/identity` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:03:01.748455 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3640da6d-7411-43a3-8071-1805bcea58ad None None] PATH_INFO: `/` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:03:01.750137 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: [pid: 80377|app: 0|req: 430/849] 10.222.0.82 () {54 vars in 926 bytes} [Tue Aug 2 10:03:01 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 02 10:03:01.757845 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25589c38-0e0f-4087-8d90-cae699f7fdb9 None None] REQUEST_METHOD: `POST` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:03:01.758338 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25589c38-0e0f-4087-8d90-cae699f7fdb9 None None] SCRIPT_NAME: `/identity` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:03:01.758677 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25589c38-0e0f-4087-8d90-cae699f7fdb9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:03:01.788090 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ca3f614-35d8-46ca-a668-0e2460862535 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '891005fa5ff74041b75747d16cc66798', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80378) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:03:01.788302 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.auth.core [None req-25589c38-0e0f-4087-8d90-cae699f7fdb9 None None] MFA Rules not processed for user `891005fa5ff74041b75747d16cc66798`. Rule list: `[]` (Enabled: `True`). {{(pid=80377) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 02 10:03:01.789259 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ca3f614-35d8-46ca-a668-0e2460862535 None admin] REQUEST_METHOD: `GET` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:03:01.789646 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.fernet_utils [None req-25589c38-0e0f-4087-8d90-cae699f7fdb9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80377) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 10:03:01.790160 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ca3f614-35d8-46ca-a668-0e2460862535 None admin] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:03:01.790637 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ca3f614-35d8-46ca-a668-0e2460862535 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:03:01.800124 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ca3f614-35d8-46ca-a668-0e2460862535 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:03:01.802157 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ca3f614-35d8-46ca-a668-0e2460862535 None admin] RBAC: Authorization granted {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:03:01.819445 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application [None req-4ca3f614-35d8-46ca-a668-0e2460862535 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 02 10:03:01.819445 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 02 10:03:01.819445 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 02 10:03:01.819445 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 02 10:03:01.819445 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 02 10:03:01.819445 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 02 10:03:01.819445 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 02 10:03:01.819445 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 02 10:03:01.819445 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 02 10:03:01.819445 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 02 10:03:01.819445 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 02 10:03:01.819445 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 02 10:03:01.819445 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 02 10:03:01.819445 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 02 10:03:01.819445 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 02 10:03:01.820533 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 02 10:03:01.820533 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 02 10:03:01.820533 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 02 10:03:01.820533 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 02 10:03:01.820533 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 02 10:03:01.820533 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 02 10:03:01.820533 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application return self.get_or_create( Aug 02 10:03:01.820533 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 02 10:03:01.820533 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application with Lock( Aug 02 10:03:01.820533 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 02 10:03:01.820533 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application return self._enter() Aug 02 10:03:01.820533 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 02 10:03:01.820533 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 02 10:03:01.820533 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 02 10:03:01.820533 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application return self.creator() Aug 02 10:03:01.820533 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 02 10:03:01.820533 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application created_value = creator( Aug 02 10:03:01.821423 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 02 10:03:01.821423 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 02 10:03:01.821423 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 02 10:03:01.821423 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 02 10:03:01.821423 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 02 10:03:01.821423 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 02 10:03:01.821423 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 02 10:03:01.821423 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application Aug 02 10:03:01.823951 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 420/850] 10.222.0.82 () {56 vars in 1111 bytes} [Tue Aug 2 10:03:01 2022] GET /identity/v3/projects/service => generated 88 bytes in 248 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 02 10:03:01.830261 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ca3f614-35d8-46ca-a668-0e2460862535 None admin] Authenticating user token {{(pid=80378) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:03:01.910266 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: [pid: 80377|app: 0|req: 431/851] 10.222.0.82 () {58 vars in 1023 bytes} [Tue Aug 2 10:03:01 2022] POST /identity/v3/auth/tokens => generated 1161 bytes in 153 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Aug 02 10:03:01.929120 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25589c38-0e0f-4087-8d90-cae699f7fdb9 None None] Authenticating user token {{(pid=80377) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:03:01.974240 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ca3f614-35d8-46ca-a668-0e2460862535 None admin] Validating token access rules against request {{(pid=80378) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:03:01.992770 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG dbcounter [-] [80377] Writing DB stats keystone:SELECT=100 {{(pid=80377) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 10:03:02.019559 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92a76df0-1477-4114-8470-a2ce0b957477 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '891005fa5ff74041b75747d16cc66798', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80378) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:03:02.020943 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92a76df0-1477-4114-8470-a2ce0b957477 None admin] REQUEST_METHOD: `GET` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:03:02.021712 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92a76df0-1477-4114-8470-a2ce0b957477 None admin] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:03:02.022450 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92a76df0-1477-4114-8470-a2ce0b957477 None admin] PATH_INFO: `/v3/projects` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:03:02.023999 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92a76df0-1477-4114-8470-a2ce0b957477 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80378) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 02 10:03:02.024800 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92a76df0-1477-4114-8470-a2ce0b957477 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:03:02.027788 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92a76df0-1477-4114-8470-a2ce0b957477 None admin] RBAC: Authorization granted {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:03:02.049874 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 421/852] 10.222.0.82 () {56 vars in 1148 bytes} [Tue Aug 2 10:03:01 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 220 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 02 10:03:02.061844 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92a76df0-1477-4114-8470-a2ce0b957477 None admin] Authenticating user token {{(pid=80378) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:03:02.089191 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG dbcounter [-] [80378] Writing DB stats keystone:SELECT=100 {{(pid=80378) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 10:03:02.090003 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25589c38-0e0f-4087-8d90-cae699f7fdb9 None None] Validating token access rules against request {{(pid=80377) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:03:02.154340 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a87c09d-0171-4c70-923c-0324f0e730f1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '891005fa5ff74041b75747d16cc66798', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80377) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:03:02.155624 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a87c09d-0171-4c70-923c-0324f0e730f1 None admin] REQUEST_METHOD: `GET` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:03:02.156351 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a87c09d-0171-4c70-923c-0324f0e730f1 None admin] SCRIPT_NAME: `/identity` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:03:02.156900 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a87c09d-0171-4c70-923c-0324f0e730f1 None admin] PATH_INFO: `/v3/services/compute_legacy` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:03:02.157998 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a87c09d-0171-4c70-923c-0324f0e730f1 None admin] RBAC: Authorizing `identity:get_service(service_id=compute_legacy)` {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:03:02.161004 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a87c09d-0171-4c70-923c-0324f0e730f1 None admin] RBAC: Authorization granted {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:03:02.177522 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application [None req-6a87c09d-0171-4c70-923c-0324f0e730f1 None admin] Could not find service: compute_legacy.: keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Aug 02 10:03:02.177522 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 02 10:03:02.177522 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Aug 02 10:03:02.177522 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Aug 02 10:03:02.177522 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Aug 02 10:03:02.177522 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Aug 02 10:03:02.177522 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Aug 02 10:03:02.177522 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 02 10:03:02.177522 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Aug 02 10:03:02.177522 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application Aug 02 10:03:02.177522 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 02 10:03:02.177522 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application Aug 02 10:03:02.177522 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 02 10:03:02.177522 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 02 10:03:02.177522 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 02 10:03:02.177522 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 02 10:03:02.179509 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 02 10:03:02.179509 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 02 10:03:02.179509 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 02 10:03:02.179509 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 02 10:03:02.179509 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 02 10:03:02.179509 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 02 10:03:02.179509 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 02 10:03:02.179509 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Aug 02 10:03:02.179509 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application return self._get_service(service_id) Aug 02 10:03:02.179509 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Aug 02 10:03:02.179509 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Aug 02 10:03:02.179509 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 02 10:03:02.179509 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 02 10:03:02.179509 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 02 10:03:02.179509 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 02 10:03:02.179509 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 02 10:03:02.181021 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application return self.get_or_create( Aug 02 10:03:02.181021 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 02 10:03:02.181021 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application with Lock( Aug 02 10:03:02.181021 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 02 10:03:02.181021 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application return self._enter() Aug 02 10:03:02.181021 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 02 10:03:02.181021 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 02 10:03:02.181021 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 02 10:03:02.181021 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application return self.creator() Aug 02 10:03:02.181021 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 02 10:03:02.181021 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application created_value = creator( Aug 02 10:03:02.181021 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Aug 02 10:03:02.181021 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 02 10:03:02.181021 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Aug 02 10:03:02.181021 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application Aug 02 10:03:02.189857 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: [pid: 80377|app: 0|req: 432/853] 10.222.0.82 () {56 vars in 1132 bytes} [Tue Aug 2 10:03:01 2022] GET /identity/v3/services/compute_legacy => generated 95 bytes in 261 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 02 10:03:02.191150 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a87c09d-0171-4c70-923c-0324f0e730f1 None admin] Authenticating user token {{(pid=80377) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:03:02.217904 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92a76df0-1477-4114-8470-a2ce0b957477 None admin] Validating token access rules against request {{(pid=80378) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:03:02.263318 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d94744c3-c925-4a33-ad54-59284b7f89d5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '891005fa5ff74041b75747d16cc66798', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80378) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:03:02.264234 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d94744c3-c925-4a33-ad54-59284b7f89d5 None admin] REQUEST_METHOD: `GET` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:03:02.264608 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d94744c3-c925-4a33-ad54-59284b7f89d5 None admin] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:03:02.264936 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d94744c3-c925-4a33-ad54-59284b7f89d5 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:03:02.267766 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d94744c3-c925-4a33-ad54-59284b7f89d5 None admin] RBAC: Adding query filter params (role.id=237966442f05411191105bf336bc14f3, scope.project.id=f1b38ebb804c41c69772c998aae72c41, user.id=bef1b2e714cb45c4a51eb6d3e15aad6d) {{(pid=80378) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 02 10:03:02.268401 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d94744c3-c925-4a33-ad54-59284b7f89d5 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:03:02.270740 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d94744c3-c925-4a33-ad54-59284b7f89d5 None admin] RBAC: Authorization granted {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:03:02.289835 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 422/854] 10.222.0.82 () {56 vars in 1374 bytes} [Tue Aug 2 10:03:02 2022] GET /identity/v3/role_assignments?user.id=bef1b2e714cb45c4a51eb6d3e15aad6d&scope.project.id=f1b38ebb804c41c69772c998aae72c41&role.id=237966442f05411191105bf336bc14f3 => generated 608 bytes in 223 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 02 10:03:02.365297 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a87c09d-0171-4c70-923c-0324f0e730f1 None admin] Validating token access rules against request {{(pid=80377) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:03:02.394225 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG dbcounter [-] [80377] Writing DB stats keystone:SELECT=100 {{(pid=80377) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 10:03:02.426075 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e31ab8d1-f5e2-4d29-bd47-f88bd252999e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '891005fa5ff74041b75747d16cc66798', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80377) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:03:02.427171 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e31ab8d1-f5e2-4d29-bd47-f88bd252999e None admin] REQUEST_METHOD: `GET` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:03:02.428163 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e31ab8d1-f5e2-4d29-bd47-f88bd252999e None admin] SCRIPT_NAME: `/identity` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:03:02.428511 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e31ab8d1-f5e2-4d29-bd47-f88bd252999e None admin] PATH_INFO: `/v3/services` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:03:02.429298 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e31ab8d1-f5e2-4d29-bd47-f88bd252999e None admin] RBAC: Adding query filter params (name=compute_legacy) {{(pid=80377) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 02 10:03:02.429677 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e31ab8d1-f5e2-4d29-bd47-f88bd252999e None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:03:02.431275 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e31ab8d1-f5e2-4d29-bd47-f88bd252999e None admin] RBAC: Authorization granted {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:03:02.439759 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: [pid: 80377|app: 0|req: 433/855] 10.222.0.82 () {56 vars in 1126 bytes} [Tue Aug 2 10:03:02 2022] GET /identity/v3/services?name=compute_legacy => generated 131 bytes in 249 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 02 10:03:02.447148 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d94744c3-c925-4a33-ad54-59284b7f89d5 None admin] Authenticating user token {{(pid=80378) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:03:02.579258 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b86bbfd7-784e-4e6c-bc93-1c9dfa27a588 None None] REQUEST_METHOD: `GET` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:03:02.579677 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b86bbfd7-784e-4e6c-bc93-1c9dfa27a588 None None] SCRIPT_NAME: `/identity` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:03:02.580089 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG dbcounter [-] [80378] Writing DB stats keystone:SELECT=100 {{(pid=80378) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 10:03:02.580218 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b86bbfd7-784e-4e6c-bc93-1c9dfa27a588 None None] PATH_INFO: `/` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:03:02.582956 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: [pid: 80377|app: 0|req: 434/856] 10.222.0.82 () {54 vars in 926 bytes} [Tue Aug 2 10:03:02 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 02 10:03:02.589370 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3da129d-3bec-4f9c-9787-102749553532 None None] REQUEST_METHOD: `POST` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:03:02.589841 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3da129d-3bec-4f9c-9787-102749553532 None None] SCRIPT_NAME: `/identity` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:03:02.590319 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3da129d-3bec-4f9c-9787-102749553532 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:03:02.613988 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d94744c3-c925-4a33-ad54-59284b7f89d5 None admin] Validating token access rules against request {{(pid=80378) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:03:02.623217 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.auth.core [None req-f3da129d-3bec-4f9c-9787-102749553532 None None] MFA Rules not processed for user `891005fa5ff74041b75747d16cc66798`. Rule list: `[]` (Enabled: `True`). {{(pid=80377) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 02 10:03:02.624462 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.fernet_utils [None req-f3da129d-3bec-4f9c-9787-102749553532 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80377) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 10:03:02.682338 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de09fcee-6815-480f-9ce9-61a6dcaeaebb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '891005fa5ff74041b75747d16cc66798', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80378) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:03:02.683186 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de09fcee-6815-480f-9ce9-61a6dcaeaebb None admin] REQUEST_METHOD: `GET` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:03:02.683531 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de09fcee-6815-480f-9ce9-61a6dcaeaebb None admin] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:03:02.683929 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de09fcee-6815-480f-9ce9-61a6dcaeaebb None admin] PATH_INFO: `/v3/services` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:03:02.684854 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de09fcee-6815-480f-9ce9-61a6dcaeaebb None admin] RBAC: Adding query filter params (type=compute_legacy) {{(pid=80378) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 02 10:03:02.685215 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de09fcee-6815-480f-9ce9-61a6dcaeaebb None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:03:02.687162 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de09fcee-6815-480f-9ce9-61a6dcaeaebb None admin] RBAC: Authorization granted {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:03:02.699232 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 423/857] 10.222.0.82 () {56 vars in 1126 bytes} [Tue Aug 2 10:03:02 2022] GET /identity/v3/services?type=compute_legacy => generated 131 bytes in 253 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 02 10:03:02.742373 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: [pid: 80377|app: 0|req: 435/858] 10.222.0.82 () {58 vars in 1023 bytes} [Tue Aug 2 10:03:02 2022] POST /identity/v3/auth/tokens => generated 1161 bytes in 154 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 02 10:03:02.773342 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de09fcee-6815-480f-9ce9-61a6dcaeaebb None admin] Authenticating user token {{(pid=80378) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:03:02.839672 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f86ac75c-5d52-46fa-b00b-23545c90f4b4 None None] REQUEST_METHOD: `GET` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:03:02.839878 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f86ac75c-5d52-46fa-b00b-23545c90f4b4 None None] SCRIPT_NAME: `/identity` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:03:02.840146 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f86ac75c-5d52-46fa-b00b-23545c90f4b4 None None] PATH_INFO: `/` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:03:02.843046 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: [pid: 80377|app: 0|req: 436/859] 10.222.0.82 () {54 vars in 926 bytes} [Tue Aug 2 10:03:02 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 02 10:03:02.853069 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcf32e5b-d189-4f37-95a1-1b3ee1acf2bc None None] REQUEST_METHOD: `POST` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:03:02.853440 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcf32e5b-d189-4f37-95a1-1b3ee1acf2bc None None] SCRIPT_NAME: `/identity` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:03:02.853724 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcf32e5b-d189-4f37-95a1-1b3ee1acf2bc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:03:02.881859 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.auth.core [None req-dcf32e5b-d189-4f37-95a1-1b3ee1acf2bc None None] MFA Rules not processed for user `891005fa5ff74041b75747d16cc66798`. Rule list: `[]` (Enabled: `True`). {{(pid=80377) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 02 10:03:02.883783 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.fernet_utils [None req-dcf32e5b-d189-4f37-95a1-1b3ee1acf2bc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80377) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 10:03:02.949289 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de09fcee-6815-480f-9ce9-61a6dcaeaebb None admin] Validating token access rules against request {{(pid=80378) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:03:02.997197 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa845c2c-f763-4b87-929d-0811e23c086c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '891005fa5ff74041b75747d16cc66798', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80378) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:03:02.998053 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa845c2c-f763-4b87-929d-0811e23c086c None admin] REQUEST_METHOD: `GET` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:03:02.998440 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa845c2c-f763-4b87-929d-0811e23c086c None admin] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:03:02.998751 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa845c2c-f763-4b87-929d-0811e23c086c None admin] PATH_INFO: `/v3/roles/service` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:03:02.999072 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: [pid: 80377|app: 0|req: 437/860] 10.222.0.82 () {58 vars in 1023 bytes} [Tue Aug 2 10:03:02 2022] POST /identity/v3/auth/tokens => generated 1161 bytes in 147 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Aug 02 10:03:03.020067 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa845c2c-f763-4b87-929d-0811e23c086c None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 02 10:03:03.023127 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dcf32e5b-d189-4f37-95a1-1b3ee1acf2bc None None] Authenticating user token {{(pid=80377) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:03:03.027409 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application [None req-fa845c2c-f763-4b87-929d-0811e23c086c None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Aug 02 10:03:03.027409 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 02 10:03:03.027409 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 02 10:03:03.027409 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 02 10:03:03.027409 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 02 10:03:03.027409 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 02 10:03:03.027409 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 02 10:03:03.027409 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 02 10:03:03.027409 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 02 10:03:03.027409 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 02 10:03:03.027409 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 02 10:03:03.027409 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 02 10:03:03.027409 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 02 10:03:03.027409 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 02 10:03:03.027409 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 02 10:03:03.027409 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 02 10:03:03.029128 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 02 10:03:03.029128 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 02 10:03:03.029128 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 02 10:03:03.029128 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 02 10:03:03.029128 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 02 10:03:03.029128 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 02 10:03:03.029128 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 02 10:03:03.029128 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 02 10:03:03.029128 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 02 10:03:03.029128 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application return self.get_or_create( Aug 02 10:03:03.029128 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 02 10:03:03.029128 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application with Lock( Aug 02 10:03:03.029128 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 02 10:03:03.029128 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application return self._enter() Aug 02 10:03:03.029128 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 02 10:03:03.029128 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 02 10:03:03.030684 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 02 10:03:03.030684 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application return self.creator() Aug 02 10:03:03.030684 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 02 10:03:03.030684 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application created_value = creator( Aug 02 10:03:03.030684 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 02 10:03:03.030684 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 02 10:03:03.030684 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 02 10:03:03.030684 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 02 10:03:03.030684 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 02 10:03:03.030684 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 02 10:03:03.030684 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Aug 02 10:03:03.030684 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application Aug 02 10:03:03.030684 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 424/861] 10.222.0.82 () {56 vars in 1102 bytes} [Tue Aug 2 10:03:02 2022] GET /identity/v3/roles/service => generated 85 bytes in 257 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 02 10:03:03.038914 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa845c2c-f763-4b87-929d-0811e23c086c None admin] Authenticating user token {{(pid=80378) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:03:03.064932 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG dbcounter [-] [80378] Writing DB stats keystone:SELECT=100 {{(pid=80378) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 10:03:03.101434 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG dbcounter [-] [80377] Writing DB stats keystone:SELECT=100 {{(pid=80377) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 10:03:03.187878 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa845c2c-f763-4b87-929d-0811e23c086c None admin] Validating token access rules against request {{(pid=80378) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:03:03.215431 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dcf32e5b-d189-4f37-95a1-1b3ee1acf2bc None None] Validating token access rules against request {{(pid=80377) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:03:03.236620 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7df71856-0da1-4b4a-a45c-6e23d75c4f51 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '891005fa5ff74041b75747d16cc66798', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80378) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:03:03.237305 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7df71856-0da1-4b4a-a45c-6e23d75c4f51 None admin] REQUEST_METHOD: `GET` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:03:03.237625 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7df71856-0da1-4b4a-a45c-6e23d75c4f51 None admin] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:03:03.237936 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7df71856-0da1-4b4a-a45c-6e23d75c4f51 None admin] PATH_INFO: `/v3/roles` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:03:03.238854 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7df71856-0da1-4b4a-a45c-6e23d75c4f51 None admin] RBAC: Adding query filter params (name=service) {{(pid=80378) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 02 10:03:03.239353 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7df71856-0da1-4b4a-a45c-6e23d75c4f51 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:03:03.241287 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7df71856-0da1-4b4a-a45c-6e23d75c4f51 None admin] RBAC: Authorization granted {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:03:03.252519 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 425/862] 10.222.0.82 () {56 vars in 1103 bytes} [Tue Aug 2 10:03:03 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 214 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 02 10:03:03.259523 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7df71856-0da1-4b4a-a45c-6e23d75c4f51 None admin] Authenticating user token {{(pid=80378) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:03:03.286979 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5334915d-ae35-4072-a3c9-24c404d5497c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '891005fa5ff74041b75747d16cc66798', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80377) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:03:03.288457 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5334915d-ae35-4072-a3c9-24c404d5497c None admin] REQUEST_METHOD: `POST` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:03:03.289386 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5334915d-ae35-4072-a3c9-24c404d5497c None admin] SCRIPT_NAME: `/identity` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:03:03.290127 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5334915d-ae35-4072-a3c9-24c404d5497c None admin] PATH_INFO: `/v3/services` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:03:03.291588 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5334915d-ae35-4072-a3c9-24c404d5497c None admin] RBAC: Authorizing `identity:create_service()` {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:03:03.294354 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5334915d-ae35-4072-a3c9-24c404d5497c None admin] RBAC: Authorization granted {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:03:03.332612 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: [pid: 80377|app: 0|req: 438/863] 10.222.0.82 () {60 vars in 1141 bytes} [Tue Aug 2 10:03:03 2022] POST /identity/v3/services => generated 251 bytes in 310 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 02 10:03:03.371116 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG dbcounter [-] [80378] Writing DB stats keystone:SELECT=100 {{(pid=80378) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 10:03:03.434254 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7df71856-0da1-4b4a-a45c-6e23d75c4f51 None admin] Validating token access rules against request {{(pid=80378) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:03:03.497028 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af1a0403-7d6d-462c-bf1b-ae74cef8d720 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '891005fa5ff74041b75747d16cc66798', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80378) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:03:03.498231 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af1a0403-7d6d-462c-bf1b-ae74cef8d720 None admin] REQUEST_METHOD: `GET` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:03:03.498742 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af1a0403-7d6d-462c-bf1b-ae74cef8d720 None admin] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:03:03.499212 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af1a0403-7d6d-462c-bf1b-ae74cef8d720 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:03:03.513535 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af1a0403-7d6d-462c-bf1b-ae74cef8d720 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:03:03.515890 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af1a0403-7d6d-462c-bf1b-ae74cef8d720 None admin] RBAC: Authorization granted {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:03:03.533812 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application [None req-af1a0403-7d6d-462c-bf1b-ae74cef8d720 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 02 10:03:03.533812 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 02 10:03:03.533812 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 02 10:03:03.533812 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 02 10:03:03.533812 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 02 10:03:03.533812 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 02 10:03:03.533812 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 02 10:03:03.533812 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 02 10:03:03.533812 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 02 10:03:03.533812 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 02 10:03:03.533812 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 02 10:03:03.533812 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 02 10:03:03.533812 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 02 10:03:03.533812 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 02 10:03:03.533812 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 02 10:03:03.535195 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 02 10:03:03.535195 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 02 10:03:03.535195 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 02 10:03:03.535195 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 02 10:03:03.535195 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 02 10:03:03.535195 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 02 10:03:03.535195 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application return self.get_or_create( Aug 02 10:03:03.535195 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 02 10:03:03.535195 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application with Lock( Aug 02 10:03:03.535195 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 02 10:03:03.535195 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application return self._enter() Aug 02 10:03:03.535195 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 02 10:03:03.535195 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 02 10:03:03.535195 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 02 10:03:03.535195 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application return self.creator() Aug 02 10:03:03.535195 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 02 10:03:03.535195 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application created_value = creator( Aug 02 10:03:03.536176 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 02 10:03:03.536176 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 02 10:03:03.536176 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 02 10:03:03.536176 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application Aug 02 10:03:03.538657 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 426/864] 10.222.0.82 () {56 vars in 1108 bytes} [Tue Aug 2 10:03:03 2022] GET /identity/v3/domains/Default => generated 87 bytes in 280 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 02 10:03:03.546561 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5334915d-ae35-4072-a3c9-24c404d5497c None admin] Authenticating user token {{(pid=80377) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:03:03.708949 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG dbcounter [-] [80377] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80377) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 10:03:03.712687 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5334915d-ae35-4072-a3c9-24c404d5497c None admin] Validating token access rules against request {{(pid=80377) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:03:03.761300 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77271598-e81d-40c6-afd2-1d7db0845c87 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '891005fa5ff74041b75747d16cc66798', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80377) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:03:03.762654 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77271598-e81d-40c6-afd2-1d7db0845c87 None admin] REQUEST_METHOD: `GET` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:03:03.763412 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77271598-e81d-40c6-afd2-1d7db0845c87 None admin] SCRIPT_NAME: `/identity` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:03:03.764171 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77271598-e81d-40c6-afd2-1d7db0845c87 None admin] PATH_INFO: `/v3/domains` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:03:03.765614 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77271598-e81d-40c6-afd2-1d7db0845c87 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80377) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 02 10:03:03.766487 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77271598-e81d-40c6-afd2-1d7db0845c87 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:03:03.769291 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77271598-e81d-40c6-afd2-1d7db0845c87 None admin] RBAC: Authorization granted {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:03:03.791513 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: [pid: 80377|app: 0|req: 439/865] 10.222.0.82 () {56 vars in 1109 bytes} [Tue Aug 2 10:03:03 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 246 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 02 10:03:03.797909 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af1a0403-7d6d-462c-bf1b-ae74cef8d720 None admin] Authenticating user token {{(pid=80378) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:03:03.995423 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af1a0403-7d6d-462c-bf1b-ae74cef8d720 None admin] Validating token access rules against request {{(pid=80378) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:03:04.019624 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG dbcounter [-] [80378] Writing DB stats keystone:SELECT=100 {{(pid=80378) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 10:03:04.051654 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f67cbc38-a47e-4f8c-8fe2-e26e923e73e5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '891005fa5ff74041b75747d16cc66798', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80378) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:03:04.052562 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f67cbc38-a47e-4f8c-8fe2-e26e923e73e5 None admin] REQUEST_METHOD: `GET` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:03:04.052906 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f67cbc38-a47e-4f8c-8fe2-e26e923e73e5 None admin] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:03:04.053223 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f67cbc38-a47e-4f8c-8fe2-e26e923e73e5 None admin] PATH_INFO: `/v3/users/glance-swift` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:03:04.068397 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f67cbc38-a47e-4f8c-8fe2-e26e923e73e5 None admin] RBAC: Authorizing `identity:get_user(user_id=glance-swift)` {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:03:04.070171 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f67cbc38-a47e-4f8c-8fe2-e26e923e73e5 None admin] RBAC: Authorization granted {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:03:04.086005 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application [None req-f67cbc38-a47e-4f8c-8fe2-e26e923e73e5 None admin] Could not find user: glance-swift.: keystone.exception.UserNotFound: Could not find user: glance-swift. Aug 02 10:03:04.086005 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 02 10:03:04.086005 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 02 10:03:04.086005 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 02 10:03:04.086005 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 02 10:03:04.086005 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 02 10:03:04.086005 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 02 10:03:04.086005 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 02 10:03:04.086005 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 02 10:03:04.086005 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 02 10:03:04.086005 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 02 10:03:04.086005 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 02 10:03:04.086005 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 02 10:03:04.086005 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 02 10:03:04.086005 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 02 10:03:04.086005 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 02 10:03:04.087617 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 02 10:03:04.087617 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 02 10:03:04.087617 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 02 10:03:04.087617 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 02 10:03:04.087617 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 02 10:03:04.087617 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 02 10:03:04.087617 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 02 10:03:04.087617 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 02 10:03:04.087617 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 02 10:03:04.087617 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application return self.get_or_create( Aug 02 10:03:04.087617 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 02 10:03:04.087617 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application with Lock( Aug 02 10:03:04.087617 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 02 10:03:04.087617 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application return self._enter() Aug 02 10:03:04.087617 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 02 10:03:04.087617 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 02 10:03:04.087617 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 02 10:03:04.087617 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application return self.creator() Aug 02 10:03:04.089269 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 02 10:03:04.089269 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application created_value = creator( Aug 02 10:03:04.089269 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 02 10:03:04.089269 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 02 10:03:04.089269 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 02 10:03:04.089269 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 02 10:03:04.089269 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 02 10:03:04.089269 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 02 10:03:04.089269 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance-swift. Aug 02 10:03:04.089269 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application Aug 02 10:03:04.089269 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 427/866] 10.222.0.82 () {56 vars in 1117 bytes} [Tue Aug 2 10:03:03 2022] GET /identity/v3/users/glance-swift => generated 90 bytes in 290 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 02 10:03:04.099529 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77271598-e81d-40c6-afd2-1d7db0845c87 None admin] Authenticating user token {{(pid=80377) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:03:04.259153 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77271598-e81d-40c6-afd2-1d7db0845c87 None admin] Validating token access rules against request {{(pid=80377) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:03:04.307596 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69a4677c-9a8b-48d5-a254-46b8c5354763 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '891005fa5ff74041b75747d16cc66798', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80377) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:03:04.308722 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69a4677c-9a8b-48d5-a254-46b8c5354763 None admin] REQUEST_METHOD: `GET` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:03:04.309280 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69a4677c-9a8b-48d5-a254-46b8c5354763 None admin] SCRIPT_NAME: `/identity` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:03:04.310098 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69a4677c-9a8b-48d5-a254-46b8c5354763 None admin] PATH_INFO: `/v3/users` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:03:04.311917 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69a4677c-9a8b-48d5-a254-46b8c5354763 None admin] RBAC: Adding query filter params (domain_id=default, name=glance-swift) {{(pid=80377) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 02 10:03:04.312761 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69a4677c-9a8b-48d5-a254-46b8c5354763 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:03:04.318919 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69a4677c-9a8b-48d5-a254-46b8c5354763 None admin] RBAC: Authorization granted {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:03:04.356290 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: [pid: 80377|app: 0|req: 440/867] 10.222.0.82 () {56 vars in 1149 bytes} [Tue Aug 2 10:03:04 2022] GET /identity/v3/users?domain_id=default&name=glance-swift => generated 384 bytes in 257 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 02 10:03:04.363172 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f67cbc38-a47e-4f8c-8fe2-e26e923e73e5 None admin] Authenticating user token {{(pid=80378) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:03:04.552018 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f67cbc38-a47e-4f8c-8fe2-e26e923e73e5 None admin] Validating token access rules against request {{(pid=80378) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:03:04.613233 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c5d2780-f861-4be3-bd56-7391ba60d09f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '891005fa5ff74041b75747d16cc66798', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80378) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:03:04.614095 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c5d2780-f861-4be3-bd56-7391ba60d09f None admin] REQUEST_METHOD: `GET` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:03:04.614442 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c5d2780-f861-4be3-bd56-7391ba60d09f None admin] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:03:04.614764 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c5d2780-f861-4be3-bd56-7391ba60d09f None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:03:04.679940 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG dbcounter [-] [80378] Writing DB stats keystone:SELECT=100 {{(pid=80378) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 10:03:04.722634 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c5d2780-f861-4be3-bd56-7391ba60d09f None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:03:04.725522 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c5d2780-f861-4be3-bd56-7391ba60d09f None admin] RBAC: Authorization granted {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:03:04.898670 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 428/868] 10.222.0.82 () {58 vars in 1282 bytes} [Tue Aug 2 10:03:04 2022] GET /identity/v3/auth/tokens => generated 1262 bytes in 536 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Aug 02 10:03:04.907222 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69a4677c-9a8b-48d5-a254-46b8c5354763 None admin] Authenticating user token {{(pid=80377) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:03:04.947801 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG dbcounter [-] [80377] Writing DB stats keystone:SELECT=100 {{(pid=80377) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 10:03:05.060060 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69a4677c-9a8b-48d5-a254-46b8c5354763 None admin] Validating token access rules against request {{(pid=80377) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:03:05.114104 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31fd1ecd-3b1a-4641-a314-93a5572277ed None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '891005fa5ff74041b75747d16cc66798', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80377) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:03:05.114678 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31fd1ecd-3b1a-4641-a314-93a5572277ed None admin] REQUEST_METHOD: `GET` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:03:05.114968 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31fd1ecd-3b1a-4641-a314-93a5572277ed None admin] SCRIPT_NAME: `/identity` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:03:05.115249 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31fd1ecd-3b1a-4641-a314-93a5572277ed None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:03:05.133359 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31fd1ecd-3b1a-4641-a314-93a5572277ed None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:03:05.134919 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31fd1ecd-3b1a-4641-a314-93a5572277ed None admin] RBAC: Authorization granted {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:03:05.157768 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application [None req-31fd1ecd-3b1a-4641-a314-93a5572277ed None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 02 10:03:05.157768 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 02 10:03:05.157768 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 02 10:03:05.157768 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 02 10:03:05.157768 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 02 10:03:05.157768 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 02 10:03:05.157768 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 02 10:03:05.157768 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 02 10:03:05.157768 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 02 10:03:05.157768 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 02 10:03:05.157768 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 02 10:03:05.157768 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 02 10:03:05.157768 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 02 10:03:05.157768 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 02 10:03:05.157768 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 02 10:03:05.161163 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 02 10:03:05.161163 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 02 10:03:05.161163 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 02 10:03:05.161163 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 02 10:03:05.161163 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 02 10:03:05.161163 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 02 10:03:05.161163 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application return self.get_or_create( Aug 02 10:03:05.161163 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 02 10:03:05.161163 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application with Lock( Aug 02 10:03:05.161163 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 02 10:03:05.161163 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application return self._enter() Aug 02 10:03:05.161163 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 02 10:03:05.161163 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 02 10:03:05.161163 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 02 10:03:05.161163 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application return self.creator() Aug 02 10:03:05.161163 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 02 10:03:05.161163 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application created_value = creator( Aug 02 10:03:05.164882 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 02 10:03:05.164882 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 02 10:03:05.164882 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 02 10:03:05.164882 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application Aug 02 10:03:05.167468 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: [pid: 80377|app: 0|req: 441/869] 10.222.0.82 () {56 vars in 1108 bytes} [Tue Aug 2 10:03:04 2022] GET /identity/v3/domains/Default => generated 87 bytes in 260 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 02 10:03:05.169828 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c5d2780-f861-4be3-bd56-7391ba60d09f None admin] Authenticating user token {{(pid=80378) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:03:05.184799 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cb03837-a5e9-41eb-a9d1-1a9e019e859a None None] REQUEST_METHOD: `GET` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:03:05.185499 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cb03837-a5e9-41eb-a9d1-1a9e019e859a None None] SCRIPT_NAME: `/identity` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:03:05.186027 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cb03837-a5e9-41eb-a9d1-1a9e019e859a None None] PATH_INFO: `/` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:03:05.189418 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: [pid: 80377|app: 0|req: 442/870] 10.222.0.82 () {54 vars in 926 bytes} [Tue Aug 2 10:03:05 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 02 10:03:05.194736 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cfc49bb-3e24-4f42-a184-44122d046220 None None] REQUEST_METHOD: `POST` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:03:05.195592 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cfc49bb-3e24-4f42-a184-44122d046220 None None] SCRIPT_NAME: `/identity` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:03:05.196150 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cfc49bb-3e24-4f42-a184-44122d046220 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:03:05.217394 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.auth.core [None req-4cfc49bb-3e24-4f42-a184-44122d046220 None None] MFA Rules not processed for user `891005fa5ff74041b75747d16cc66798`. Rule list: `[]` (Enabled: `True`). {{(pid=80377) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 02 10:03:05.218942 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.fernet_utils [None req-4cfc49bb-3e24-4f42-a184-44122d046220 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80377) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 10:03:05.283561 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG dbcounter [-] [80378] Writing DB stats keystone:SELECT=100 {{(pid=80378) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 10:03:05.321162 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG dbcounter [-] [80377] Writing DB stats keystone:SELECT=100 {{(pid=80377) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 10:03:05.332780 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: [pid: 80377|app: 0|req: 443/871] 10.222.0.82 () {58 vars in 1023 bytes} [Tue Aug 2 10:03:05 2022] POST /identity/v3/auth/tokens => generated 1262 bytes in 138 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Aug 02 10:03:05.350775 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c5d2780-f861-4be3-bd56-7391ba60d09f None admin] Validating token access rules against request {{(pid=80378) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:03:05.362485 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4cfc49bb-3e24-4f42-a184-44122d046220 None None] Authenticating user token {{(pid=80377) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:03:05.425756 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4a70323-dfbf-49f0-8869-446a7d31f62e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '891005fa5ff74041b75747d16cc66798', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80378) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:03:05.427133 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4a70323-dfbf-49f0-8869-446a7d31f62e None admin] REQUEST_METHOD: `GET` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:03:05.427628 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4a70323-dfbf-49f0-8869-446a7d31f62e None admin] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:03:05.428113 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4a70323-dfbf-49f0-8869-446a7d31f62e None admin] PATH_INFO: `/v3/domains` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:03:05.429046 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4a70323-dfbf-49f0-8869-446a7d31f62e None admin] RBAC: Adding query filter params (name=Default) {{(pid=80378) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 02 10:03:05.430440 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4a70323-dfbf-49f0-8869-446a7d31f62e None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:03:05.432063 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4a70323-dfbf-49f0-8869-446a7d31f62e None admin] RBAC: Authorization granted {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:03:05.458328 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 429/872] 10.222.0.82 () {56 vars in 1109 bytes} [Tue Aug 2 10:03:05 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 290 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 02 10:03:05.464668 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4a70323-dfbf-49f0-8869-446a7d31f62e None admin] Authenticating user token {{(pid=80378) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:03:05.533735 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4cfc49bb-3e24-4f42-a184-44122d046220 None None] Validating token access rules against request {{(pid=80377) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:03:05.591283 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e292906b-848c-4e80-88c8-6de4c493e5f8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '891005fa5ff74041b75747d16cc66798', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80377) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:03:05.591868 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e292906b-848c-4e80-88c8-6de4c493e5f8 None admin] REQUEST_METHOD: `GET` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:03:05.592290 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e292906b-848c-4e80-88c8-6de4c493e5f8 None admin] SCRIPT_NAME: `/identity` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:03:05.592439 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e292906b-848c-4e80-88c8-6de4c493e5f8 None admin] PATH_INFO: `/v3/services/object-store` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:03:05.593043 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e292906b-848c-4e80-88c8-6de4c493e5f8 None admin] RBAC: Authorizing `identity:get_service(service_id=object-store)` {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:03:05.594483 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e292906b-848c-4e80-88c8-6de4c493e5f8 None admin] RBAC: Authorization granted {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:03:05.614340 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application [None req-e292906b-848c-4e80-88c8-6de4c493e5f8 None admin] Could not find service: object-store.: keystone.exception.ServiceNotFound: Could not find service: object-store. Aug 02 10:03:05.614340 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 02 10:03:05.614340 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Aug 02 10:03:05.614340 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Aug 02 10:03:05.614340 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Aug 02 10:03:05.614340 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Aug 02 10:03:05.614340 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Aug 02 10:03:05.614340 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 02 10:03:05.614340 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Aug 02 10:03:05.614340 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application Aug 02 10:03:05.614340 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 02 10:03:05.614340 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application Aug 02 10:03:05.614340 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 02 10:03:05.614340 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 02 10:03:05.614340 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 02 10:03:05.614340 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 02 10:03:05.614340 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 02 10:03:05.615570 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 02 10:03:05.615570 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 02 10:03:05.615570 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 02 10:03:05.615570 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 02 10:03:05.615570 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 02 10:03:05.615570 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 02 10:03:05.615570 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Aug 02 10:03:05.615570 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application return self._get_service(service_id) Aug 02 10:03:05.615570 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Aug 02 10:03:05.615570 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Aug 02 10:03:05.615570 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 02 10:03:05.615570 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 02 10:03:05.615570 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 02 10:03:05.615570 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 02 10:03:05.615570 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 02 10:03:05.615570 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application return self.get_or_create( Aug 02 10:03:05.616517 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 02 10:03:05.616517 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application with Lock( Aug 02 10:03:05.616517 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 02 10:03:05.616517 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application return self._enter() Aug 02 10:03:05.616517 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 02 10:03:05.616517 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 02 10:03:05.616517 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 02 10:03:05.616517 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application return self.creator() Aug 02 10:03:05.616517 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 02 10:03:05.616517 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application created_value = creator( Aug 02 10:03:05.616517 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Aug 02 10:03:05.616517 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 02 10:03:05.616517 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Aug 02 10:03:05.616517 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application Aug 02 10:03:05.616517 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: [pid: 80377|app: 0|req: 444/873] 10.222.0.82 () {56 vars in 1126 bytes} [Tue Aug 2 10:03:05 2022] GET /identity/v3/services/object-store => generated 93 bytes in 254 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 02 10:03:05.618740 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58d05232-5643-4645-8dbd-9329d3fc902d None None] REQUEST_METHOD: `GET` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:03:05.618740 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58d05232-5643-4645-8dbd-9329d3fc902d None None] SCRIPT_NAME: `/identity` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:03:05.618740 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58d05232-5643-4645-8dbd-9329d3fc902d None None] PATH_INFO: `/` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:03:05.619782 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: [pid: 80377|app: 0|req: 445/874] 10.222.0.82 () {54 vars in 926 bytes} [Tue Aug 2 10:03:05 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 02 10:03:05.626261 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f432271-910e-455f-b3d9-d7632b9bae6a None None] REQUEST_METHOD: `POST` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:03:05.626261 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f432271-910e-455f-b3d9-d7632b9bae6a None None] SCRIPT_NAME: `/identity` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:03:05.626261 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f432271-910e-455f-b3d9-d7632b9bae6a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:03:05.648959 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4a70323-dfbf-49f0-8869-446a7d31f62e None admin] Validating token access rules against request {{(pid=80378) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:03:05.649538 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.auth.core [None req-4f432271-910e-455f-b3d9-d7632b9bae6a None None] MFA Rules not processed for user `891005fa5ff74041b75747d16cc66798`. Rule list: `[]` (Enabled: `True`). {{(pid=80377) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 02 10:03:05.651535 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.fernet_utils [None req-4f432271-910e-455f-b3d9-d7632b9bae6a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80377) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 10:03:05.694884 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG dbcounter [-] [80378] Writing DB stats keystone:SELECT=100 {{(pid=80378) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 10:03:05.705225 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ec674c6-bec1-46b0-b5a1-78850e70f5ab None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '891005fa5ff74041b75747d16cc66798', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80378) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:03:05.706733 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ec674c6-bec1-46b0-b5a1-78850e70f5ab None admin] REQUEST_METHOD: `GET` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:03:05.708531 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ec674c6-bec1-46b0-b5a1-78850e70f5ab None admin] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:03:05.709002 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ec674c6-bec1-46b0-b5a1-78850e70f5ab None admin] PATH_INFO: `/v3/projects/service` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:03:05.723081 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ec674c6-bec1-46b0-b5a1-78850e70f5ab None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:03:05.725925 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ec674c6-bec1-46b0-b5a1-78850e70f5ab None admin] RBAC: Authorization granted {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:03:05.736789 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG dbcounter [-] [80377] Writing DB stats keystone:SELECT=100 {{(pid=80377) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 10:03:05.745664 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application [None req-7ec674c6-bec1-46b0-b5a1-78850e70f5ab None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 02 10:03:05.745664 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 02 10:03:05.745664 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 02 10:03:05.745664 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 02 10:03:05.745664 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 02 10:03:05.745664 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 02 10:03:05.745664 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 02 10:03:05.745664 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 02 10:03:05.745664 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 02 10:03:05.745664 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 02 10:03:05.745664 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 02 10:03:05.745664 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 02 10:03:05.745664 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 02 10:03:05.745664 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 02 10:03:05.745664 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 02 10:03:05.746759 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 02 10:03:05.746759 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 02 10:03:05.746759 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 02 10:03:05.746759 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 02 10:03:05.746759 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 02 10:03:05.746759 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 02 10:03:05.746759 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application return self.get_or_create( Aug 02 10:03:05.746759 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 02 10:03:05.746759 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application with Lock( Aug 02 10:03:05.746759 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 02 10:03:05.746759 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application return self._enter() Aug 02 10:03:05.746759 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 02 10:03:05.746759 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 02 10:03:05.746759 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 02 10:03:05.746759 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application return self.creator() Aug 02 10:03:05.746759 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 02 10:03:05.746759 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application created_value = creator( Aug 02 10:03:05.750652 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 02 10:03:05.750652 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 02 10:03:05.750652 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 02 10:03:05.750652 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 02 10:03:05.750652 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 02 10:03:05.750652 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 02 10:03:05.750652 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 02 10:03:05.750652 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application Aug 02 10:03:05.750652 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 430/875] 10.222.0.82 () {56 vars in 1111 bytes} [Tue Aug 2 10:03:05 2022] GET /identity/v3/projects/service => generated 88 bytes in 283 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 02 10:03:05.750652 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ec674c6-bec1-46b0-b5a1-78850e70f5ab None admin] Authenticating user token {{(pid=80378) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:03:05.759270 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: [pid: 80377|app: 0|req: 446/876] 10.222.0.82 () {58 vars in 1023 bytes} [Tue Aug 2 10:03:05 2022] POST /identity/v3/auth/tokens => generated 1262 bytes in 135 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Aug 02 10:03:05.760133 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f432271-910e-455f-b3d9-d7632b9bae6a None None] Authenticating user token {{(pid=80377) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:03:05.906643 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ec674c6-bec1-46b0-b5a1-78850e70f5ab None admin] Validating token access rules against request {{(pid=80378) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:03:05.941894 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f432271-910e-455f-b3d9-d7632b9bae6a None None] Validating token access rules against request {{(pid=80377) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:03:05.964142 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08876d18-0f28-4438-afc7-31ddc9b446af None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '891005fa5ff74041b75747d16cc66798', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80378) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:03:05.964951 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08876d18-0f28-4438-afc7-31ddc9b446af None admin] REQUEST_METHOD: `GET` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:03:05.965236 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08876d18-0f28-4438-afc7-31ddc9b446af None admin] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:03:05.965534 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08876d18-0f28-4438-afc7-31ddc9b446af None admin] PATH_INFO: `/v3/services` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:03:05.966260 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08876d18-0f28-4438-afc7-31ddc9b446af None admin] RBAC: Adding query filter params (name=object-store) {{(pid=80378) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 02 10:03:05.966566 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08876d18-0f28-4438-afc7-31ddc9b446af None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:03:05.967977 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08876d18-0f28-4438-afc7-31ddc9b446af None admin] RBAC: Authorization granted {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:03:05.979072 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 431/877] 10.222.0.82 () {56 vars in 1122 bytes} [Tue Aug 2 10:03:05 2022] GET /identity/v3/services?name=object-store => generated 129 bytes in 229 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 02 10:03:05.979072 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08876d18-0f28-4438-afc7-31ddc9b446af None admin] Authenticating user token {{(pid=80378) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:03:06.008934 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa611c09-19f6-4780-945c-29c6fdaefc82 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '891005fa5ff74041b75747d16cc66798', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80377) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:03:06.009882 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa611c09-19f6-4780-945c-29c6fdaefc82 None admin] REQUEST_METHOD: `GET` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:03:06.010245 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa611c09-19f6-4780-945c-29c6fdaefc82 None admin] SCRIPT_NAME: `/identity` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:03:06.010575 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa611c09-19f6-4780-945c-29c6fdaefc82 None admin] PATH_INFO: `/v3/projects` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:03:06.011673 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa611c09-19f6-4780-945c-29c6fdaefc82 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80377) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 02 10:03:06.012146 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa611c09-19f6-4780-945c-29c6fdaefc82 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:03:06.013851 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa611c09-19f6-4780-945c-29c6fdaefc82 None admin] RBAC: Authorization granted {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:03:06.033834 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: [pid: 80377|app: 0|req: 447/878] 10.222.0.82 () {56 vars in 1148 bytes} [Tue Aug 2 10:03:05 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 272 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 02 10:03:06.033834 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa611c09-19f6-4780-945c-29c6fdaefc82 None admin] Authenticating user token {{(pid=80377) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:03:06.076420 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG dbcounter [-] [80378] Writing DB stats keystone:SELECT=100 {{(pid=80378) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 10:03:06.108584 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG dbcounter [-] [80377] Writing DB stats keystone:SELECT=100 {{(pid=80377) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 10:03:06.147346 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08876d18-0f28-4438-afc7-31ddc9b446af None admin] Validating token access rules against request {{(pid=80378) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:03:06.184358 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa611c09-19f6-4780-945c-29c6fdaefc82 None admin] Validating token access rules against request {{(pid=80377) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:03:06.216762 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-225b53fd-fc15-4d56-bdab-59c43fd3af68 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '891005fa5ff74041b75747d16cc66798', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80378) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:03:06.217814 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-225b53fd-fc15-4d56-bdab-59c43fd3af68 None admin] REQUEST_METHOD: `POST` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:03:06.218289 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-225b53fd-fc15-4d56-bdab-59c43fd3af68 None admin] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:03:06.218758 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-225b53fd-fc15-4d56-bdab-59c43fd3af68 None admin] PATH_INFO: `/v3/services` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:03:06.219609 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-225b53fd-fc15-4d56-bdab-59c43fd3af68 None admin] RBAC: Authorizing `identity:create_service()` {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:03:06.221585 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-225b53fd-fc15-4d56-bdab-59c43fd3af68 None admin] RBAC: Authorization granted {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:03:06.239278 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4a541c4-54a0-4926-b7cd-ca94ecd12f4f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '891005fa5ff74041b75747d16cc66798', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80377) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:03:06.239994 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 432/879] 10.222.0.82 () {60 vars in 1141 bytes} [Tue Aug 2 10:03:05 2022] POST /identity/v3/services => generated 269 bytes in 262 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 02 10:03:06.240209 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4a541c4-54a0-4926-b7cd-ca94ecd12f4f None admin] REQUEST_METHOD: `GET` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:03:06.240358 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4a541c4-54a0-4926-b7cd-ca94ecd12f4f None admin] SCRIPT_NAME: `/identity` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:03:06.240680 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4a541c4-54a0-4926-b7cd-ca94ecd12f4f None admin] PATH_INFO: `/v3/services` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:03:06.240998 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-225b53fd-fc15-4d56-bdab-59c43fd3af68 None admin] Authenticating user token {{(pid=80378) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:03:06.241526 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4a541c4-54a0-4926-b7cd-ca94ecd12f4f None admin] RBAC: Adding query filter params (type=object-store) {{(pid=80377) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 02 10:03:06.241694 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4a541c4-54a0-4926-b7cd-ca94ecd12f4f None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:03:06.243179 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4a541c4-54a0-4926-b7cd-ca94ecd12f4f None admin] RBAC: Authorization granted {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:03:06.255681 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: [pid: 80377|app: 0|req: 448/880] 10.222.0.82 () {56 vars in 1122 bytes} [Tue Aug 2 10:03:06 2022] GET /identity/v3/services?type=object-store => generated 129 bytes in 223 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 02 10:03:06.258614 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-119ce6e0-aa37-47ad-9b1d-b40730a78ae1 None None] REQUEST_METHOD: `GET` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:03:06.259420 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-119ce6e0-aa37-47ad-9b1d-b40730a78ae1 None None] SCRIPT_NAME: `/identity` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:03:06.260161 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-119ce6e0-aa37-47ad-9b1d-b40730a78ae1 None None] PATH_INFO: `/` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:03:06.272057 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: [pid: 80377|app: 0|req: 449/881] 10.222.0.82 () {54 vars in 926 bytes} [Tue Aug 2 10:03:06 2022] GET /identity => generated 270 bytes in 10 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 02 10:03:06.280185 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e54ac5a-4cba-4d20-9f05-39caa4a16031 None None] REQUEST_METHOD: `POST` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:03:06.280702 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e54ac5a-4cba-4d20-9f05-39caa4a16031 None None] SCRIPT_NAME: `/identity` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:03:06.281130 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e54ac5a-4cba-4d20-9f05-39caa4a16031 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:03:06.332864 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.auth.core [None req-4e54ac5a-4cba-4d20-9f05-39caa4a16031 None None] MFA Rules not processed for user `891005fa5ff74041b75747d16cc66798`. Rule list: `[]` (Enabled: `True`). {{(pid=80377) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 02 10:03:06.335039 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.fernet_utils [None req-4e54ac5a-4cba-4d20-9f05-39caa4a16031 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80377) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 10:03:06.479903 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-225b53fd-fc15-4d56-bdab-59c43fd3af68 None admin] Validating token access rules against request {{(pid=80378) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:03:06.501032 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG dbcounter [-] [80378] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80378) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 10:03:06.503193 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: [pid: 80377|app: 0|req: 450/882] 10.222.0.82 () {58 vars in 1023 bytes} [Tue Aug 2 10:03:06 2022] POST /identity/v3/auth/tokens => generated 1372 bytes in 224 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Aug 02 10:03:06.531799 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-418d94f7-a8a3-4356-b51c-b848300c6728 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '891005fa5ff74041b75747d16cc66798', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80378) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:03:06.532483 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-418d94f7-a8a3-4356-b51c-b848300c6728 None admin] REQUEST_METHOD: `GET` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:03:06.532875 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-418d94f7-a8a3-4356-b51c-b848300c6728 None admin] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:03:06.533194 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-418d94f7-a8a3-4356-b51c-b848300c6728 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:03:06.534550 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-418d94f7-a8a3-4356-b51c-b848300c6728 None admin] RBAC: Adding query filter params (role.id=02003a0be60f47f2a8dfea17053af52c, scope.project.id=f1b38ebb804c41c69772c998aae72c41, user.id=bd8dd72086db48b8aaac2ed0058872df) {{(pid=80378) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 02 10:03:06.534906 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-418d94f7-a8a3-4356-b51c-b848300c6728 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:03:06.537408 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-418d94f7-a8a3-4356-b51c-b848300c6728 None admin] RBAC: Authorization granted {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:03:06.543403 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e54ac5a-4cba-4d20-9f05-39caa4a16031 None None] Authenticating user token {{(pid=80377) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:03:06.548149 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 433/883] 10.222.0.82 () {56 vars in 1374 bytes} [Tue Aug 2 10:03:06 2022] GET /identity/v3/role_assignments?user.id=bd8dd72086db48b8aaac2ed0058872df&scope.project.id=f1b38ebb804c41c69772c998aae72c41&role.id=02003a0be60f47f2a8dfea17053af52c => generated 608 bytes in 307 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 02 10:03:06.636556 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG dbcounter [-] [80377] Writing DB stats keystone:SELECT=100 {{(pid=80377) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 10:03:06.724262 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e54ac5a-4cba-4d20-9f05-39caa4a16031 None None] Validating token access rules against request {{(pid=80377) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:03:06.769461 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87723009-6568-4963-8409-c7c6e92cc165 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '891005fa5ff74041b75747d16cc66798', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80377) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:03:06.770068 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87723009-6568-4963-8409-c7c6e92cc165 None admin] REQUEST_METHOD: `GET` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:03:06.770356 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87723009-6568-4963-8409-c7c6e92cc165 None admin] SCRIPT_NAME: `/identity` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:03:06.770635 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87723009-6568-4963-8409-c7c6e92cc165 None admin] PATH_INFO: `/v3/services/volumev3` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:03:06.771291 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87723009-6568-4963-8409-c7c6e92cc165 None admin] RBAC: Authorizing `identity:get_service(service_id=volumev3)` {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:03:06.772733 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87723009-6568-4963-8409-c7c6e92cc165 None admin] RBAC: Authorization granted {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:03:06.782944 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application [None req-87723009-6568-4963-8409-c7c6e92cc165 None admin] Could not find service: volumev3.: keystone.exception.ServiceNotFound: Could not find service: volumev3. Aug 02 10:03:06.782944 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 02 10:03:06.782944 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Aug 02 10:03:06.782944 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Aug 02 10:03:06.782944 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Aug 02 10:03:06.782944 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Aug 02 10:03:06.782944 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Aug 02 10:03:06.782944 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 02 10:03:06.782944 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: volumev3. Aug 02 10:03:06.782944 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application Aug 02 10:03:06.782944 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 02 10:03:06.782944 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application Aug 02 10:03:06.782944 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 02 10:03:06.782944 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 02 10:03:06.782944 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 02 10:03:06.782944 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 02 10:03:06.782944 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 02 10:03:06.784574 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 02 10:03:06.784574 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 02 10:03:06.784574 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 02 10:03:06.784574 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 02 10:03:06.784574 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 02 10:03:06.784574 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 02 10:03:06.784574 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Aug 02 10:03:06.784574 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application return self._get_service(service_id) Aug 02 10:03:06.784574 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Aug 02 10:03:06.784574 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Aug 02 10:03:06.784574 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 02 10:03:06.784574 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 02 10:03:06.784574 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 02 10:03:06.784574 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 02 10:03:06.784574 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 02 10:03:06.784574 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application return self.get_or_create( Aug 02 10:03:06.785442 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 02 10:03:06.785442 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application with Lock( Aug 02 10:03:06.785442 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 02 10:03:06.785442 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application return self._enter() Aug 02 10:03:06.785442 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 02 10:03:06.785442 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 02 10:03:06.785442 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 02 10:03:06.785442 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application return self.creator() Aug 02 10:03:06.785442 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 02 10:03:06.785442 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application created_value = creator( Aug 02 10:03:06.785442 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Aug 02 10:03:06.785442 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 02 10:03:06.785442 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: volumev3. Aug 02 10:03:06.785442 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application Aug 02 10:03:06.785442 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: [pid: 80377|app: 0|req: 451/884] 10.222.0.82 () {56 vars in 1114 bytes} [Tue Aug 2 10:03:06 2022] GET /identity/v3/services/volumev3 => generated 89 bytes in 243 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 02 10:03:06.794207 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-418d94f7-a8a3-4356-b51c-b848300c6728 None admin] Authenticating user token {{(pid=80378) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:03:06.956255 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-418d94f7-a8a3-4356-b51c-b848300c6728 None admin] Validating token access rules against request {{(pid=80378) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:03:07.024195 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8592d4f-a9dd-408c-abaf-f4fa4f62691f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '891005fa5ff74041b75747d16cc66798', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80378) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:03:07.024755 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8592d4f-a9dd-408c-abaf-f4fa4f62691f None admin] REQUEST_METHOD: `GET` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:03:07.025039 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8592d4f-a9dd-408c-abaf-f4fa4f62691f None admin] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:03:07.025309 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8592d4f-a9dd-408c-abaf-f4fa4f62691f None admin] PATH_INFO: `/v3/services` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:03:07.026139 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8592d4f-a9dd-408c-abaf-f4fa4f62691f None admin] RBAC: Adding query filter params (name=volumev3) {{(pid=80378) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 02 10:03:07.026516 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8592d4f-a9dd-408c-abaf-f4fa4f62691f None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:03:07.027933 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8592d4f-a9dd-408c-abaf-f4fa4f62691f None admin] RBAC: Authorization granted {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:03:07.034930 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 434/885] 10.222.0.82 () {56 vars in 1114 bytes} [Tue Aug 2 10:03:06 2022] GET /identity/v3/services?name=volumev3 => generated 125 bytes in 241 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 02 10:03:07.040657 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87723009-6568-4963-8409-c7c6e92cc165 None admin] Authenticating user token {{(pid=80377) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:03:07.166915 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87723009-6568-4963-8409-c7c6e92cc165 None admin] Validating token access rules against request {{(pid=80377) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:03:07.186623 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG dbcounter [-] [80377] Writing DB stats keystone:SELECT=100 {{(pid=80377) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 10:03:07.211012 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7a0d9b1-ce8d-4445-814e-b65614d1ebbd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '891005fa5ff74041b75747d16cc66798', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80377) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:03:07.211761 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7a0d9b1-ce8d-4445-814e-b65614d1ebbd None admin] REQUEST_METHOD: `GET` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:03:07.212117 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7a0d9b1-ce8d-4445-814e-b65614d1ebbd None admin] SCRIPT_NAME: `/identity` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:03:07.212399 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7a0d9b1-ce8d-4445-814e-b65614d1ebbd None admin] PATH_INFO: `/v3/services` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:03:07.213174 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7a0d9b1-ce8d-4445-814e-b65614d1ebbd None admin] RBAC: Adding query filter params (type=volumev3) {{(pid=80377) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 02 10:03:07.213487 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7a0d9b1-ce8d-4445-814e-b65614d1ebbd None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:03:07.215077 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7a0d9b1-ce8d-4445-814e-b65614d1ebbd None admin] RBAC: Authorization granted {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:03:07.222180 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: [pid: 80377|app: 0|req: 452/886] 10.222.0.82 () {56 vars in 1114 bytes} [Tue Aug 2 10:03:07 2022] GET /identity/v3/services?type=volumev3 => generated 362 bytes in 182 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 02 10:03:07.228066 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8592d4f-a9dd-408c-abaf-f4fa4f62691f None admin] Authenticating user token {{(pid=80378) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:03:07.285661 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG dbcounter [-] [80378] Writing DB stats keystone:SELECT=100 {{(pid=80378) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 10:03:07.388267 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8592d4f-a9dd-408c-abaf-f4fa4f62691f None admin] Validating token access rules against request {{(pid=80378) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:03:07.437891 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8d663e4-9dd2-42e5-91fb-eaf58bdcf5b4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '891005fa5ff74041b75747d16cc66798', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80378) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:03:07.439887 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8d663e4-9dd2-42e5-91fb-eaf58bdcf5b4 None admin] REQUEST_METHOD: `GET` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:03:07.440300 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8d663e4-9dd2-42e5-91fb-eaf58bdcf5b4 None admin] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:03:07.440711 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8d663e4-9dd2-42e5-91fb-eaf58bdcf5b4 None admin] PATH_INFO: `/v3/endpoints` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:03:07.441904 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8d663e4-9dd2-42e5-91fb-eaf58bdcf5b4 None admin] RBAC: Adding query filter params (interface=public, service_id=2a458a5944a34767a573bdc5ee31b2fa, region_id=RegionOne) {{(pid=80378) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 02 10:03:07.442376 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8d663e4-9dd2-42e5-91fb-eaf58bdcf5b4 None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:03:07.444612 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8d663e4-9dd2-42e5-91fb-eaf58bdcf5b4 None admin] RBAC: Authorization granted {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:03:07.453535 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 435/887] 10.222.0.82 () {56 vars in 1251 bytes} [Tue Aug 2 10:03:07 2022] GET /identity/v3/endpoints?service_id=2a458a5944a34767a573bdc5ee31b2fa&interface=public®ion_id=RegionOne => generated 194 bytes in 226 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 02 10:03:07.461769 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7a0d9b1-ce8d-4445-814e-b65614d1ebbd None admin] Authenticating user token {{(pid=80377) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:03:07.635967 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7a0d9b1-ce8d-4445-814e-b65614d1ebbd None admin] Validating token access rules against request {{(pid=80377) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:03:07.683794 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a70f0fb6-ee82-458a-b9a2-fc23d4723698 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '891005fa5ff74041b75747d16cc66798', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80377) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:03:07.684403 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a70f0fb6-ee82-458a-b9a2-fc23d4723698 None admin] REQUEST_METHOD: `GET` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:03:07.684673 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a70f0fb6-ee82-458a-b9a2-fc23d4723698 None admin] SCRIPT_NAME: `/identity` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:03:07.684949 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a70f0fb6-ee82-458a-b9a2-fc23d4723698 None admin] PATH_INFO: `/v3/services` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:03:07.685735 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a70f0fb6-ee82-458a-b9a2-fc23d4723698 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:03:07.687152 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a70f0fb6-ee82-458a-b9a2-fc23d4723698 None admin] RBAC: Authorization granted {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:03:07.695352 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: [pid: 80377|app: 0|req: 453/888] 10.222.0.82 () {56 vars in 1087 bytes} [Tue Aug 2 10:03:07 2022] GET /identity/v3/services => generated 1268 bytes in 234 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 02 10:03:08.977754 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9821770a-1497-45f3-b553-2a10287f3d12 None None] REQUEST_METHOD: `GET` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:03:08.978979 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9821770a-1497-45f3-b553-2a10287f3d12 None None] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:03:08.979625 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9821770a-1497-45f3-b553-2a10287f3d12 None None] PATH_INFO: `/` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:03:08.982165 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 436/889] 10.222.0.82 () {54 vars in 926 bytes} [Tue Aug 2 10:03:08 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 02 10:03:08.989610 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab6892ff-a71f-4ea6-87ba-9bf0236798ad None None] REQUEST_METHOD: `POST` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:03:08.990782 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab6892ff-a71f-4ea6-87ba-9bf0236798ad None None] SCRIPT_NAME: `/identity` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:03:08.991509 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab6892ff-a71f-4ea6-87ba-9bf0236798ad None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:03:09.013462 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.auth.core [None req-ab6892ff-a71f-4ea6-87ba-9bf0236798ad None None] MFA Rules not processed for user `891005fa5ff74041b75747d16cc66798`. Rule list: `[]` (Enabled: `True`). {{(pid=80377) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 02 10:03:09.015068 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.fernet_utils [None req-ab6892ff-a71f-4ea6-87ba-9bf0236798ad None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80377) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 10:03:09.058565 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG dbcounter [-] [80377] Writing DB stats keystone:SELECT=100 {{(pid=80377) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 10:03:09.095044 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8de7e30-b4d0-4174-8800-ce04a52e9bc7 None None] REQUEST_METHOD: `GET` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:03:09.096656 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8de7e30-b4d0-4174-8800-ce04a52e9bc7 None None] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:03:09.097450 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8de7e30-b4d0-4174-8800-ce04a52e9bc7 None None] PATH_INFO: `/` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:03:09.106737 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 437/890] 10.222.0.82 () {54 vars in 926 bytes} [Tue Aug 2 10:03:09 2022] GET /identity => generated 270 bytes in 7 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 02 10:03:09.109006 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8283c936-b7ef-4dbd-861b-a66ec79c36f9 None None] REQUEST_METHOD: `POST` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:03:09.109701 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8283c936-b7ef-4dbd-861b-a66ec79c36f9 None None] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:03:09.110397 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8283c936-b7ef-4dbd-861b-a66ec79c36f9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:03:09.121044 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: [pid: 80377|app: 0|req: 454/891] 10.222.0.82 () {58 vars in 1023 bytes} [Tue Aug 2 10:03:08 2022] POST /identity/v3/auth/tokens => generated 1372 bytes in 132 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Aug 02 10:03:09.137756 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.auth.core [None req-8283c936-b7ef-4dbd-861b-a66ec79c36f9 None None] MFA Rules not processed for user `891005fa5ff74041b75747d16cc66798`. Rule list: `[]` (Enabled: `True`). {{(pid=80378) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 02 10:03:09.139399 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.fernet_utils [None req-8283c936-b7ef-4dbd-861b-a66ec79c36f9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80378) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 10:03:09.156114 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab6892ff-a71f-4ea6-87ba-9bf0236798ad None None] Authenticating user token {{(pid=80377) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:03:09.230469 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 438/892] 10.222.0.82 () {58 vars in 1023 bytes} [Tue Aug 2 10:03:09 2022] POST /identity/v3/auth/tokens => generated 1372 bytes in 123 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 02 10:03:09.268279 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8283c936-b7ef-4dbd-861b-a66ec79c36f9 None None] Authenticating user token {{(pid=80378) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:03:09.280935 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab6892ff-a71f-4ea6-87ba-9bf0236798ad None None] Validating token access rules against request {{(pid=80377) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:03:09.330900 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9e164ca-7d7e-47cc-9b73-8c1e9d2496b0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '891005fa5ff74041b75747d16cc66798', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80377) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:03:09.331578 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9e164ca-7d7e-47cc-9b73-8c1e9d2496b0 None admin] REQUEST_METHOD: `POST` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:03:09.332084 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9e164ca-7d7e-47cc-9b73-8c1e9d2496b0 None admin] SCRIPT_NAME: `/identity` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:03:09.332305 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG dbcounter [-] [80378] Writing DB stats keystone:SELECT=100 {{(pid=80378) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 10:03:09.332532 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9e164ca-7d7e-47cc-9b73-8c1e9d2496b0 None admin] PATH_INFO: `/v3/services` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:03:09.333143 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9e164ca-7d7e-47cc-9b73-8c1e9d2496b0 None admin] RBAC: Authorizing `identity:create_service()` {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:03:09.334670 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9e164ca-7d7e-47cc-9b73-8c1e9d2496b0 None admin] RBAC: Authorization granted {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:03:09.355622 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: [pid: 80377|app: 0|req: 455/893] 10.222.0.82 () {60 vars in 1141 bytes} [Tue Aug 2 10:03:09 2022] POST /identity/v3/services => generated 241 bytes in 206 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 02 10:03:09.357932 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f57e32b-1714-4133-8c3c-9e94171a90b3 None None] REQUEST_METHOD: `GET` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:03:09.358449 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f57e32b-1714-4133-8c3c-9e94171a90b3 None None] SCRIPT_NAME: `/identity` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:03:09.358927 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f57e32b-1714-4133-8c3c-9e94171a90b3 None None] PATH_INFO: `/` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:03:09.361699 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: [pid: 80377|app: 0|req: 456/894] 10.222.0.82 () {54 vars in 926 bytes} [Tue Aug 2 10:03:09 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 02 10:03:09.370101 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6ed774e-fff2-4bb5-b23a-a2cb9ed646ac None None] REQUEST_METHOD: `POST` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:03:09.370653 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6ed774e-fff2-4bb5-b23a-a2cb9ed646ac None None] SCRIPT_NAME: `/identity` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:03:09.371211 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6ed774e-fff2-4bb5-b23a-a2cb9ed646ac None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:03:09.402174 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.auth.core [None req-c6ed774e-fff2-4bb5-b23a-a2cb9ed646ac None None] MFA Rules not processed for user `891005fa5ff74041b75747d16cc66798`. Rule list: `[]` (Enabled: `True`). {{(pid=80377) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 02 10:03:09.404387 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.fernet_utils [None req-c6ed774e-fff2-4bb5-b23a-a2cb9ed646ac None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80377) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 10:03:09.453760 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG dbcounter [-] [80377] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80377) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 10:03:09.484956 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8283c936-b7ef-4dbd-861b-a66ec79c36f9 None None] Validating token access rules against request {{(pid=80378) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:03:09.533454 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: [pid: 80377|app: 0|req: 457/895] 10.222.0.82 () {58 vars in 1023 bytes} [Tue Aug 2 10:03:09 2022] POST /identity/v3/auth/tokens => generated 1474 bytes in 165 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Aug 02 10:03:09.545230 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40bbcc95-8eed-43c2-9fcc-7bae315cdc7d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '891005fa5ff74041b75747d16cc66798', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80378) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:03:09.546333 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40bbcc95-8eed-43c2-9fcc-7bae315cdc7d None admin] REQUEST_METHOD: `GET` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:03:09.546897 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40bbcc95-8eed-43c2-9fcc-7bae315cdc7d None admin] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:03:09.547473 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40bbcc95-8eed-43c2-9fcc-7bae315cdc7d None admin] PATH_INFO: `/v3/services/compute_legacy` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:03:09.549129 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40bbcc95-8eed-43c2-9fcc-7bae315cdc7d None admin] RBAC: Authorizing `identity:get_service(service_id=compute_legacy)` {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:03:09.551423 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40bbcc95-8eed-43c2-9fcc-7bae315cdc7d None admin] RBAC: Authorization granted {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:03:09.562057 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6ed774e-fff2-4bb5-b23a-a2cb9ed646ac None None] Authenticating user token {{(pid=80377) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:03:09.563918 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application [None req-40bbcc95-8eed-43c2-9fcc-7bae315cdc7d None admin] Could not find service: compute_legacy.: keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Aug 02 10:03:09.563918 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 02 10:03:09.563918 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Aug 02 10:03:09.563918 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Aug 02 10:03:09.563918 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Aug 02 10:03:09.563918 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Aug 02 10:03:09.563918 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Aug 02 10:03:09.563918 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 02 10:03:09.563918 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Aug 02 10:03:09.563918 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application Aug 02 10:03:09.563918 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 02 10:03:09.563918 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application Aug 02 10:03:09.563918 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 02 10:03:09.563918 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 02 10:03:09.563918 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 02 10:03:09.563918 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 02 10:03:09.565472 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 02 10:03:09.565472 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 02 10:03:09.565472 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 02 10:03:09.565472 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 02 10:03:09.565472 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 02 10:03:09.565472 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 02 10:03:09.565472 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 02 10:03:09.565472 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Aug 02 10:03:09.565472 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application return self._get_service(service_id) Aug 02 10:03:09.565472 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Aug 02 10:03:09.565472 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Aug 02 10:03:09.565472 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 02 10:03:09.565472 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 02 10:03:09.565472 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 02 10:03:09.565472 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 02 10:03:09.565472 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 02 10:03:09.566379 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application return self.get_or_create( Aug 02 10:03:09.566379 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 02 10:03:09.566379 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application with Lock( Aug 02 10:03:09.566379 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 02 10:03:09.566379 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application return self._enter() Aug 02 10:03:09.566379 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 02 10:03:09.566379 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 02 10:03:09.566379 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 02 10:03:09.566379 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application return self.creator() Aug 02 10:03:09.566379 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 02 10:03:09.566379 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application created_value = creator( Aug 02 10:03:09.566379 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Aug 02 10:03:09.566379 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 02 10:03:09.566379 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Aug 02 10:03:09.566379 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application Aug 02 10:03:09.566379 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 439/896] 10.222.0.82 () {56 vars in 1132 bytes} [Tue Aug 2 10:03:09 2022] GET /identity/v3/services/compute_legacy => generated 95 bytes in 299 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 02 10:03:09.579693 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40bbcc95-8eed-43c2-9fcc-7bae315cdc7d None admin] Authenticating user token {{(pid=80378) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:03:09.753886 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG dbcounter [-] [80378] Writing DB stats keystone:SELECT=100 {{(pid=80378) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 10:03:09.758310 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6ed774e-fff2-4bb5-b23a-a2cb9ed646ac None None] Validating token access rules against request {{(pid=80377) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:03:09.766044 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40bbcc95-8eed-43c2-9fcc-7bae315cdc7d None admin] Validating token access rules against request {{(pid=80378) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:03:09.816363 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59405566-2f18-43c3-9e22-609322faba12 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '891005fa5ff74041b75747d16cc66798', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80377) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:03:09.817777 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59405566-2f18-43c3-9e22-609322faba12 None admin] REQUEST_METHOD: `GET` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:03:09.818494 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59405566-2f18-43c3-9e22-609322faba12 None admin] SCRIPT_NAME: `/identity` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:03:09.819093 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59405566-2f18-43c3-9e22-609322faba12 None admin] PATH_INFO: `/v3/roles/ResellerAdmin` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:03:09.827971 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f13b9f05-42f0-4ba9-85d5-1edc574412d0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '891005fa5ff74041b75747d16cc66798', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80378) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:03:09.830134 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f13b9f05-42f0-4ba9-85d5-1edc574412d0 None admin] REQUEST_METHOD: `GET` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:03:09.830837 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f13b9f05-42f0-4ba9-85d5-1edc574412d0 None admin] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:03:09.831546 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f13b9f05-42f0-4ba9-85d5-1edc574412d0 None admin] PATH_INFO: `/v3/services` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:03:09.832856 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f13b9f05-42f0-4ba9-85d5-1edc574412d0 None admin] RBAC: Adding query filter params (name=compute_legacy) {{(pid=80378) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 02 10:03:09.833858 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f13b9f05-42f0-4ba9-85d5-1edc574412d0 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:03:09.836606 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f13b9f05-42f0-4ba9-85d5-1edc574412d0 None admin] RBAC: Authorization granted {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:03:09.839130 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59405566-2f18-43c3-9e22-609322faba12 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 02 10:03:09.845102 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application [None req-59405566-2f18-43c3-9e22-609322faba12 None admin] Could not find role: ResellerAdmin.: keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Aug 02 10:03:09.845102 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 02 10:03:09.845102 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 02 10:03:09.845102 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 02 10:03:09.845102 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 02 10:03:09.845102 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 02 10:03:09.845102 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 02 10:03:09.845102 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 02 10:03:09.845102 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 02 10:03:09.845102 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 02 10:03:09.845102 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 02 10:03:09.845102 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 02 10:03:09.845102 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 02 10:03:09.845102 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 02 10:03:09.845102 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 02 10:03:09.845102 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 02 10:03:09.847243 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 02 10:03:09.847243 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 02 10:03:09.847243 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 02 10:03:09.847243 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 02 10:03:09.847243 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 02 10:03:09.847243 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 02 10:03:09.847243 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 02 10:03:09.847243 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 02 10:03:09.847243 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 02 10:03:09.847243 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application return self.get_or_create( Aug 02 10:03:09.847243 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 02 10:03:09.847243 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application with Lock( Aug 02 10:03:09.847243 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 02 10:03:09.847243 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application return self._enter() Aug 02 10:03:09.847243 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 02 10:03:09.847243 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 02 10:03:09.848787 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 02 10:03:09.848787 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application return self.creator() Aug 02 10:03:09.848787 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 02 10:03:09.848787 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application created_value = creator( Aug 02 10:03:09.848787 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 02 10:03:09.848787 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 02 10:03:09.848787 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 02 10:03:09.848787 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 02 10:03:09.848787 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 02 10:03:09.848787 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 02 10:03:09.848787 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Aug 02 10:03:09.848787 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application Aug 02 10:03:09.848787 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: [pid: 80377|app: 0|req: 458/897] 10.222.0.82 () {56 vars in 1120 bytes} [Tue Aug 2 10:03:09 2022] GET /identity/v3/roles/ResellerAdmin => generated 91 bytes in 286 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 02 10:03:09.851921 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 440/898] 10.222.0.82 () {56 vars in 1126 bytes} [Tue Aug 2 10:03:09 2022] GET /identity/v3/services?name=compute_legacy => generated 131 bytes in 274 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 02 10:03:09.859116 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59405566-2f18-43c3-9e22-609322faba12 None admin] Authenticating user token {{(pid=80377) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:03:09.863480 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f13b9f05-42f0-4ba9-85d5-1edc574412d0 None admin] Authenticating user token {{(pid=80378) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:03:09.895532 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG dbcounter [-] [80377] Writing DB stats keystone:SELECT=100 {{(pid=80377) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 10:03:10.041766 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f13b9f05-42f0-4ba9-85d5-1edc574412d0 None admin] Validating token access rules against request {{(pid=80378) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:03:10.043691 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59405566-2f18-43c3-9e22-609322faba12 None admin] Validating token access rules against request {{(pid=80377) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:03:10.088615 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14bac207-a03d-42cc-b563-256f3ca4edcf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '891005fa5ff74041b75747d16cc66798', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80378) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:03:10.089265 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14bac207-a03d-42cc-b563-256f3ca4edcf None admin] REQUEST_METHOD: `GET` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:03:10.089596 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14bac207-a03d-42cc-b563-256f3ca4edcf None admin] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:03:10.089953 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14bac207-a03d-42cc-b563-256f3ca4edcf None admin] PATH_INFO: `/v3/services` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:03:10.090804 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14bac207-a03d-42cc-b563-256f3ca4edcf None admin] RBAC: Adding query filter params (type=compute_legacy) {{(pid=80378) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 02 10:03:10.091126 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14bac207-a03d-42cc-b563-256f3ca4edcf None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:03:10.092876 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14bac207-a03d-42cc-b563-256f3ca4edcf None admin] RBAC: Authorization granted {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:03:10.093536 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1eba5020-f287-44cd-9078-9f36a47d5015 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '891005fa5ff74041b75747d16cc66798', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80377) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:03:10.094520 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1eba5020-f287-44cd-9078-9f36a47d5015 None admin] REQUEST_METHOD: `GET` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:03:10.095418 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1eba5020-f287-44cd-9078-9f36a47d5015 None admin] SCRIPT_NAME: `/identity` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:03:10.096653 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1eba5020-f287-44cd-9078-9f36a47d5015 None admin] PATH_INFO: `/v3/roles` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:03:10.098414 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1eba5020-f287-44cd-9078-9f36a47d5015 None admin] RBAC: Adding query filter params (name=ResellerAdmin) {{(pid=80377) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 02 10:03:10.098952 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1eba5020-f287-44cd-9078-9f36a47d5015 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:03:10.105811 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 441/899] 10.222.0.82 () {56 vars in 1126 bytes} [Tue Aug 2 10:03:09 2022] GET /identity/v3/services?type=compute_legacy => generated 386 bytes in 238 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 02 10:03:10.105999 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1eba5020-f287-44cd-9078-9f36a47d5015 None admin] RBAC: Authorization granted {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:03:10.108649 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14bac207-a03d-42cc-b563-256f3ca4edcf None admin] Authenticating user token {{(pid=80378) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:03:10.116601 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: [pid: 80377|app: 0|req: 459/900] 10.222.0.82 () {56 vars in 1115 bytes} [Tue Aug 2 10:03:09 2022] GET /identity/v3/roles?name=ResellerAdmin => generated 338 bytes in 258 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 02 10:03:10.124632 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1eba5020-f287-44cd-9078-9f36a47d5015 None admin] Authenticating user token {{(pid=80377) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:03:10.142964 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG dbcounter [-] [80378] Writing DB stats keystone:SELECT=100 {{(pid=80378) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 10:03:10.238666 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG dbcounter [-] [80377] Writing DB stats keystone:SELECT=100 {{(pid=80377) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 10:03:10.239105 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14bac207-a03d-42cc-b563-256f3ca4edcf None admin] Validating token access rules against request {{(pid=80378) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:03:10.286249 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1eba5020-f287-44cd-9078-9f36a47d5015 None admin] Validating token access rules against request {{(pid=80377) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:03:10.287696 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87baa9b6-685f-4228-a583-35910c7d2efe None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '891005fa5ff74041b75747d16cc66798', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80378) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:03:10.288613 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87baa9b6-685f-4228-a583-35910c7d2efe None admin] REQUEST_METHOD: `GET` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:03:10.289259 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87baa9b6-685f-4228-a583-35910c7d2efe None admin] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:03:10.290119 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87baa9b6-685f-4228-a583-35910c7d2efe None admin] PATH_INFO: `/v3/endpoints` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:03:10.291704 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87baa9b6-685f-4228-a583-35910c7d2efe None admin] RBAC: Adding query filter params (interface=public, service_id=72eef4328989434b875b0a9a8e12386e, region_id=RegionOne) {{(pid=80378) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 02 10:03:10.292307 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87baa9b6-685f-4228-a583-35910c7d2efe None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:03:10.293945 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87baa9b6-685f-4228-a583-35910c7d2efe None admin] RBAC: Authorization granted {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:03:10.306440 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 442/901] 10.222.0.82 () {56 vars in 1251 bytes} [Tue Aug 2 10:03:10 2022] GET /identity/v3/endpoints?service_id=72eef4328989434b875b0a9a8e12386e&interface=public®ion_id=RegionOne => generated 194 bytes in 198 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 02 10:03:10.307699 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87baa9b6-685f-4228-a583-35910c7d2efe None admin] Authenticating user token {{(pid=80378) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:03:10.336735 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8da82c82-93cd-44a1-85e9-658731fa6ba9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '891005fa5ff74041b75747d16cc66798', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80377) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:03:10.337586 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8da82c82-93cd-44a1-85e9-658731fa6ba9 None admin] REQUEST_METHOD: `GET` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:03:10.337969 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8da82c82-93cd-44a1-85e9-658731fa6ba9 None admin] SCRIPT_NAME: `/identity` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:03:10.338250 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8da82c82-93cd-44a1-85e9-658731fa6ba9 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:03:10.354914 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8da82c82-93cd-44a1-85e9-658731fa6ba9 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:03:10.356456 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8da82c82-93cd-44a1-85e9-658731fa6ba9 None admin] RBAC: Authorization granted {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:03:10.373429 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application [None req-8da82c82-93cd-44a1-85e9-658731fa6ba9 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 02 10:03:10.373429 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 02 10:03:10.373429 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 02 10:03:10.373429 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 02 10:03:10.373429 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 02 10:03:10.373429 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 02 10:03:10.373429 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 02 10:03:10.373429 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 02 10:03:10.373429 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 02 10:03:10.373429 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 02 10:03:10.373429 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 02 10:03:10.373429 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 02 10:03:10.373429 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 02 10:03:10.373429 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 02 10:03:10.373429 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 02 10:03:10.374524 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 02 10:03:10.374524 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 02 10:03:10.374524 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 02 10:03:10.374524 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 02 10:03:10.374524 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 02 10:03:10.374524 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 02 10:03:10.374524 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application return self.get_or_create( Aug 02 10:03:10.374524 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 02 10:03:10.374524 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application with Lock( Aug 02 10:03:10.374524 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 02 10:03:10.374524 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application return self._enter() Aug 02 10:03:10.374524 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 02 10:03:10.374524 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 02 10:03:10.374524 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 02 10:03:10.374524 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application return self.creator() Aug 02 10:03:10.374524 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 02 10:03:10.374524 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application created_value = creator( Aug 02 10:03:10.375620 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 02 10:03:10.375620 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 02 10:03:10.375620 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 02 10:03:10.375620 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application Aug 02 10:03:10.381528 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: [pid: 80377|app: 0|req: 460/902] 10.222.0.82 () {56 vars in 1108 bytes} [Tue Aug 2 10:03:10 2022] GET /identity/v3/domains/Default => generated 87 bytes in 253 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 02 10:03:10.382705 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8da82c82-93cd-44a1-85e9-658731fa6ba9 None admin] Authenticating user token {{(pid=80377) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:03:10.419750 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG dbcounter [-] [80378] Writing DB stats keystone:SELECT=100 {{(pid=80378) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 10:03:10.438414 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87baa9b6-685f-4228-a583-35910c7d2efe None admin] Validating token access rules against request {{(pid=80378) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:03:10.485152 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b02d2a42-71e9-4cc7-801c-b7da6a263d5d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '891005fa5ff74041b75747d16cc66798', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80378) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:03:10.486914 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b02d2a42-71e9-4cc7-801c-b7da6a263d5d None admin] REQUEST_METHOD: `GET` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:03:10.487438 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b02d2a42-71e9-4cc7-801c-b7da6a263d5d None admin] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:03:10.487945 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b02d2a42-71e9-4cc7-801c-b7da6a263d5d None admin] PATH_INFO: `/v3/services` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:03:10.488891 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b02d2a42-71e9-4cc7-801c-b7da6a263d5d None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:03:10.490531 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b02d2a42-71e9-4cc7-801c-b7da6a263d5d None admin] RBAC: Authorization granted {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:03:10.498624 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 443/903] 10.222.0.82 () {56 vars in 1087 bytes} [Tue Aug 2 10:03:10 2022] GET /identity/v3/services => generated 1497 bytes in 191 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 02 10:03:10.540307 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-587c6a19-6e75-4195-99a6-517f0da198f1 None None] REQUEST_METHOD: `GET` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:03:10.540869 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-587c6a19-6e75-4195-99a6-517f0da198f1 None None] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:03:10.541014 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8da82c82-93cd-44a1-85e9-658731fa6ba9 None admin] Validating token access rules against request {{(pid=80377) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:03:10.541429 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-587c6a19-6e75-4195-99a6-517f0da198f1 None None] PATH_INFO: `/` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:03:10.544153 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 444/904] 10.222.0.82 () {54 vars in 926 bytes} [Tue Aug 2 10:03:10 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 02 10:03:10.551254 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-204b8b47-a53d-4c25-bf64-d03776b139dd None None] REQUEST_METHOD: `POST` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:03:10.551737 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-204b8b47-a53d-4c25-bf64-d03776b139dd None None] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:03:10.552177 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-204b8b47-a53d-4c25-bf64-d03776b139dd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:03:10.587065 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.auth.core [None req-204b8b47-a53d-4c25-bf64-d03776b139dd None None] MFA Rules not processed for user `891005fa5ff74041b75747d16cc66798`. Rule list: `[]` (Enabled: `True`). {{(pid=80378) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 02 10:03:10.589726 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.fernet_utils [None req-204b8b47-a53d-4c25-bf64-d03776b139dd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80378) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 10:03:10.591245 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG dbcounter [-] [80377] Writing DB stats keystone:SELECT=100 {{(pid=80377) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 10:03:10.607194 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd7e023a-4f60-42cb-b580-6b5617a9a405 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '891005fa5ff74041b75747d16cc66798', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80377) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:03:10.608434 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd7e023a-4f60-42cb-b580-6b5617a9a405 None admin] REQUEST_METHOD: `GET` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:03:10.608822 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd7e023a-4f60-42cb-b580-6b5617a9a405 None admin] SCRIPT_NAME: `/identity` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:03:10.609170 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd7e023a-4f60-42cb-b580-6b5617a9a405 None admin] PATH_INFO: `/v3/domains` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:03:10.610035 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd7e023a-4f60-42cb-b580-6b5617a9a405 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80377) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 02 10:03:10.610431 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd7e023a-4f60-42cb-b580-6b5617a9a405 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:03:10.612076 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd7e023a-4f60-42cb-b580-6b5617a9a405 None admin] RBAC: Authorization granted {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:03:10.646651 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: [pid: 80377|app: 0|req: 461/905] 10.222.0.82 () {56 vars in 1109 bytes} [Tue Aug 2 10:03:10 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 260 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 02 10:03:10.649646 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd7e023a-4f60-42cb-b580-6b5617a9a405 None admin] Authenticating user token {{(pid=80377) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:03:10.717973 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 445/906] 10.222.0.82 () {58 vars in 1023 bytes} [Tue Aug 2 10:03:10 2022] POST /identity/v3/auth/tokens => generated 1474 bytes in 167 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Aug 02 10:03:10.760514 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-204b8b47-a53d-4c25-bf64-d03776b139dd None None] Authenticating user token {{(pid=80378) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:03:10.819851 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd7e023a-4f60-42cb-b580-6b5617a9a405 None admin] Validating token access rules against request {{(pid=80377) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:03:10.881989 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9069a2cb-f628-40a2-852c-6ff273da4212 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '891005fa5ff74041b75747d16cc66798', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80377) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:03:10.882707 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9069a2cb-f628-40a2-852c-6ff273da4212 None admin] REQUEST_METHOD: `GET` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:03:10.883015 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9069a2cb-f628-40a2-852c-6ff273da4212 None admin] SCRIPT_NAME: `/identity` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:03:10.883416 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9069a2cb-f628-40a2-852c-6ff273da4212 None admin] PATH_INFO: `/v3/users/glance-swift` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:03:10.891612 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG dbcounter [-] [80378] Writing DB stats keystone:SELECT=100 {{(pid=80378) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 10:03:10.894446 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9069a2cb-f628-40a2-852c-6ff273da4212 None admin] RBAC: Authorizing `identity:get_user(user_id=glance-swift)` {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:03:10.896680 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9069a2cb-f628-40a2-852c-6ff273da4212 None admin] RBAC: Authorization granted {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:03:10.911222 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application [None req-9069a2cb-f628-40a2-852c-6ff273da4212 None admin] Could not find user: glance-swift.: keystone.exception.UserNotFound: Could not find user: glance-swift. Aug 02 10:03:10.911222 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 02 10:03:10.911222 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 02 10:03:10.911222 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 02 10:03:10.911222 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 02 10:03:10.911222 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 02 10:03:10.911222 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 02 10:03:10.911222 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 02 10:03:10.911222 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 02 10:03:10.911222 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 02 10:03:10.911222 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 02 10:03:10.911222 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 02 10:03:10.911222 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 02 10:03:10.911222 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 02 10:03:10.911222 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 02 10:03:10.911222 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 02 10:03:10.912376 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 02 10:03:10.912376 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 02 10:03:10.912376 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 02 10:03:10.912376 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 02 10:03:10.912376 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 02 10:03:10.912376 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 02 10:03:10.912376 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 02 10:03:10.912376 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 02 10:03:10.912376 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 02 10:03:10.912376 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application return self.get_or_create( Aug 02 10:03:10.912376 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 02 10:03:10.912376 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application with Lock( Aug 02 10:03:10.912376 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 02 10:03:10.912376 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application return self._enter() Aug 02 10:03:10.912376 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 02 10:03:10.912376 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 02 10:03:10.912376 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 02 10:03:10.912376 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application return self.creator() Aug 02 10:03:10.914012 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 02 10:03:10.914012 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application created_value = creator( Aug 02 10:03:10.914012 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 02 10:03:10.914012 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 02 10:03:10.914012 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 02 10:03:10.914012 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 02 10:03:10.914012 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 02 10:03:10.914012 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 02 10:03:10.914012 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance-swift. Aug 02 10:03:10.914012 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application Aug 02 10:03:10.914012 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: [pid: 80377|app: 0|req: 462/907] 10.222.0.82 () {56 vars in 1117 bytes} [Tue Aug 2 10:03:10 2022] GET /identity/v3/users/glance-swift => generated 90 bytes in 263 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 02 10:03:10.921898 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-204b8b47-a53d-4c25-bf64-d03776b139dd None None] Validating token access rules against request {{(pid=80378) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:03:10.924405 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9069a2cb-f628-40a2-852c-6ff273da4212 None admin] Authenticating user token {{(pid=80377) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:03:10.961024 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b65ae64d-520f-4131-b948-fbc2974a478d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '891005fa5ff74041b75747d16cc66798', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80378) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:03:10.961744 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b65ae64d-520f-4131-b948-fbc2974a478d None admin] REQUEST_METHOD: `GET` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:03:10.962061 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b65ae64d-520f-4131-b948-fbc2974a478d None admin] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:03:10.962334 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b65ae64d-520f-4131-b948-fbc2974a478d None admin] PATH_INFO: `/v3/services/volumev3` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:03:10.962971 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b65ae64d-520f-4131-b948-fbc2974a478d None admin] RBAC: Authorizing `identity:get_service(service_id=volumev3)` {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:03:10.964471 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b65ae64d-520f-4131-b948-fbc2974a478d None admin] RBAC: Authorization granted {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:03:10.974319 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application [None req-b65ae64d-520f-4131-b948-fbc2974a478d None admin] Could not find service: volumev3.: keystone.exception.ServiceNotFound: Could not find service: volumev3. Aug 02 10:03:10.974319 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 02 10:03:10.974319 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Aug 02 10:03:10.974319 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Aug 02 10:03:10.974319 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Aug 02 10:03:10.974319 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Aug 02 10:03:10.974319 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Aug 02 10:03:10.974319 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 02 10:03:10.974319 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: volumev3. Aug 02 10:03:10.974319 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application Aug 02 10:03:10.974319 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 02 10:03:10.974319 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application Aug 02 10:03:10.974319 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 02 10:03:10.974319 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 02 10:03:10.974319 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 02 10:03:10.974319 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 02 10:03:10.974319 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 02 10:03:10.977874 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 02 10:03:10.977874 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 02 10:03:10.977874 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 02 10:03:10.977874 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 02 10:03:10.977874 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 02 10:03:10.977874 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 02 10:03:10.977874 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Aug 02 10:03:10.977874 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application return self._get_service(service_id) Aug 02 10:03:10.977874 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Aug 02 10:03:10.977874 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Aug 02 10:03:10.977874 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 02 10:03:10.977874 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 02 10:03:10.977874 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 02 10:03:10.977874 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 02 10:03:10.977874 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 02 10:03:10.977874 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application return self.get_or_create( Aug 02 10:03:10.980089 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 02 10:03:10.980089 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application with Lock( Aug 02 10:03:10.980089 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 02 10:03:10.980089 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application return self._enter() Aug 02 10:03:10.980089 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 02 10:03:10.980089 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 02 10:03:10.980089 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 02 10:03:10.980089 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application return self.creator() Aug 02 10:03:10.980089 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 02 10:03:10.980089 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application created_value = creator( Aug 02 10:03:10.980089 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Aug 02 10:03:10.980089 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 02 10:03:10.980089 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: volumev3. Aug 02 10:03:10.980089 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application Aug 02 10:03:10.980089 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 446/908] 10.222.0.82 () {56 vars in 1114 bytes} [Tue Aug 2 10:03:10 2022] GET /identity/v3/services/volumev3 => generated 89 bytes in 216 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 02 10:03:10.985558 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b65ae64d-520f-4131-b948-fbc2974a478d None admin] Authenticating user token {{(pid=80378) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:03:11.000293 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG dbcounter [-] [80377] Writing DB stats keystone:SELECT=100 {{(pid=80377) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 10:03:11.105528 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9069a2cb-f628-40a2-852c-6ff273da4212 None admin] Validating token access rules against request {{(pid=80377) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:03:11.140876 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b65ae64d-520f-4131-b948-fbc2974a478d None admin] Validating token access rules against request {{(pid=80378) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:03:11.164704 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4a0cc7b-9a56-4a51-af9b-2f90e54c1ad8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '891005fa5ff74041b75747d16cc66798', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80377) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:03:11.166036 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4a0cc7b-9a56-4a51-af9b-2f90e54c1ad8 None admin] REQUEST_METHOD: `GET` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:03:11.166539 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4a0cc7b-9a56-4a51-af9b-2f90e54c1ad8 None admin] SCRIPT_NAME: `/identity` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:03:11.166997 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4a0cc7b-9a56-4a51-af9b-2f90e54c1ad8 None admin] PATH_INFO: `/v3/users` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:03:11.168349 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4a0cc7b-9a56-4a51-af9b-2f90e54c1ad8 None admin] RBAC: Adding query filter params (domain_id=default, name=glance-swift) {{(pid=80377) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 02 10:03:11.169129 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4a0cc7b-9a56-4a51-af9b-2f90e54c1ad8 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:03:11.171859 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4a0cc7b-9a56-4a51-af9b-2f90e54c1ad8 None admin] RBAC: Authorization granted {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:03:11.195196 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32c1280f-2a53-4a29-a40d-79966f5c1823 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '891005fa5ff74041b75747d16cc66798', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80378) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:03:11.197771 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32c1280f-2a53-4a29-a40d-79966f5c1823 None admin] REQUEST_METHOD: `GET` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:03:11.198307 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32c1280f-2a53-4a29-a40d-79966f5c1823 None admin] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:03:11.198785 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32c1280f-2a53-4a29-a40d-79966f5c1823 None admin] PATH_INFO: `/v3/services` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:03:11.199934 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32c1280f-2a53-4a29-a40d-79966f5c1823 None admin] RBAC: Adding query filter params (name=volumev3) {{(pid=80378) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 02 10:03:11.200473 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32c1280f-2a53-4a29-a40d-79966f5c1823 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:03:11.202004 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32c1280f-2a53-4a29-a40d-79966f5c1823 None admin] RBAC: Authorization granted {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:03:11.210003 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 447/909] 10.222.0.82 () {56 vars in 1114 bytes} [Tue Aug 2 10:03:10 2022] GET /identity/v3/services?name=volumev3 => generated 125 bytes in 228 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 02 10:03:11.215284 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32c1280f-2a53-4a29-a40d-79966f5c1823 None admin] Authenticating user token {{(pid=80378) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:03:11.224069 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: [pid: 80377|app: 0|req: 463/910] 10.222.0.82 () {56 vars in 1149 bytes} [Tue Aug 2 10:03:10 2022] GET /identity/v3/users?domain_id=default&name=glance-swift => generated 384 bytes in 295 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 02 10:03:11.225984 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4a0cc7b-9a56-4a51-af9b-2f90e54c1ad8 None admin] Authenticating user token {{(pid=80377) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:03:11.231910 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG dbcounter [-] [80378] Writing DB stats keystone:SELECT=100 {{(pid=80378) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 10:03:11.383547 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32c1280f-2a53-4a29-a40d-79966f5c1823 None admin] Validating token access rules against request {{(pid=80378) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:03:11.398248 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4a0cc7b-9a56-4a51-af9b-2f90e54c1ad8 None admin] Validating token access rules against request {{(pid=80377) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:03:11.411597 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG dbcounter [-] [80377] Writing DB stats keystone:SELECT=100 {{(pid=80377) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 10:03:11.454585 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b68f9a0-5cd5-4b4e-8cea-1f4f45bce53e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '891005fa5ff74041b75747d16cc66798', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80378) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:03:11.456014 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b68f9a0-5cd5-4b4e-8cea-1f4f45bce53e None admin] REQUEST_METHOD: `GET` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:03:11.456650 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f971474-d21d-4297-8ca9-dc96a9290d48 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '891005fa5ff74041b75747d16cc66798', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80377) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:03:11.456869 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b68f9a0-5cd5-4b4e-8cea-1f4f45bce53e None admin] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:03:11.457299 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b68f9a0-5cd5-4b4e-8cea-1f4f45bce53e None admin] PATH_INFO: `/v3/services` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:03:11.457739 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f971474-d21d-4297-8ca9-dc96a9290d48 None admin] REQUEST_METHOD: `GET` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:03:11.458194 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f971474-d21d-4297-8ca9-dc96a9290d48 None admin] SCRIPT_NAME: `/identity` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:03:11.458571 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f971474-d21d-4297-8ca9-dc96a9290d48 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:03:11.458686 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b68f9a0-5cd5-4b4e-8cea-1f4f45bce53e None admin] RBAC: Adding query filter params (type=volumev3) {{(pid=80378) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 02 10:03:11.459176 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b68f9a0-5cd5-4b4e-8cea-1f4f45bce53e None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:03:11.461102 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b68f9a0-5cd5-4b4e-8cea-1f4f45bce53e None admin] RBAC: Authorization granted {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:03:11.475479 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 448/911] 10.222.0.82 () {56 vars in 1114 bytes} [Tue Aug 2 10:03:11 2022] GET /identity/v3/services?type=volumev3 => generated 362 bytes in 257 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 02 10:03:11.477269 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b68f9a0-5cd5-4b4e-8cea-1f4f45bce53e None admin] Authenticating user token {{(pid=80378) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:03:11.566294 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f971474-d21d-4297-8ca9-dc96a9290d48 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:03:11.570028 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f971474-d21d-4297-8ca9-dc96a9290d48 None admin] RBAC: Authorization granted {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:03:11.606498 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG dbcounter [-] [80378] Writing DB stats keystone:SELECT=100 {{(pid=80378) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 10:03:11.651807 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b68f9a0-5cd5-4b4e-8cea-1f4f45bce53e None admin] Validating token access rules against request {{(pid=80378) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:03:11.698702 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d19f1c6-7206-4a58-ba2e-d3f0ba1ffd50 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '891005fa5ff74041b75747d16cc66798', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80378) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:03:11.699592 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d19f1c6-7206-4a58-ba2e-d3f0ba1ffd50 None admin] REQUEST_METHOD: `POST` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:03:11.699930 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d19f1c6-7206-4a58-ba2e-d3f0ba1ffd50 None admin] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:03:11.700325 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d19f1c6-7206-4a58-ba2e-d3f0ba1ffd50 None admin] PATH_INFO: `/v3/endpoints` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:03:11.701236 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d19f1c6-7206-4a58-ba2e-d3f0ba1ffd50 None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:03:11.703127 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d19f1c6-7206-4a58-ba2e-d3f0ba1ffd50 None admin] RBAC: Authorization granted {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:03:11.712835 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: [pid: 80377|app: 0|req: 464/912] 10.222.0.82 () {58 vars in 1282 bytes} [Tue Aug 2 10:03:11 2022] GET /identity/v3/auth/tokens => generated 1474 bytes in 487 msecs (HTTP/1.1 200) 6 headers in 359 bytes (2 switches on core 0) Aug 02 10:03:11.718691 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f971474-d21d-4297-8ca9-dc96a9290d48 None admin] Authenticating user token {{(pid=80377) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:03:11.730178 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG dbcounter [-] [80377] Writing DB stats keystone:SELECT=100 {{(pid=80377) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 10:03:11.745238 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 449/913] 10.222.0.82 () {60 vars in 1144 bytes} [Tue Aug 2 10:03:11 2022] POST /identity/v3/endpoints => generated 346 bytes in 269 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 02 10:03:11.866695 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f971474-d21d-4297-8ca9-dc96a9290d48 None admin] Validating token access rules against request {{(pid=80377) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:03:11.923046 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56145948-6908-4c3a-adea-c96f9b2c840b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '891005fa5ff74041b75747d16cc66798', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80377) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:03:11.924275 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56145948-6908-4c3a-adea-c96f9b2c840b None admin] REQUEST_METHOD: `GET` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:03:11.924553 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56145948-6908-4c3a-adea-c96f9b2c840b None admin] SCRIPT_NAME: `/identity` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:03:11.924822 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56145948-6908-4c3a-adea-c96f9b2c840b None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:03:11.938622 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56145948-6908-4c3a-adea-c96f9b2c840b None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:03:11.940828 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56145948-6908-4c3a-adea-c96f9b2c840b None admin] RBAC: Authorization granted {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:03:11.963618 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application [None req-56145948-6908-4c3a-adea-c96f9b2c840b None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 02 10:03:11.963618 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 02 10:03:11.963618 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 02 10:03:11.963618 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 02 10:03:11.963618 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 02 10:03:11.963618 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 02 10:03:11.963618 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 02 10:03:11.963618 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 02 10:03:11.963618 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 02 10:03:11.963618 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 02 10:03:11.963618 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 02 10:03:11.963618 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 02 10:03:11.963618 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 02 10:03:11.963618 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 02 10:03:11.963618 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 02 10:03:11.965687 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 02 10:03:11.965687 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 02 10:03:11.965687 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 02 10:03:11.965687 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 02 10:03:11.965687 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 02 10:03:11.965687 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 02 10:03:11.965687 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application return self.get_or_create( Aug 02 10:03:11.965687 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 02 10:03:11.965687 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application with Lock( Aug 02 10:03:11.965687 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 02 10:03:11.965687 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application return self._enter() Aug 02 10:03:11.965687 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 02 10:03:11.965687 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 02 10:03:11.965687 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 02 10:03:11.965687 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application return self.creator() Aug 02 10:03:11.965687 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 02 10:03:11.965687 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application created_value = creator( Aug 02 10:03:11.967048 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 02 10:03:11.967048 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 02 10:03:11.967048 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 02 10:03:11.967048 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application Aug 02 10:03:11.967048 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: [pid: 80377|app: 0|req: 465/914] 10.222.0.82 () {56 vars in 1108 bytes} [Tue Aug 2 10:03:11 2022] GET /identity/v3/domains/Default => generated 87 bytes in 243 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 02 10:03:11.980302 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d19f1c6-7206-4a58-ba2e-d3f0ba1ffd50 None admin] Authenticating user token {{(pid=80378) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:03:12.117837 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d19f1c6-7206-4a58-ba2e-d3f0ba1ffd50 None admin] Validating token access rules against request {{(pid=80378) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:03:12.144679 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28e05c28-b119-4098-b56f-443d8f8ba19d None None] REQUEST_METHOD: `GET` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:03:12.145414 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28e05c28-b119-4098-b56f-443d8f8ba19d None None] SCRIPT_NAME: `/identity` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:03:12.145567 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28e05c28-b119-4098-b56f-443d8f8ba19d None None] PATH_INFO: `/` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:03:12.147767 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: [pid: 80377|app: 0|req: 466/915] 10.222.0.82 () {54 vars in 926 bytes} [Tue Aug 2 10:03:12 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 02 10:03:12.154323 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbe133cf-2b29-48f3-b7dd-13aaee1a6462 None None] REQUEST_METHOD: `POST` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:03:12.154742 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbe133cf-2b29-48f3-b7dd-13aaee1a6462 None None] SCRIPT_NAME: `/identity` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:03:12.155103 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbe133cf-2b29-48f3-b7dd-13aaee1a6462 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:03:12.156077 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG dbcounter [-] [80378] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80378) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 10:03:12.173515 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-976a13f6-3b26-4e86-8830-f17b5c4bf108 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '891005fa5ff74041b75747d16cc66798', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80378) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:03:12.174649 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-976a13f6-3b26-4e86-8830-f17b5c4bf108 None admin] REQUEST_METHOD: `GET` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:03:12.174926 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-976a13f6-3b26-4e86-8830-f17b5c4bf108 None admin] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:03:12.175192 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-976a13f6-3b26-4e86-8830-f17b5c4bf108 None admin] PATH_INFO: `/v3/domains` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:03:12.175889 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-976a13f6-3b26-4e86-8830-f17b5c4bf108 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80378) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 02 10:03:12.176190 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-976a13f6-3b26-4e86-8830-f17b5c4bf108 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:03:12.177754 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.auth.core [None req-bbe133cf-2b29-48f3-b7dd-13aaee1a6462 None None] MFA Rules not processed for user `891005fa5ff74041b75747d16cc66798`. Rule list: `[]` (Enabled: `True`). {{(pid=80377) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 02 10:03:12.177903 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-976a13f6-3b26-4e86-8830-f17b5c4bf108 None admin] RBAC: Authorization granted {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:03:12.179229 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.fernet_utils [None req-bbe133cf-2b29-48f3-b7dd-13aaee1a6462 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80377) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 10:03:12.193445 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 450/916] 10.222.0.82 () {56 vars in 1109 bytes} [Tue Aug 2 10:03:11 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 214 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 02 10:03:12.203014 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-976a13f6-3b26-4e86-8830-f17b5c4bf108 None admin] Authenticating user token {{(pid=80378) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:03:12.266613 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG dbcounter [-] [80377] Writing DB stats keystone:SELECT=100 {{(pid=80377) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 10:03:12.321441 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: [pid: 80377|app: 0|req: 467/917] 10.222.0.82 () {58 vars in 1023 bytes} [Tue Aug 2 10:03:12 2022] POST /identity/v3/auth/tokens => generated 1474 bytes in 168 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Aug 02 10:03:12.347047 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbe133cf-2b29-48f3-b7dd-13aaee1a6462 None None] Authenticating user token {{(pid=80377) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:03:12.405016 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-976a13f6-3b26-4e86-8830-f17b5c4bf108 None admin] Validating token access rules against request {{(pid=80378) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:03:12.514870 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c32dff05-2ddd-4c5e-9fe2-6ce8a3e1f5c5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '891005fa5ff74041b75747d16cc66798', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80378) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:03:12.517894 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c32dff05-2ddd-4c5e-9fe2-6ce8a3e1f5c5 None admin] REQUEST_METHOD: `GET` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:03:12.521812 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c32dff05-2ddd-4c5e-9fe2-6ce8a3e1f5c5 None admin] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:03:12.521812 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c32dff05-2ddd-4c5e-9fe2-6ce8a3e1f5c5 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:03:12.558369 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c32dff05-2ddd-4c5e-9fe2-6ce8a3e1f5c5 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:03:12.561816 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c32dff05-2ddd-4c5e-9fe2-6ce8a3e1f5c5 None admin] RBAC: Authorization granted {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:03:12.621828 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application [None req-c32dff05-2ddd-4c5e-9fe2-6ce8a3e1f5c5 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 02 10:03:12.621828 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 02 10:03:12.621828 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 02 10:03:12.621828 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 02 10:03:12.621828 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 02 10:03:12.621828 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 02 10:03:12.621828 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 02 10:03:12.621828 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 02 10:03:12.621828 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 02 10:03:12.621828 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 02 10:03:12.621828 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 02 10:03:12.621828 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 02 10:03:12.621828 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 02 10:03:12.621828 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 02 10:03:12.621828 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 02 10:03:12.623075 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 02 10:03:12.623075 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 02 10:03:12.623075 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 02 10:03:12.623075 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 02 10:03:12.623075 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 02 10:03:12.623075 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 02 10:03:12.623075 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application return self.get_or_create( Aug 02 10:03:12.623075 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 02 10:03:12.623075 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application with Lock( Aug 02 10:03:12.623075 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 02 10:03:12.623075 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application return self._enter() Aug 02 10:03:12.623075 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 02 10:03:12.623075 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 02 10:03:12.623075 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 02 10:03:12.623075 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application return self.creator() Aug 02 10:03:12.623075 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 02 10:03:12.623075 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application created_value = creator( Aug 02 10:03:12.624071 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 02 10:03:12.624071 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 02 10:03:12.624071 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 02 10:03:12.624071 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 02 10:03:12.624071 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 02 10:03:12.624071 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 02 10:03:12.624071 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 02 10:03:12.624071 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application Aug 02 10:03:12.641813 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 451/918] 10.222.0.82 () {56 vars in 1111 bytes} [Tue Aug 2 10:03:12 2022] GET /identity/v3/projects/service => generated 88 bytes in 437 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 02 10:03:12.641813 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c32dff05-2ddd-4c5e-9fe2-6ce8a3e1f5c5 None admin] Authenticating user token {{(pid=80378) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:03:12.673816 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbe133cf-2b29-48f3-b7dd-13aaee1a6462 None None] Validating token access rules against request {{(pid=80377) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:03:12.754786 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG dbcounter [-] [80378] Writing DB stats keystone:SELECT=100 {{(pid=80378) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 10:03:12.769029 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-363b9b43-11f0-400d-a044-79f538b9ad3a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '891005fa5ff74041b75747d16cc66798', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80377) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:03:12.771173 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-363b9b43-11f0-400d-a044-79f538b9ad3a None admin] REQUEST_METHOD: `GET` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:03:12.771482 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-363b9b43-11f0-400d-a044-79f538b9ad3a None admin] SCRIPT_NAME: `/identity` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:03:12.771730 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-363b9b43-11f0-400d-a044-79f538b9ad3a None admin] PATH_INFO: `/v3/services/object-store` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:03:12.772384 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-363b9b43-11f0-400d-a044-79f538b9ad3a None admin] RBAC: Authorizing `identity:get_service(service_id=object-store)` {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:03:12.775999 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-363b9b43-11f0-400d-a044-79f538b9ad3a None admin] RBAC: Authorization granted {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:03:12.787347 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application [None req-363b9b43-11f0-400d-a044-79f538b9ad3a None admin] Could not find service: object-store.: keystone.exception.ServiceNotFound: Could not find service: object-store. Aug 02 10:03:12.787347 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 02 10:03:12.787347 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Aug 02 10:03:12.787347 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Aug 02 10:03:12.787347 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Aug 02 10:03:12.787347 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Aug 02 10:03:12.787347 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Aug 02 10:03:12.787347 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 02 10:03:12.787347 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Aug 02 10:03:12.787347 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application Aug 02 10:03:12.787347 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 02 10:03:12.787347 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application Aug 02 10:03:12.787347 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 02 10:03:12.787347 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 02 10:03:12.787347 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 02 10:03:12.787347 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 02 10:03:12.787347 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 02 10:03:12.788584 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 02 10:03:12.788584 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 02 10:03:12.788584 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 02 10:03:12.788584 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 02 10:03:12.788584 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 02 10:03:12.788584 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 02 10:03:12.788584 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Aug 02 10:03:12.788584 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application return self._get_service(service_id) Aug 02 10:03:12.788584 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Aug 02 10:03:12.788584 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Aug 02 10:03:12.788584 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 02 10:03:12.788584 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 02 10:03:12.788584 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 02 10:03:12.788584 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 02 10:03:12.788584 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 02 10:03:12.788584 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application return self.get_or_create( Aug 02 10:03:12.789698 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 02 10:03:12.789698 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application with Lock( Aug 02 10:03:12.789698 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 02 10:03:12.789698 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application return self._enter() Aug 02 10:03:12.789698 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 02 10:03:12.789698 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 02 10:03:12.789698 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 02 10:03:12.789698 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application return self.creator() Aug 02 10:03:12.789698 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 02 10:03:12.789698 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application created_value = creator( Aug 02 10:03:12.789698 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Aug 02 10:03:12.789698 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 02 10:03:12.789698 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Aug 02 10:03:12.789698 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application Aug 02 10:03:12.789698 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: [pid: 80377|app: 0|req: 468/919] 10.222.0.82 () {56 vars in 1126 bytes} [Tue Aug 2 10:03:12 2022] GET /identity/v3/services/object-store => generated 93 bytes in 442 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 02 10:03:12.797628 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-363b9b43-11f0-400d-a044-79f538b9ad3a None admin] Authenticating user token {{(pid=80377) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:03:12.864355 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c32dff05-2ddd-4c5e-9fe2-6ce8a3e1f5c5 None admin] Validating token access rules against request {{(pid=80378) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:03:12.954919 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11217a97-52a4-4bcf-a57e-f4aedebd59da None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '891005fa5ff74041b75747d16cc66798', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80378) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:03:12.961349 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11217a97-52a4-4bcf-a57e-f4aedebd59da None admin] REQUEST_METHOD: `GET` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:03:12.961630 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11217a97-52a4-4bcf-a57e-f4aedebd59da None admin] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:03:12.961993 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11217a97-52a4-4bcf-a57e-f4aedebd59da None admin] PATH_INFO: `/v3/projects` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:03:12.963524 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11217a97-52a4-4bcf-a57e-f4aedebd59da None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80378) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 02 10:03:12.963843 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11217a97-52a4-4bcf-a57e-f4aedebd59da None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:03:12.969275 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG dbcounter [-] [80377] Writing DB stats keystone:SELECT=100 {{(pid=80377) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 10:03:12.976373 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11217a97-52a4-4bcf-a57e-f4aedebd59da None admin] RBAC: Authorization granted {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:03:13.011010 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 452/920] 10.222.0.82 () {56 vars in 1148 bytes} [Tue Aug 2 10:03:12 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 370 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 02 10:03:13.011483 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11217a97-52a4-4bcf-a57e-f4aedebd59da None admin] Authenticating user token {{(pid=80378) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:03:13.086974 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-363b9b43-11f0-400d-a044-79f538b9ad3a None admin] Validating token access rules against request {{(pid=80377) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:03:13.129060 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-feee2778-237c-49ef-9055-2a6f16c0fc65 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '891005fa5ff74041b75747d16cc66798', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80377) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:03:13.129723 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-feee2778-237c-49ef-9055-2a6f16c0fc65 None admin] REQUEST_METHOD: `GET` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:03:13.130030 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-feee2778-237c-49ef-9055-2a6f16c0fc65 None admin] SCRIPT_NAME: `/identity` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:03:13.130385 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-feee2778-237c-49ef-9055-2a6f16c0fc65 None admin] PATH_INFO: `/v3/services` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:03:13.130989 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-feee2778-237c-49ef-9055-2a6f16c0fc65 None admin] RBAC: Adding query filter params (name=object-store) {{(pid=80377) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 02 10:03:13.131294 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-feee2778-237c-49ef-9055-2a6f16c0fc65 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:03:13.132840 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-feee2778-237c-49ef-9055-2a6f16c0fc65 None admin] RBAC: Authorization granted {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:03:13.143429 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: [pid: 80377|app: 0|req: 469/921] 10.222.0.82 () {56 vars in 1122 bytes} [Tue Aug 2 10:03:12 2022] GET /identity/v3/services?name=object-store => generated 129 bytes in 347 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 02 10:03:13.149290 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f8d8a21-66f0-43b2-a184-0933a018bef3 None None] REQUEST_METHOD: `GET` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:03:13.150033 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f8d8a21-66f0-43b2-a184-0933a018bef3 None None] SCRIPT_NAME: `/identity` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:03:13.150266 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f8d8a21-66f0-43b2-a184-0933a018bef3 None None] PATH_INFO: `/` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:03:13.155276 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: [pid: 80377|app: 0|req: 470/922] 10.222.0.82 () {54 vars in 926 bytes} [Tue Aug 2 10:03:13 2022] GET /identity => generated 270 bytes in 11 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 02 10:03:13.156579 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f8d8a21-66f0-43b2-a184-0933a018bef3 None None] Authenticating user token {{(pid=80377) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:03:13.195760 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11217a97-52a4-4bcf-a57e-f4aedebd59da None admin] Validating token access rules against request {{(pid=80378) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:03:13.215152 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG dbcounter [-] [80378] Writing DB stats keystone:SELECT=100 {{(pid=80378) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 10:03:13.256798 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e39a2330-a438-4fae-8144-289c672079fa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '891005fa5ff74041b75747d16cc66798', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80378) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:03:13.258141 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e39a2330-a438-4fae-8144-289c672079fa None admin] REQUEST_METHOD: `GET` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:03:13.258701 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e39a2330-a438-4fae-8144-289c672079fa None admin] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:03:13.259321 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e39a2330-a438-4fae-8144-289c672079fa None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:03:13.261077 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e39a2330-a438-4fae-8144-289c672079fa None admin] RBAC: Adding query filter params (role.id=f57a652471c8414597c5fdde3d6f4636, scope.project.id=f1b38ebb804c41c69772c998aae72c41, user.id=bd8dd72086db48b8aaac2ed0058872df) {{(pid=80378) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 02 10:03:13.261749 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e39a2330-a438-4fae-8144-289c672079fa None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:03:13.265123 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e39a2330-a438-4fae-8144-289c672079fa None admin] RBAC: Authorization granted {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:03:13.275827 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 453/923] 10.222.0.82 () {56 vars in 1374 bytes} [Tue Aug 2 10:03:13 2022] GET /identity/v3/role_assignments?user.id=bd8dd72086db48b8aaac2ed0058872df&scope.project.id=f1b38ebb804c41c69772c998aae72c41&role.id=f57a652471c8414597c5fdde3d6f4636 => generated 259 bytes in 265 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 02 10:03:13.278169 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-690d6579-73cc-4ad1-a169-6aa56fa41eb0 None None] REQUEST_METHOD: `POST` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:03:13.278636 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-690d6579-73cc-4ad1-a169-6aa56fa41eb0 None None] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:03:13.279097 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-690d6579-73cc-4ad1-a169-6aa56fa41eb0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:03:13.310796 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.auth.core [None req-690d6579-73cc-4ad1-a169-6aa56fa41eb0 None None] MFA Rules not processed for user `891005fa5ff74041b75747d16cc66798`. Rule list: `[]` (Enabled: `True`). {{(pid=80378) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 02 10:03:13.312766 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.fernet_utils [None req-690d6579-73cc-4ad1-a169-6aa56fa41eb0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80378) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 10:03:13.337820 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f8d8a21-66f0-43b2-a184-0933a018bef3 None None] Validating token access rules against request {{(pid=80377) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:03:13.365185 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG dbcounter [-] [80377] Writing DB stats keystone:SELECT=100 {{(pid=80377) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 10:03:13.414979 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-931aaadc-cd49-44cb-a072-0b32fdad775e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '891005fa5ff74041b75747d16cc66798', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80377) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:03:13.416060 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-931aaadc-cd49-44cb-a072-0b32fdad775e None admin] REQUEST_METHOD: `GET` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:03:13.416576 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-931aaadc-cd49-44cb-a072-0b32fdad775e None admin] SCRIPT_NAME: `/identity` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:03:13.417052 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-931aaadc-cd49-44cb-a072-0b32fdad775e None admin] PATH_INFO: `/v3/services` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:03:13.417950 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-931aaadc-cd49-44cb-a072-0b32fdad775e None admin] RBAC: Adding query filter params (type=object-store) {{(pid=80377) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 02 10:03:13.418495 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-931aaadc-cd49-44cb-a072-0b32fdad775e None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:03:13.420229 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-931aaadc-cd49-44cb-a072-0b32fdad775e None admin] RBAC: Authorization granted {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:03:13.432132 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 454/924] 10.222.0.82 () {58 vars in 1023 bytes} [Tue Aug 2 10:03:13 2022] POST /identity/v3/auth/tokens => generated 1474 bytes in 155 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 02 10:03:13.432657 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: [pid: 80377|app: 0|req: 471/925] 10.222.0.82 () {56 vars in 1122 bytes} [Tue Aug 2 10:03:13 2022] GET /identity/v3/services?type=object-store => generated 356 bytes in 276 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 02 10:03:13.434347 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-690d6579-73cc-4ad1-a169-6aa56fa41eb0 None None] Authenticating user token {{(pid=80378) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:03:13.478398 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-931aaadc-cd49-44cb-a072-0b32fdad775e None admin] Authenticating user token {{(pid=80377) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:03:13.630174 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-690d6579-73cc-4ad1-a169-6aa56fa41eb0 None None] Validating token access rules against request {{(pid=80378) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:03:13.633096 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG dbcounter [-] [80378] Writing DB stats keystone:SELECT=100 {{(pid=80378) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 10:03:13.700617 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1dcd975e-c7ee-455a-a027-07f07b45bf3d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '891005fa5ff74041b75747d16cc66798', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80378) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:03:13.701812 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1dcd975e-c7ee-455a-a027-07f07b45bf3d None admin] REQUEST_METHOD: `GET` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:03:13.702224 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1dcd975e-c7ee-455a-a027-07f07b45bf3d None admin] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:03:13.702668 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1dcd975e-c7ee-455a-a027-07f07b45bf3d None admin] PATH_INFO: `/v3/endpoints` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:03:13.704313 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1dcd975e-c7ee-455a-a027-07f07b45bf3d None admin] RBAC: Adding query filter params (interface=public, service_id=afb73e70db0a46bd8e098a5c181d16cd, region_id=RegionOne) {{(pid=80378) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 02 10:03:13.704992 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1dcd975e-c7ee-455a-a027-07f07b45bf3d None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:03:13.708007 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1dcd975e-c7ee-455a-a027-07f07b45bf3d None admin] RBAC: Authorization granted {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:03:13.715655 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-931aaadc-cd49-44cb-a072-0b32fdad775e None admin] Validating token access rules against request {{(pid=80377) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:03:13.729030 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 455/926] 10.222.0.82 () {56 vars in 1251 bytes} [Tue Aug 2 10:03:13 2022] GET /identity/v3/endpoints?service_id=afb73e70db0a46bd8e098a5c181d16cd&interface=public®ion_id=RegionOne => generated 194 bytes in 289 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 02 10:03:13.735226 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1dcd975e-c7ee-455a-a027-07f07b45bf3d None admin] Authenticating user token {{(pid=80378) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:03:13.778549 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff91c813-6019-4927-b3e4-fe8d58aba6df None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '891005fa5ff74041b75747d16cc66798', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80377) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:03:13.780077 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff91c813-6019-4927-b3e4-fe8d58aba6df None admin] REQUEST_METHOD: `GET` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:03:13.781274 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff91c813-6019-4927-b3e4-fe8d58aba6df None admin] SCRIPT_NAME: `/identity` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:03:13.781881 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff91c813-6019-4927-b3e4-fe8d58aba6df None admin] PATH_INFO: `/v3/services/compute_legacy` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:03:13.783202 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff91c813-6019-4927-b3e4-fe8d58aba6df None admin] RBAC: Authorizing `identity:get_service(service_id=compute_legacy)` {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:03:13.785962 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff91c813-6019-4927-b3e4-fe8d58aba6df None admin] RBAC: Authorization granted {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:03:13.807244 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application [None req-ff91c813-6019-4927-b3e4-fe8d58aba6df None admin] Could not find service: compute_legacy.: keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Aug 02 10:03:13.807244 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 02 10:03:13.807244 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Aug 02 10:03:13.807244 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Aug 02 10:03:13.807244 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Aug 02 10:03:13.807244 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Aug 02 10:03:13.807244 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Aug 02 10:03:13.807244 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 02 10:03:13.807244 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Aug 02 10:03:13.807244 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application Aug 02 10:03:13.807244 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 02 10:03:13.807244 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application Aug 02 10:03:13.807244 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 02 10:03:13.807244 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 02 10:03:13.807244 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 02 10:03:13.807244 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 02 10:03:13.808573 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 02 10:03:13.808573 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 02 10:03:13.808573 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 02 10:03:13.808573 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 02 10:03:13.808573 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 02 10:03:13.808573 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 02 10:03:13.808573 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 02 10:03:13.808573 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Aug 02 10:03:13.808573 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application return self._get_service(service_id) Aug 02 10:03:13.808573 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Aug 02 10:03:13.808573 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Aug 02 10:03:13.808573 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 02 10:03:13.808573 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 02 10:03:13.808573 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 02 10:03:13.808573 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 02 10:03:13.808573 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 02 10:03:13.809642 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application return self.get_or_create( Aug 02 10:03:13.809642 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 02 10:03:13.809642 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application with Lock( Aug 02 10:03:13.809642 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 02 10:03:13.809642 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application return self._enter() Aug 02 10:03:13.809642 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 02 10:03:13.809642 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 02 10:03:13.809642 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 02 10:03:13.809642 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application return self.creator() Aug 02 10:03:13.809642 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 02 10:03:13.809642 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application created_value = creator( Aug 02 10:03:13.809642 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Aug 02 10:03:13.809642 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 02 10:03:13.809642 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Aug 02 10:03:13.809642 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application Aug 02 10:03:13.813838 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: [pid: 80377|app: 0|req: 472/927] 10.222.0.82 () {56 vars in 1132 bytes} [Tue Aug 2 10:03:13 2022] GET /identity/v3/services/compute_legacy => generated 95 bytes in 332 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 02 10:03:13.815534 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff91c813-6019-4927-b3e4-fe8d58aba6df None admin] Authenticating user token {{(pid=80377) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:03:13.879942 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG dbcounter [-] [80377] Writing DB stats keystone:SELECT=100 {{(pid=80377) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 10:03:13.902375 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1dcd975e-c7ee-455a-a027-07f07b45bf3d None admin] Validating token access rules against request {{(pid=80378) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:03:13.950995 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-378799db-9dd3-417f-a27d-d11cbde38df7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '891005fa5ff74041b75747d16cc66798', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80378) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:03:13.952179 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-378799db-9dd3-417f-a27d-d11cbde38df7 None admin] REQUEST_METHOD: `GET` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:03:13.952781 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-378799db-9dd3-417f-a27d-d11cbde38df7 None admin] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:03:13.953351 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-378799db-9dd3-417f-a27d-d11cbde38df7 None admin] PATH_INFO: `/v3/services` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:03:13.954461 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-378799db-9dd3-417f-a27d-d11cbde38df7 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:03:13.956877 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-378799db-9dd3-417f-a27d-d11cbde38df7 None admin] RBAC: Authorization granted {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:03:13.961199 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff91c813-6019-4927-b3e4-fe8d58aba6df None admin] Validating token access rules against request {{(pid=80377) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:03:13.970270 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 456/928] 10.222.0.82 () {56 vars in 1087 bytes} [Tue Aug 2 10:03:13 2022] GET /identity/v3/services => generated 1497 bytes in 240 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 02 10:03:14.014350 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-824ce52a-adfd-41c8-810f-d34e7f6eb457 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '891005fa5ff74041b75747d16cc66798', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80377) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:03:14.015323 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-824ce52a-adfd-41c8-810f-d34e7f6eb457 None admin] REQUEST_METHOD: `GET` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:03:14.015781 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-824ce52a-adfd-41c8-810f-d34e7f6eb457 None admin] SCRIPT_NAME: `/identity` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:03:14.016223 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-824ce52a-adfd-41c8-810f-d34e7f6eb457 None admin] PATH_INFO: `/v3/services` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:03:14.017983 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-824ce52a-adfd-41c8-810f-d34e7f6eb457 None admin] RBAC: Adding query filter params (name=compute_legacy) {{(pid=80377) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 02 10:03:14.018489 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-824ce52a-adfd-41c8-810f-d34e7f6eb457 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:03:14.020869 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-824ce52a-adfd-41c8-810f-d34e7f6eb457 None admin] RBAC: Authorization granted {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:03:14.029026 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: [pid: 80377|app: 0|req: 473/929] 10.222.0.82 () {56 vars in 1126 bytes} [Tue Aug 2 10:03:13 2022] GET /identity/v3/services?name=compute_legacy => generated 131 bytes in 214 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 02 10:03:14.036172 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-378799db-9dd3-417f-a27d-d11cbde38df7 None admin] Authenticating user token {{(pid=80378) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:03:14.078584 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG dbcounter [-] [80378] Writing DB stats keystone:SELECT=100 {{(pid=80378) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 10:03:14.183643 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-378799db-9dd3-417f-a27d-d11cbde38df7 None admin] Validating token access rules against request {{(pid=80378) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:03:14.244991 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8ad666a-8d10-42e7-a338-b0d2bc98525f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '891005fa5ff74041b75747d16cc66798', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80378) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:03:14.247020 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8ad666a-8d10-42e7-a338-b0d2bc98525f None admin] REQUEST_METHOD: `GET` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:03:14.252362 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8ad666a-8d10-42e7-a338-b0d2bc98525f None admin] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:03:14.252362 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8ad666a-8d10-42e7-a338-b0d2bc98525f None admin] PATH_INFO: `/v3/services` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:03:14.252362 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8ad666a-8d10-42e7-a338-b0d2bc98525f None admin] RBAC: Adding query filter params (type=compute_legacy) {{(pid=80378) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 02 10:03:14.252362 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8ad666a-8d10-42e7-a338-b0d2bc98525f None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:03:14.252362 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8ad666a-8d10-42e7-a338-b0d2bc98525f None admin] RBAC: Authorization granted {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:03:14.257698 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 457/930] 10.222.0.82 () {56 vars in 1126 bytes} [Tue Aug 2 10:03:14 2022] GET /identity/v3/services?type=compute_legacy => generated 386 bytes in 222 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 02 10:03:14.270145 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-824ce52a-adfd-41c8-810f-d34e7f6eb457 None admin] Authenticating user token {{(pid=80377) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:03:14.422019 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-824ce52a-adfd-41c8-810f-d34e7f6eb457 None admin] Validating token access rules against request {{(pid=80377) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:03:14.427526 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG dbcounter [-] [80377] Writing DB stats keystone:SELECT=100 {{(pid=80377) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 10:03:14.433441 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-935f6b55-73ad-4f33-add4-ff4a81334877 None None] REQUEST_METHOD: `GET` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:03:14.434253 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-935f6b55-73ad-4f33-add4-ff4a81334877 None None] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:03:14.434779 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-935f6b55-73ad-4f33-add4-ff4a81334877 None None] PATH_INFO: `/` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:03:14.436950 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 458/931] 10.222.0.82 () {54 vars in 926 bytes} [Tue Aug 2 10:03:14 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 02 10:03:14.444000 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8250ec7d-a946-4b53-a5c8-2c9be027ebf4 None None] REQUEST_METHOD: `POST` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:03:14.444952 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8250ec7d-a946-4b53-a5c8-2c9be027ebf4 None None] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:03:14.445484 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8250ec7d-a946-4b53-a5c8-2c9be027ebf4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:03:14.472174 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.auth.core [None req-8250ec7d-a946-4b53-a5c8-2c9be027ebf4 None None] MFA Rules not processed for user `891005fa5ff74041b75747d16cc66798`. Rule list: `[]` (Enabled: `True`). {{(pid=80378) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 02 10:03:14.474581 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.fernet_utils [None req-8250ec7d-a946-4b53-a5c8-2c9be027ebf4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80378) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 10:03:14.483433 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59d767fe-6c22-4978-a423-88ed62fe9c12 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '891005fa5ff74041b75747d16cc66798', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80377) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:03:14.484478 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59d767fe-6c22-4978-a423-88ed62fe9c12 None admin] REQUEST_METHOD: `POST` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:03:14.485185 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59d767fe-6c22-4978-a423-88ed62fe9c12 None admin] SCRIPT_NAME: `/identity` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:03:14.485852 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59d767fe-6c22-4978-a423-88ed62fe9c12 None admin] PATH_INFO: `/v3/endpoints` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:03:14.487746 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59d767fe-6c22-4978-a423-88ed62fe9c12 None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:03:14.489390 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59d767fe-6c22-4978-a423-88ed62fe9c12 None admin] RBAC: Authorization granted {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:03:14.516860 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: [pid: 80377|app: 0|req: 474/932] 10.222.0.82 () {60 vars in 1144 bytes} [Tue Aug 2 10:03:14 2022] POST /identity/v3/endpoints => generated 347 bytes in 248 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 02 10:03:14.609069 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 459/933] 10.222.0.82 () {58 vars in 1023 bytes} [Tue Aug 2 10:03:14 2022] POST /identity/v3/auth/tokens => generated 1474 bytes in 167 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Aug 02 10:03:14.638049 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59d767fe-6c22-4978-a423-88ed62fe9c12 None admin] Authenticating user token {{(pid=80377) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:03:14.818054 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59d767fe-6c22-4978-a423-88ed62fe9c12 None admin] Validating token access rules against request {{(pid=80377) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:03:14.868508 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dacab04b-31f4-46dd-9180-bda9a8ffc007 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '891005fa5ff74041b75747d16cc66798', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80377) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:03:14.869637 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dacab04b-31f4-46dd-9180-bda9a8ffc007 None admin] REQUEST_METHOD: `POST` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:03:14.870108 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dacab04b-31f4-46dd-9180-bda9a8ffc007 None admin] SCRIPT_NAME: `/identity` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:03:14.873180 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dacab04b-31f4-46dd-9180-bda9a8ffc007 None admin] PATH_INFO: `/v3/projects` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:03:14.873180 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dacab04b-31f4-46dd-9180-bda9a8ffc007 None admin] RBAC: Authorizing `identity:create_project()` {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:03:14.873180 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dacab04b-31f4-46dd-9180-bda9a8ffc007 None admin] RBAC: Authorization granted {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:03:14.876144 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: WARNING keystone.server.flask.common [None req-dacab04b-31f4-46dd-9180-bda9a8ffc007 None admin] Deprecated: Not specifying a domain during a create user, group or project call, and relying on falling back to the default domain, is deprecated as of Liberty. There is no plan to remove this compatibility, however, future API versions may remove this, so please specify the domain explicitly or use a domain-scoped token. Aug 02 10:03:14.904371 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.sql.core [None req-dacab04b-31f4-46dd-9180-bda9a8ffc007 None admin] Conflict project: (pymysql.err.IntegrityError) (1062, "Duplicate entry 'default-service' for key 'project.ixu_project_name_domain_id'") Aug 02 10:03:14.904371 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: [SQL: INSERT INTO project (id, name, domain_id, description, enabled, extra, parent_id, is_domain) VALUES (%(id)s, %(name)s, %(domain_id)s, %(description)s, %(enabled)s, %(extra)s, %(parent_id)s, %(is_domain)s)] Aug 02 10:03:14.904371 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: [parameters: {'id': 'e3f9ead86e9d4362954729de36c6adf4', 'name': 'service', 'domain_id': 'default', 'description': '', 'enabled': 1, 'extra': '{}', 'parent_id': 'default', 'is_domain': 0}] Aug 02 10:03:14.904371 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: (Background on this error at: https://sqlalche.me/e/14/gkpj) {{(pid=80377) wrapper /opt/stack/keystone/keystone/common/sql/core.py:533}} Aug 02 10:03:14.918805 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application [None req-dacab04b-31f4-46dd-9180-bda9a8ffc007 None admin] Conflict occurred attempting to store project - it is not permitted to have two projects with either the same name or same id in the same domain: name is service, project id e3f9ead86e9d4362954729de36c6adf4.: keystone.exception.Conflict: Conflict occurred attempting to store project - it is not permitted to have two projects with either the same name or same id in the same domain: name is service, project id e3f9ead86e9d4362954729de36c6adf4. Aug 02 10:03:14.918805 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 02 10:03:14.918805 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1819, in _execute_context Aug 02 10:03:14.918805 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application self.dialect.do_execute( Aug 02 10:03:14.918805 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/default.py", line 732, in do_execute Aug 02 10:03:14.918805 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application cursor.execute(statement, parameters) Aug 02 10:03:14.918805 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 148, in execute Aug 02 10:03:14.918805 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application result = self._query(query) Aug 02 10:03:14.918805 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 310, in _query Aug 02 10:03:14.918805 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application conn.query(q) Aug 02 10:03:14.918805 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 548, in query Aug 02 10:03:14.918805 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application self._affected_rows = self._read_query_result(unbuffered=unbuffered) Aug 02 10:03:14.918805 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 775, in _read_query_result Aug 02 10:03:14.919700 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application result.read() Aug 02 10:03:14.919700 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 1156, in read Aug 02 10:03:14.919700 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application first_packet = self.connection._read_packet() Aug 02 10:03:14.919700 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 725, in _read_packet Aug 02 10:03:14.919700 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application packet.raise_for_error() Aug 02 10:03:14.919700 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/protocol.py", line 221, in raise_for_error Aug 02 10:03:14.919700 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application err.raise_mysql_exception(self._data) Aug 02 10:03:14.919700 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/err.py", line 143, in raise_mysql_exception Aug 02 10:03:14.919700 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application raise errorclass(errno, errval) Aug 02 10:03:14.919700 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application pymysql.err.IntegrityError: (1062, "Duplicate entry 'default-service' for key 'project.ixu_project_name_domain_id'") Aug 02 10:03:14.919700 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application Aug 02 10:03:14.919700 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application The above exception was the direct cause of the following exception: Aug 02 10:03:14.919700 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application Aug 02 10:03:14.919700 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 02 10:03:14.919700 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/sql/core.py", line 528, in wrapper Aug 02 10:03:14.919700 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application return method(*args, **kwargs) Aug 02 10:03:14.919700 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 241, in create_project Aug 02 10:03:14.919700 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application return project_ref.to_dict() Aug 02 10:03:14.920654 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ Aug 02 10:03:14.920654 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application next(self.gen) Aug 02 10:03:14.920654 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 1060, in _transaction_scope Aug 02 10:03:14.920654 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application yield resource Aug 02 10:03:14.920654 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ Aug 02 10:03:14.920654 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application next(self.gen) Aug 02 10:03:14.920654 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 662, in _session Aug 02 10:03:14.920654 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application self.session.rollback() Aug 02 10:03:14.920654 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 227, in __exit__ Aug 02 10:03:14.920654 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application self.force_reraise() Aug 02 10:03:14.920654 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 200, in force_reraise Aug 02 10:03:14.920654 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application raise self.value Aug 02 10:03:14.920654 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 659, in _session Aug 02 10:03:14.920654 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application self._end_session_transaction(self.session) Aug 02 10:03:14.920654 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 687, in _end_session_transaction Aug 02 10:03:14.920654 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application session.commit() Aug 02 10:03:14.920654 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 1451, in commit Aug 02 10:03:14.921561 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application self._transaction.commit(_to_root=self.future) Aug 02 10:03:14.921561 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 829, in commit Aug 02 10:03:14.921561 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application self._prepare_impl() Aug 02 10:03:14.921561 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 808, in _prepare_impl Aug 02 10:03:14.921561 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application self.session.flush() Aug 02 10:03:14.921561 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3383, in flush Aug 02 10:03:14.921561 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application self._flush(objects) Aug 02 10:03:14.921561 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3523, in _flush Aug 02 10:03:14.921561 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application transaction.rollback(_capture_exception=True) Aug 02 10:03:14.921561 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/langhelpers.py", line 70, in __exit__ Aug 02 10:03:14.921561 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application compat.raise_( Aug 02 10:03:14.921561 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/compat.py", line 208, in raise_ Aug 02 10:03:14.921561 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application raise exception Aug 02 10:03:14.921561 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3483, in _flush Aug 02 10:03:14.921561 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application flush_context.execute() Aug 02 10:03:14.921561 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/unitofwork.py", line 456, in execute Aug 02 10:03:14.921561 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application rec.execute(self) Aug 02 10:03:14.922601 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/unitofwork.py", line 630, in execute Aug 02 10:03:14.922601 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application util.preloaded.orm_persistence.save_obj( Aug 02 10:03:14.922601 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/persistence.py", line 245, in save_obj Aug 02 10:03:14.922601 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application _emit_insert_statements( Aug 02 10:03:14.922601 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/persistence.py", line 1097, in _emit_insert_statements Aug 02 10:03:14.922601 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application c = connection._execute_20( Aug 02 10:03:14.922601 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1631, in _execute_20 Aug 02 10:03:14.922601 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application return meth(self, args_10style, kwargs_10style, execution_options) Aug 02 10:03:14.922601 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/sql/elements.py", line 332, in _execute_on_connection Aug 02 10:03:14.922601 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application return connection._execute_clauseelement( Aug 02 10:03:14.922601 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1498, in _execute_clauseelement Aug 02 10:03:14.922601 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application ret = self._execute_context( Aug 02 10:03:14.922601 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1862, in _execute_context Aug 02 10:03:14.922601 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application self._handle_dbapi_exception( Aug 02 10:03:14.922601 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 2041, in _handle_dbapi_exception Aug 02 10:03:14.925754 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application util.raise_(newraise, with_traceback=exc_info[2], from_=e) Aug 02 10:03:14.925754 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/compat.py", line 208, in raise_ Aug 02 10:03:14.925754 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application raise exception Aug 02 10:03:14.925754 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1819, in _execute_context Aug 02 10:03:14.925754 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application self.dialect.do_execute( Aug 02 10:03:14.925754 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/default.py", line 732, in do_execute Aug 02 10:03:14.925754 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application cursor.execute(statement, parameters) Aug 02 10:03:14.925754 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 148, in execute Aug 02 10:03:14.925754 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application result = self._query(query) Aug 02 10:03:14.925754 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 310, in _query Aug 02 10:03:14.925754 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application conn.query(q) Aug 02 10:03:14.925754 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 548, in query Aug 02 10:03:14.925754 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application self._affected_rows = self._read_query_result(unbuffered=unbuffered) Aug 02 10:03:14.925754 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 775, in _read_query_result Aug 02 10:03:14.925754 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application result.read() Aug 02 10:03:14.925754 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 1156, in read Aug 02 10:03:14.925754 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application first_packet = self.connection._read_packet() Aug 02 10:03:14.928072 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 725, in _read_packet Aug 02 10:03:14.928072 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application packet.raise_for_error() Aug 02 10:03:14.928072 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/protocol.py", line 221, in raise_for_error Aug 02 10:03:14.928072 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application err.raise_mysql_exception(self._data) Aug 02 10:03:14.928072 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/err.py", line 143, in raise_mysql_exception Aug 02 10:03:14.928072 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application raise errorclass(errno, errval) Aug 02 10:03:14.928072 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application oslo_db.exception.DBDuplicateEntry: (pymysql.err.IntegrityError) (1062, "Duplicate entry 'default-service' for key 'project.ixu_project_name_domain_id'") Aug 02 10:03:14.928072 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application [SQL: INSERT INTO project (id, name, domain_id, description, enabled, extra, parent_id, is_domain) VALUES (%(id)s, %(name)s, %(domain_id)s, %(description)s, %(enabled)s, %(extra)s, %(parent_id)s, %(is_domain)s)] Aug 02 10:03:14.928072 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application [parameters: {'id': 'e3f9ead86e9d4362954729de36c6adf4', 'name': 'service', 'domain_id': 'default', 'description': '', 'enabled': 1, 'extra': '{}', 'parent_id': 'default', 'is_domain': 0}] Aug 02 10:03:14.928072 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application (Background on this error at: https://sqlalche.me/e/14/gkpj) Aug 02 10:03:14.928072 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application Aug 02 10:03:14.928072 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 02 10:03:14.928072 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application Aug 02 10:03:14.928072 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 02 10:03:14.928072 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 216, in create_project Aug 02 10:03:14.929283 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application ret = self.driver.create_project(project_id, project) Aug 02 10:03:14.929283 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/sql/core.py", line 563, in wrapper Aug 02 10:03:14.929283 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application raise exception.Conflict(type=conflict_type, Aug 02 10:03:14.929283 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application keystone.exception.Conflict: Conflict occurred attempting to store project - Duplicate entry found with name service at domain ID default. Aug 02 10:03:14.929283 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application Aug 02 10:03:14.929283 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 02 10:03:14.929283 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application Aug 02 10:03:14.929283 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 02 10:03:14.929283 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 02 10:03:14.929283 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 02 10:03:14.929283 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 02 10:03:14.929283 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 02 10:03:14.929283 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 02 10:03:14.929283 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 02 10:03:14.929283 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 02 10:03:14.929283 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 02 10:03:14.929283 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 02 10:03:14.934144 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 02 10:03:14.934144 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 176, in post Aug 02 10:03:14.934144 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application ref = PROVIDERS.resource_api.create_project( Aug 02 10:03:14.934144 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 02 10:03:14.934144 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 02 10:03:14.934144 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 218, in create_project Aug 02 10:03:14.934144 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application raise exception.Conflict( Aug 02 10:03:14.934144 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application keystone.exception.Conflict: Conflict occurred attempting to store project - it is not permitted to have two projects with either the same name or same id in the same domain: name is service, project id e3f9ead86e9d4362954729de36c6adf4. Aug 02 10:03:14.934144 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application Aug 02 10:03:14.934144 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: [pid: 80377|app: 0|req: 475/934] 10.222.0.82 () {60 vars in 1140 bytes} [Tue Aug 2 10:03:14 2022] POST /identity/v3/projects => generated 262 bytes in 284 msecs (HTTP/1.1 409) 5 headers in 183 bytes (1 switches on core 0) Aug 02 10:03:14.934836 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8250ec7d-a946-4b53-a5c8-2c9be027ebf4 None None] Authenticating user token {{(pid=80378) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:03:14.968446 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG dbcounter [-] [80378] Writing DB stats keystone:SELECT=100 {{(pid=80378) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 10:03:15.073655 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8250ec7d-a946-4b53-a5c8-2c9be027ebf4 None None] Validating token access rules against request {{(pid=80378) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:03:15.119242 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6e71ccb-ed17-4bad-aef0-326eb02be222 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '891005fa5ff74041b75747d16cc66798', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80378) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:03:15.120793 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6e71ccb-ed17-4bad-aef0-326eb02be222 None admin] REQUEST_METHOD: `GET` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:03:15.121559 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6e71ccb-ed17-4bad-aef0-326eb02be222 None admin] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:03:15.122358 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6e71ccb-ed17-4bad-aef0-326eb02be222 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:03:15.132127 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6e71ccb-ed17-4bad-aef0-326eb02be222 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:03:15.134191 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6e71ccb-ed17-4bad-aef0-326eb02be222 None admin] RBAC: Authorization granted {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:03:15.147830 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application [None req-d6e71ccb-ed17-4bad-aef0-326eb02be222 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 02 10:03:15.147830 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 02 10:03:15.147830 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 02 10:03:15.147830 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 02 10:03:15.147830 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 02 10:03:15.147830 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 02 10:03:15.147830 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 02 10:03:15.147830 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 02 10:03:15.147830 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 02 10:03:15.147830 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 02 10:03:15.147830 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 02 10:03:15.147830 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 02 10:03:15.147830 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 02 10:03:15.147830 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 02 10:03:15.147830 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 02 10:03:15.148823 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 02 10:03:15.148823 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 02 10:03:15.148823 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 02 10:03:15.148823 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 02 10:03:15.148823 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 02 10:03:15.148823 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 02 10:03:15.148823 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application return self.get_or_create( Aug 02 10:03:15.148823 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 02 10:03:15.148823 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application with Lock( Aug 02 10:03:15.148823 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 02 10:03:15.148823 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application return self._enter() Aug 02 10:03:15.148823 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 02 10:03:15.148823 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 02 10:03:15.148823 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 02 10:03:15.148823 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application return self.creator() Aug 02 10:03:15.148823 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 02 10:03:15.148823 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application created_value = creator( Aug 02 10:03:15.149894 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 02 10:03:15.149894 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 02 10:03:15.149894 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 02 10:03:15.149894 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 02 10:03:15.149894 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 02 10:03:15.149894 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 02 10:03:15.149894 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 02 10:03:15.149894 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application Aug 02 10:03:15.153138 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 460/935] 10.222.0.82 () {56 vars in 1111 bytes} [Tue Aug 2 10:03:14 2022] GET /identity/v3/projects/service => generated 88 bytes in 224 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 02 10:03:15.162452 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dacab04b-31f4-46dd-9180-bda9a8ffc007 None admin] Authenticating user token {{(pid=80377) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:03:15.190605 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG dbcounter [-] [80377] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80377) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 10:03:15.302211 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dacab04b-31f4-46dd-9180-bda9a8ffc007 None admin] Validating token access rules against request {{(pid=80377) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:03:15.348847 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-275de738-7e3c-4d17-9fc1-1abed3c3df0d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '891005fa5ff74041b75747d16cc66798', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80377) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:03:15.350394 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-275de738-7e3c-4d17-9fc1-1abed3c3df0d None admin] REQUEST_METHOD: `GET` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:03:15.351289 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-275de738-7e3c-4d17-9fc1-1abed3c3df0d None admin] SCRIPT_NAME: `/identity` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:03:15.352076 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-275de738-7e3c-4d17-9fc1-1abed3c3df0d None admin] PATH_INFO: `/v3/projects` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:03:15.353732 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-275de738-7e3c-4d17-9fc1-1abed3c3df0d None admin] RBAC: Adding query filter params (name=service) {{(pid=80377) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 02 10:03:15.354402 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-275de738-7e3c-4d17-9fc1-1abed3c3df0d None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:03:15.356728 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-275de738-7e3c-4d17-9fc1-1abed3c3df0d None admin] RBAC: Authorization granted {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:03:15.383188 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: [pid: 80377|app: 0|req: 476/936] 10.222.0.82 () {56 vars in 1112 bytes} [Tue Aug 2 10:03:15 2022] GET /identity/v3/projects?name=service => generated 411 bytes in 222 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 02 10:03:15.919783 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-722200b8-9c45-4338-903d-e2ee50c38044 None None] REQUEST_METHOD: `GET` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:03:15.920303 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-722200b8-9c45-4338-903d-e2ee50c38044 None None] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:03:15.920545 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-722200b8-9c45-4338-903d-e2ee50c38044 None None] PATH_INFO: `/` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:03:15.922545 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 461/937] 10.222.0.82 () {54 vars in 926 bytes} [Tue Aug 2 10:03:15 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 02 10:03:15.929576 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd33f7ee-85bf-4a72-9ffd-691adedddb92 None None] REQUEST_METHOD: `POST` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:03:15.929938 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd33f7ee-85bf-4a72-9ffd-691adedddb92 None None] SCRIPT_NAME: `/identity` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:03:15.930300 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd33f7ee-85bf-4a72-9ffd-691adedddb92 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:03:15.957337 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.auth.core [None req-dd33f7ee-85bf-4a72-9ffd-691adedddb92 None None] MFA Rules not processed for user `891005fa5ff74041b75747d16cc66798`. Rule list: `[]` (Enabled: `True`). {{(pid=80377) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 02 10:03:15.960085 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.fernet_utils [None req-dd33f7ee-85bf-4a72-9ffd-691adedddb92 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80377) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 10:03:16.067006 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: [pid: 80377|app: 0|req: 477/938] 10.222.0.82 () {58 vars in 1023 bytes} [Tue Aug 2 10:03:15 2022] POST /identity/v3/auth/tokens => generated 1474 bytes in 138 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 02 10:03:16.099112 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-722200b8-9c45-4338-903d-e2ee50c38044 None None] Authenticating user token {{(pid=80378) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:03:16.246204 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG dbcounter [-] [80378] Writing DB stats keystone:SELECT=100 {{(pid=80378) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 10:03:16.279899 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-722200b8-9c45-4338-903d-e2ee50c38044 None None] Validating token access rules against request {{(pid=80378) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:03:16.326479 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d5a1693-0bbc-4b96-b7b3-f4bc8188b83c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '891005fa5ff74041b75747d16cc66798', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80378) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:03:16.327913 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d5a1693-0bbc-4b96-b7b3-f4bc8188b83c None admin] REQUEST_METHOD: `GET` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:03:16.328578 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d5a1693-0bbc-4b96-b7b3-f4bc8188b83c None admin] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:03:16.329164 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d5a1693-0bbc-4b96-b7b3-f4bc8188b83c None admin] PATH_INFO: `/v3/roles/ResellerAdmin` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:03:16.345207 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d5a1693-0bbc-4b96-b7b3-f4bc8188b83c None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 02 10:03:16.351674 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application [None req-3d5a1693-0bbc-4b96-b7b3-f4bc8188b83c None admin] Could not find role: ResellerAdmin.: keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Aug 02 10:03:16.351674 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 02 10:03:16.351674 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 02 10:03:16.351674 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 02 10:03:16.351674 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 02 10:03:16.351674 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 02 10:03:16.351674 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 02 10:03:16.351674 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 02 10:03:16.351674 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 02 10:03:16.351674 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 02 10:03:16.351674 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 02 10:03:16.351674 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 02 10:03:16.351674 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 02 10:03:16.351674 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 02 10:03:16.351674 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 02 10:03:16.351674 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 02 10:03:16.352879 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 02 10:03:16.352879 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 02 10:03:16.352879 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 02 10:03:16.352879 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 02 10:03:16.352879 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 02 10:03:16.352879 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 02 10:03:16.352879 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 02 10:03:16.352879 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 02 10:03:16.352879 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 02 10:03:16.352879 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application return self.get_or_create( Aug 02 10:03:16.352879 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 02 10:03:16.352879 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application with Lock( Aug 02 10:03:16.352879 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 02 10:03:16.352879 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application return self._enter() Aug 02 10:03:16.352879 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 02 10:03:16.352879 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 02 10:03:16.354260 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 02 10:03:16.354260 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application return self.creator() Aug 02 10:03:16.354260 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 02 10:03:16.354260 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application created_value = creator( Aug 02 10:03:16.354260 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 02 10:03:16.354260 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 02 10:03:16.354260 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 02 10:03:16.354260 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 02 10:03:16.354260 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 02 10:03:16.354260 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 02 10:03:16.354260 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Aug 02 10:03:16.354260 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application Aug 02 10:03:16.357970 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 462/939] 10.222.0.82 () {56 vars in 1120 bytes} [Tue Aug 2 10:03:16 2022] GET /identity/v3/roles/ResellerAdmin => generated 91 bytes in 260 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 02 10:03:16.366444 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd33f7ee-85bf-4a72-9ffd-691adedddb92 None None] Authenticating user token {{(pid=80377) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:03:16.379118 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG dbcounter [-] [80377] Writing DB stats keystone:SELECT=100 {{(pid=80377) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 10:03:16.527754 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd33f7ee-85bf-4a72-9ffd-691adedddb92 None None] Validating token access rules against request {{(pid=80377) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:03:16.588455 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b8a56d3-c9d7-42b1-ba38-ef5e31577fb0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '891005fa5ff74041b75747d16cc66798', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80377) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:03:16.589501 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b8a56d3-c9d7-42b1-ba38-ef5e31577fb0 None admin] REQUEST_METHOD: `GET` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:03:16.589974 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b8a56d3-c9d7-42b1-ba38-ef5e31577fb0 None admin] SCRIPT_NAME: `/identity` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:03:16.590438 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b8a56d3-c9d7-42b1-ba38-ef5e31577fb0 None admin] PATH_INFO: `/v3/roles` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:03:16.591683 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b8a56d3-c9d7-42b1-ba38-ef5e31577fb0 None admin] RBAC: Adding query filter params (name=ResellerAdmin) {{(pid=80377) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 02 10:03:16.592188 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b8a56d3-c9d7-42b1-ba38-ef5e31577fb0 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:03:16.594568 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b8a56d3-c9d7-42b1-ba38-ef5e31577fb0 None admin] RBAC: Authorization granted {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:03:16.611765 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: [pid: 80377|app: 0|req: 478/940] 10.222.0.82 () {56 vars in 1115 bytes} [Tue Aug 2 10:03:16 2022] GET /identity/v3/roles?name=ResellerAdmin => generated 338 bytes in 246 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 02 10:03:16.619233 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d5a1693-0bbc-4b96-b7b3-f4bc8188b83c None admin] Authenticating user token {{(pid=80378) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:03:16.757245 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1aff341-bb72-4f95-b7eb-5706432e0a74 None None] REQUEST_METHOD: `GET` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:03:16.757577 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1aff341-bb72-4f95-b7eb-5706432e0a74 None None] SCRIPT_NAME: `/identity` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:03:16.757834 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1aff341-bb72-4f95-b7eb-5706432e0a74 None None] PATH_INFO: `/` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:03:16.760068 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: [pid: 80377|app: 0|req: 479/941] 10.222.0.82 () {54 vars in 926 bytes} [Tue Aug 2 10:03:16 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 02 10:03:16.766256 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc8026c9-bf24-4dd8-a00f-1a9d6549e9eb None None] REQUEST_METHOD: `POST` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:03:16.766526 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc8026c9-bf24-4dd8-a00f-1a9d6549e9eb None None] SCRIPT_NAME: `/identity` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:03:16.766874 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc8026c9-bf24-4dd8-a00f-1a9d6549e9eb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:03:16.778379 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d5a1693-0bbc-4b96-b7b3-f4bc8188b83c None admin] Validating token access rules against request {{(pid=80378) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:03:16.794373 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.auth.core [None req-dc8026c9-bf24-4dd8-a00f-1a9d6549e9eb None None] MFA Rules not processed for user `891005fa5ff74041b75747d16cc66798`. Rule list: `[]` (Enabled: `True`). {{(pid=80377) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 02 10:03:16.796137 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.fernet_utils [None req-dc8026c9-bf24-4dd8-a00f-1a9d6549e9eb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80377) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 10:03:16.838435 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c1915c1-624b-4977-bb3a-00ee953a42c9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '891005fa5ff74041b75747d16cc66798', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80378) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:03:16.840302 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c1915c1-624b-4977-bb3a-00ee953a42c9 None admin] REQUEST_METHOD: `GET` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:03:16.840934 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c1915c1-624b-4977-bb3a-00ee953a42c9 None admin] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:03:16.841393 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c1915c1-624b-4977-bb3a-00ee953a42c9 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:03:16.857419 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG dbcounter [-] [80378] Writing DB stats keystone:SELECT=100 {{(pid=80378) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 10:03:16.861833 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c1915c1-624b-4977-bb3a-00ee953a42c9 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:03:16.864429 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c1915c1-624b-4977-bb3a-00ee953a42c9 None admin] RBAC: Authorization granted {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:03:16.893105 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG dbcounter [-] [80377] Writing DB stats keystone:SELECT=100 {{(pid=80377) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 10:03:16.894828 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application [None req-6c1915c1-624b-4977-bb3a-00ee953a42c9 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 02 10:03:16.894828 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 02 10:03:16.894828 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 02 10:03:16.894828 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 02 10:03:16.894828 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 02 10:03:16.894828 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 02 10:03:16.894828 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 02 10:03:16.894828 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 02 10:03:16.894828 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 02 10:03:16.894828 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 02 10:03:16.894828 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 02 10:03:16.894828 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 02 10:03:16.894828 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 02 10:03:16.894828 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 02 10:03:16.894828 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 02 10:03:16.895812 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 02 10:03:16.895812 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 02 10:03:16.895812 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 02 10:03:16.895812 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 02 10:03:16.895812 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 02 10:03:16.895812 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 02 10:03:16.895812 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application return self.get_or_create( Aug 02 10:03:16.895812 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 02 10:03:16.895812 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application with Lock( Aug 02 10:03:16.895812 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 02 10:03:16.895812 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application return self._enter() Aug 02 10:03:16.895812 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 02 10:03:16.895812 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 02 10:03:16.895812 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 02 10:03:16.895812 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application return self.creator() Aug 02 10:03:16.895812 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 02 10:03:16.895812 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application created_value = creator( Aug 02 10:03:16.896798 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 02 10:03:16.896798 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 02 10:03:16.896798 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 02 10:03:16.896798 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application Aug 02 10:03:16.897867 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 463/942] 10.222.0.82 () {56 vars in 1108 bytes} [Tue Aug 2 10:03:16 2022] GET /identity/v3/domains/Default => generated 87 bytes in 278 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 02 10:03:16.902865 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c1915c1-624b-4977-bb3a-00ee953a42c9 None admin] Authenticating user token {{(pid=80378) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:03:16.912264 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: [pid: 80377|app: 0|req: 480/943] 10.222.0.82 () {58 vars in 1023 bytes} [Tue Aug 2 10:03:16 2022] POST /identity/v3/auth/tokens => generated 1474 bytes in 147 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Aug 02 10:03:16.944153 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc8026c9-bf24-4dd8-a00f-1a9d6549e9eb None None] Authenticating user token {{(pid=80377) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:03:17.051549 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c1915c1-624b-4977-bb3a-00ee953a42c9 None admin] Validating token access rules against request {{(pid=80378) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:03:17.098760 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc8026c9-bf24-4dd8-a00f-1a9d6549e9eb None None] Validating token access rules against request {{(pid=80377) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:03:17.105688 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c7d12ce-8e1d-4a3d-9d2a-f828c4746660 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '891005fa5ff74041b75747d16cc66798', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80378) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:03:17.106520 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c7d12ce-8e1d-4a3d-9d2a-f828c4746660 None admin] REQUEST_METHOD: `GET` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:03:17.106943 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c7d12ce-8e1d-4a3d-9d2a-f828c4746660 None admin] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:03:17.107229 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c7d12ce-8e1d-4a3d-9d2a-f828c4746660 None admin] PATH_INFO: `/v3/domains` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:03:17.108186 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c7d12ce-8e1d-4a3d-9d2a-f828c4746660 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80378) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 02 10:03:17.108546 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c7d12ce-8e1d-4a3d-9d2a-f828c4746660 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:03:17.110722 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c7d12ce-8e1d-4a3d-9d2a-f828c4746660 None admin] RBAC: Authorization granted {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:03:17.133414 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 464/944] 10.222.0.82 () {56 vars in 1109 bytes} [Tue Aug 2 10:03:16 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 226 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 02 10:03:17.134474 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c7d12ce-8e1d-4a3d-9d2a-f828c4746660 None admin] Authenticating user token {{(pid=80378) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:03:17.149208 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af87f9b0-1c58-4ec9-b3a4-744674a17e3f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '891005fa5ff74041b75747d16cc66798', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80377) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:03:17.150070 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af87f9b0-1c58-4ec9-b3a4-744674a17e3f None admin] REQUEST_METHOD: `GET` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:03:17.150639 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af87f9b0-1c58-4ec9-b3a4-744674a17e3f None admin] SCRIPT_NAME: `/identity` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:03:17.151153 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af87f9b0-1c58-4ec9-b3a4-744674a17e3f None admin] PATH_INFO: `/v3/services/object-store` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:03:17.152376 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af87f9b0-1c58-4ec9-b3a4-744674a17e3f None admin] RBAC: Authorizing `identity:get_service(service_id=object-store)` {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:03:17.154752 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af87f9b0-1c58-4ec9-b3a4-744674a17e3f None admin] RBAC: Authorization granted {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:03:17.170381 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application [None req-af87f9b0-1c58-4ec9-b3a4-744674a17e3f None admin] Could not find service: object-store.: keystone.exception.ServiceNotFound: Could not find service: object-store. Aug 02 10:03:17.170381 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 02 10:03:17.170381 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Aug 02 10:03:17.170381 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Aug 02 10:03:17.170381 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Aug 02 10:03:17.170381 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Aug 02 10:03:17.170381 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Aug 02 10:03:17.170381 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 02 10:03:17.170381 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Aug 02 10:03:17.170381 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application Aug 02 10:03:17.170381 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 02 10:03:17.170381 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application Aug 02 10:03:17.170381 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 02 10:03:17.170381 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 02 10:03:17.170381 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 02 10:03:17.170381 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 02 10:03:17.170381 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 02 10:03:17.172158 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 02 10:03:17.172158 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 02 10:03:17.172158 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 02 10:03:17.172158 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 02 10:03:17.172158 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 02 10:03:17.172158 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 02 10:03:17.172158 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Aug 02 10:03:17.172158 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application return self._get_service(service_id) Aug 02 10:03:17.172158 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Aug 02 10:03:17.172158 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Aug 02 10:03:17.172158 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 02 10:03:17.172158 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 02 10:03:17.172158 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 02 10:03:17.172158 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 02 10:03:17.172158 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 02 10:03:17.172158 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application return self.get_or_create( Aug 02 10:03:17.175570 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 02 10:03:17.175570 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application with Lock( Aug 02 10:03:17.175570 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 02 10:03:17.175570 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application return self._enter() Aug 02 10:03:17.175570 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 02 10:03:17.175570 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 02 10:03:17.175570 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 02 10:03:17.175570 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application return self.creator() Aug 02 10:03:17.175570 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 02 10:03:17.175570 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application created_value = creator( Aug 02 10:03:17.175570 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Aug 02 10:03:17.175570 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 02 10:03:17.175570 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Aug 02 10:03:17.175570 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application Aug 02 10:03:17.181958 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: [pid: 80377|app: 0|req: 481/945] 10.222.0.82 () {56 vars in 1126 bytes} [Tue Aug 2 10:03:16 2022] GET /identity/v3/services/object-store => generated 93 bytes in 229 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 02 10:03:17.184553 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af87f9b0-1c58-4ec9-b3a4-744674a17e3f None admin] Authenticating user token {{(pid=80377) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:03:17.213711 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG dbcounter [-] [80378] Writing DB stats keystone:SELECT=100 {{(pid=80378) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 10:03:17.283836 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG dbcounter [-] [80377] Writing DB stats keystone:SELECT=100 {{(pid=80377) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 10:03:17.288363 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c7d12ce-8e1d-4a3d-9d2a-f828c4746660 None admin] Validating token access rules against request {{(pid=80378) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:03:17.351279 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6213585-0613-4a8a-acbd-37e01019c1d1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '891005fa5ff74041b75747d16cc66798', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80378) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:03:17.352880 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6213585-0613-4a8a-acbd-37e01019c1d1 None admin] REQUEST_METHOD: `GET` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:03:17.353421 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6213585-0613-4a8a-acbd-37e01019c1d1 None admin] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:03:17.354022 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6213585-0613-4a8a-acbd-37e01019c1d1 None admin] PATH_INFO: `/v3/users/glance-swift` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:03:17.366797 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af87f9b0-1c58-4ec9-b3a4-744674a17e3f None admin] Validating token access rules against request {{(pid=80377) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:03:17.368994 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6213585-0613-4a8a-acbd-37e01019c1d1 None admin] RBAC: Authorizing `identity:get_user(user_id=glance-swift)` {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:03:17.371389 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6213585-0613-4a8a-acbd-37e01019c1d1 None admin] RBAC: Authorization granted {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:03:17.399011 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application [None req-b6213585-0613-4a8a-acbd-37e01019c1d1 None admin] Could not find user: glance-swift.: keystone.exception.UserNotFound: Could not find user: glance-swift. Aug 02 10:03:17.399011 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 02 10:03:17.399011 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 02 10:03:17.399011 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 02 10:03:17.399011 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 02 10:03:17.399011 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 02 10:03:17.399011 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 02 10:03:17.399011 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 02 10:03:17.399011 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 02 10:03:17.399011 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 02 10:03:17.399011 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 02 10:03:17.399011 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 02 10:03:17.399011 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 02 10:03:17.399011 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 02 10:03:17.399011 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 02 10:03:17.399011 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 02 10:03:17.400185 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 02 10:03:17.400185 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 02 10:03:17.400185 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 02 10:03:17.400185 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 02 10:03:17.400185 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 02 10:03:17.400185 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 02 10:03:17.400185 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 02 10:03:17.400185 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 02 10:03:17.400185 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 02 10:03:17.400185 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application return self.get_or_create( Aug 02 10:03:17.400185 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 02 10:03:17.400185 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application with Lock( Aug 02 10:03:17.400185 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 02 10:03:17.400185 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application return self._enter() Aug 02 10:03:17.400185 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 02 10:03:17.400185 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 02 10:03:17.400185 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 02 10:03:17.400185 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application return self.creator() Aug 02 10:03:17.401247 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 02 10:03:17.401247 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application created_value = creator( Aug 02 10:03:17.401247 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 02 10:03:17.401247 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 02 10:03:17.401247 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 02 10:03:17.401247 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 02 10:03:17.401247 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 02 10:03:17.401247 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 02 10:03:17.401247 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance-swift. Aug 02 10:03:17.401247 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application Aug 02 10:03:17.403184 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 465/946] 10.222.0.82 () {56 vars in 1117 bytes} [Tue Aug 2 10:03:17 2022] GET /identity/v3/users/glance-swift => generated 90 bytes in 269 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 02 10:03:17.406117 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25cc9c74-a1ad-42ac-85b6-47155cb2cf9c None None] REQUEST_METHOD: `GET` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:03:17.406117 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25cc9c74-a1ad-42ac-85b6-47155cb2cf9c None None] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:03:17.406117 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25cc9c74-a1ad-42ac-85b6-47155cb2cf9c None None] PATH_INFO: `/` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:03:17.407927 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 466/947] 10.222.0.82 () {54 vars in 926 bytes} [Tue Aug 2 10:03:17 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 02 10:03:17.410199 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25cc9c74-a1ad-42ac-85b6-47155cb2cf9c None None] Authenticating user token {{(pid=80378) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:03:17.431360 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfb202d1-3352-4893-a7a7-0f37a89edd8f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '891005fa5ff74041b75747d16cc66798', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80377) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:03:17.432082 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfb202d1-3352-4893-a7a7-0f37a89edd8f None admin] REQUEST_METHOD: `GET` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:03:17.432423 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfb202d1-3352-4893-a7a7-0f37a89edd8f None admin] SCRIPT_NAME: `/identity` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:03:17.433001 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfb202d1-3352-4893-a7a7-0f37a89edd8f None admin] PATH_INFO: `/v3/services` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:03:17.433495 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dfb202d1-3352-4893-a7a7-0f37a89edd8f None admin] RBAC: Adding query filter params (name=object-store) {{(pid=80377) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 02 10:03:17.433892 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dfb202d1-3352-4893-a7a7-0f37a89edd8f None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:03:17.435530 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dfb202d1-3352-4893-a7a7-0f37a89edd8f None admin] RBAC: Authorization granted {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:03:17.449618 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: [pid: 80377|app: 0|req: 482/948] 10.222.0.82 () {56 vars in 1122 bytes} [Tue Aug 2 10:03:17 2022] GET /identity/v3/services?name=object-store => generated 129 bytes in 266 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 02 10:03:17.452280 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-faf43760-e404-4ced-9640-a47dcb71cb24 None None] REQUEST_METHOD: `POST` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:03:17.452590 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-faf43760-e404-4ced-9640-a47dcb71cb24 None None] SCRIPT_NAME: `/identity` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:03:17.453086 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-faf43760-e404-4ced-9640-a47dcb71cb24 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:03:17.481516 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.auth.core [None req-faf43760-e404-4ced-9640-a47dcb71cb24 None None] MFA Rules not processed for user `891005fa5ff74041b75747d16cc66798`. Rule list: `[]` (Enabled: `True`). {{(pid=80377) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 02 10:03:17.483485 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.fernet_utils [None req-faf43760-e404-4ced-9640-a47dcb71cb24 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80377) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 10:03:17.599925 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: [pid: 80377|app: 0|req: 483/949] 10.222.0.82 () {58 vars in 1023 bytes} [Tue Aug 2 10:03:17 2022] POST /identity/v3/auth/tokens => generated 1474 bytes in 150 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 02 10:03:17.601223 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-faf43760-e404-4ced-9640-a47dcb71cb24 None None] Authenticating user token {{(pid=80377) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:03:17.630839 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25cc9c74-a1ad-42ac-85b6-47155cb2cf9c None None] Validating token access rules against request {{(pid=80378) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:03:17.656246 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG dbcounter [-] [80378] Writing DB stats keystone:SELECT=100 {{(pid=80378) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 10:03:17.697311 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8de56299-c97c-44d8-937d-f49e50cabd2f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '891005fa5ff74041b75747d16cc66798', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80378) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:03:17.698028 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8de56299-c97c-44d8-937d-f49e50cabd2f None admin] REQUEST_METHOD: `GET` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:03:17.698352 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8de56299-c97c-44d8-937d-f49e50cabd2f None admin] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:03:17.698687 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8de56299-c97c-44d8-937d-f49e50cabd2f None admin] PATH_INFO: `/v3/users` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:03:17.699744 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8de56299-c97c-44d8-937d-f49e50cabd2f None admin] RBAC: Adding query filter params (domain_id=default, name=glance-swift) {{(pid=80378) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 02 10:03:17.700439 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8de56299-c97c-44d8-937d-f49e50cabd2f None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:03:17.703291 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8de56299-c97c-44d8-937d-f49e50cabd2f None admin] RBAC: Authorization granted {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:03:17.735657 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 467/950] 10.222.0.82 () {56 vars in 1149 bytes} [Tue Aug 2 10:03:17 2022] GET /identity/v3/users?domain_id=default&name=glance-swift => generated 384 bytes in 320 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 02 10:03:17.735657 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8de56299-c97c-44d8-937d-f49e50cabd2f None admin] Authenticating user token {{(pid=80378) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:03:17.751280 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG dbcounter [-] [80377] Writing DB stats keystone:SELECT=100 {{(pid=80377) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 10:03:17.844239 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-faf43760-e404-4ced-9640-a47dcb71cb24 None None] Validating token access rules against request {{(pid=80377) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:03:17.910551 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8de56299-c97c-44d8-937d-f49e50cabd2f None admin] Validating token access rules against request {{(pid=80378) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:03:17.913595 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c06f517-a096-4e09-b0d5-91220cc8dc4c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '891005fa5ff74041b75747d16cc66798', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80377) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:03:17.914694 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c06f517-a096-4e09-b0d5-91220cc8dc4c None admin] REQUEST_METHOD: `GET` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:03:17.915187 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c06f517-a096-4e09-b0d5-91220cc8dc4c None admin] SCRIPT_NAME: `/identity` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:03:17.915688 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c06f517-a096-4e09-b0d5-91220cc8dc4c None admin] PATH_INFO: `/v3/services` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:03:17.916943 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c06f517-a096-4e09-b0d5-91220cc8dc4c None admin] RBAC: Adding query filter params (type=object-store) {{(pid=80377) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 02 10:03:17.917817 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c06f517-a096-4e09-b0d5-91220cc8dc4c None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:03:17.920114 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c06f517-a096-4e09-b0d5-91220cc8dc4c None admin] RBAC: Authorization granted {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:03:17.930882 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: [pid: 80377|app: 0|req: 484/951] 10.222.0.82 () {56 vars in 1122 bytes} [Tue Aug 2 10:03:17 2022] GET /identity/v3/services?type=object-store => generated 356 bytes in 330 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 02 10:03:17.932380 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c06f517-a096-4e09-b0d5-91220cc8dc4c None admin] Authenticating user token {{(pid=80377) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:03:17.972900 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2a1e833-a56f-4b1d-8223-9237b0e1c8f1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '891005fa5ff74041b75747d16cc66798', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80378) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:03:17.974099 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2a1e833-a56f-4b1d-8223-9237b0e1c8f1 None admin] REQUEST_METHOD: `GET` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:03:17.974598 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2a1e833-a56f-4b1d-8223-9237b0e1c8f1 None admin] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:03:17.975074 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2a1e833-a56f-4b1d-8223-9237b0e1c8f1 None admin] PATH_INFO: `/v3/services/compute` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:03:17.976255 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2a1e833-a56f-4b1d-8223-9237b0e1c8f1 None admin] RBAC: Authorizing `identity:get_service(service_id=compute)` {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:03:17.978976 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2a1e833-a56f-4b1d-8223-9237b0e1c8f1 None admin] RBAC: Authorization granted {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:03:18.000861 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application [None req-a2a1e833-a56f-4b1d-8223-9237b0e1c8f1 None admin] Could not find service: compute.: keystone.exception.ServiceNotFound: Could not find service: compute. Aug 02 10:03:18.000861 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 02 10:03:18.000861 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Aug 02 10:03:18.000861 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Aug 02 10:03:18.000861 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Aug 02 10:03:18.000861 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Aug 02 10:03:18.000861 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Aug 02 10:03:18.000861 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 02 10:03:18.000861 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute. Aug 02 10:03:18.000861 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application Aug 02 10:03:18.000861 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 02 10:03:18.000861 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application Aug 02 10:03:18.000861 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 02 10:03:18.000861 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 02 10:03:18.000861 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 02 10:03:18.000861 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 02 10:03:18.000861 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 02 10:03:18.003761 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 02 10:03:18.003761 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 02 10:03:18.003761 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 02 10:03:18.003761 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 02 10:03:18.003761 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 02 10:03:18.003761 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 02 10:03:18.003761 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Aug 02 10:03:18.003761 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application return self._get_service(service_id) Aug 02 10:03:18.003761 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Aug 02 10:03:18.003761 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Aug 02 10:03:18.003761 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 02 10:03:18.003761 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 02 10:03:18.003761 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 02 10:03:18.003761 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 02 10:03:18.003761 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 02 10:03:18.003761 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application return self.get_or_create( Aug 02 10:03:18.005435 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 02 10:03:18.005435 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application with Lock( Aug 02 10:03:18.005435 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 02 10:03:18.005435 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application return self._enter() Aug 02 10:03:18.005435 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 02 10:03:18.005435 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 02 10:03:18.005435 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 02 10:03:18.005435 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application return self.creator() Aug 02 10:03:18.005435 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 02 10:03:18.005435 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application created_value = creator( Aug 02 10:03:18.005435 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Aug 02 10:03:18.005435 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 02 10:03:18.005435 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute. Aug 02 10:03:18.005435 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application Aug 02 10:03:18.005435 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 468/952] 10.222.0.82 () {56 vars in 1111 bytes} [Tue Aug 2 10:03:17 2022] GET /identity/v3/services/compute => generated 88 bytes in 274 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 02 10:03:18.008366 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2a1e833-a56f-4b1d-8223-9237b0e1c8f1 None admin] Authenticating user token {{(pid=80378) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:03:18.065680 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG dbcounter [-] [80378] Writing DB stats keystone:SELECT=100 {{(pid=80378) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 10:03:18.136666 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c06f517-a096-4e09-b0d5-91220cc8dc4c None admin] Validating token access rules against request {{(pid=80377) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:03:18.170960 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG dbcounter [-] [80377] Writing DB stats keystone:SELECT=100 {{(pid=80377) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 10:03:18.190423 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22f37ef8-b5cd-4bc1-8651-12937caa80e0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '891005fa5ff74041b75747d16cc66798', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80377) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:03:18.191241 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22f37ef8-b5cd-4bc1-8651-12937caa80e0 None admin] REQUEST_METHOD: `GET` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:03:18.191637 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22f37ef8-b5cd-4bc1-8651-12937caa80e0 None admin] SCRIPT_NAME: `/identity` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:03:18.192044 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22f37ef8-b5cd-4bc1-8651-12937caa80e0 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:03:18.210950 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22f37ef8-b5cd-4bc1-8651-12937caa80e0 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:03:18.213305 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22f37ef8-b5cd-4bc1-8651-12937caa80e0 None admin] RBAC: Authorization granted {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:03:18.214060 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2a1e833-a56f-4b1d-8223-9237b0e1c8f1 None admin] Validating token access rules against request {{(pid=80378) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:03:18.240366 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application [None req-22f37ef8-b5cd-4bc1-8651-12937caa80e0 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 02 10:03:18.240366 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 02 10:03:18.240366 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 02 10:03:18.240366 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 02 10:03:18.240366 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 02 10:03:18.240366 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 02 10:03:18.240366 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 02 10:03:18.240366 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 02 10:03:18.240366 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 02 10:03:18.240366 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 02 10:03:18.240366 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 02 10:03:18.240366 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 02 10:03:18.240366 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 02 10:03:18.240366 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 02 10:03:18.240366 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 02 10:03:18.241424 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 02 10:03:18.241424 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 02 10:03:18.241424 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 02 10:03:18.241424 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 02 10:03:18.241424 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 02 10:03:18.241424 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 02 10:03:18.241424 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application return self.get_or_create( Aug 02 10:03:18.241424 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 02 10:03:18.241424 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application with Lock( Aug 02 10:03:18.241424 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 02 10:03:18.241424 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application return self._enter() Aug 02 10:03:18.241424 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 02 10:03:18.241424 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 02 10:03:18.241424 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 02 10:03:18.241424 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application return self.creator() Aug 02 10:03:18.241424 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 02 10:03:18.241424 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application created_value = creator( Aug 02 10:03:18.242486 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 02 10:03:18.242486 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 02 10:03:18.242486 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 02 10:03:18.242486 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application Aug 02 10:03:18.242486 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: [pid: 80377|app: 0|req: 485/953] 10.222.0.82 () {56 vars in 1108 bytes} [Tue Aug 2 10:03:17 2022] GET /identity/v3/domains/Default => generated 87 bytes in 311 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 02 10:03:18.242856 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22f37ef8-b5cd-4bc1-8651-12937caa80e0 None admin] Authenticating user token {{(pid=80377) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:03:18.288336 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-643fdda0-d8f7-4864-ade9-514cef56dd81 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '891005fa5ff74041b75747d16cc66798', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80378) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:03:18.289388 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-643fdda0-d8f7-4864-ade9-514cef56dd81 None admin] REQUEST_METHOD: `POST` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:03:18.289880 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-643fdda0-d8f7-4864-ade9-514cef56dd81 None admin] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:03:18.290329 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-643fdda0-d8f7-4864-ade9-514cef56dd81 None admin] PATH_INFO: `/v3/endpoints` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:03:18.291613 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-643fdda0-d8f7-4864-ade9-514cef56dd81 None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:03:18.294146 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-643fdda0-d8f7-4864-ade9-514cef56dd81 None admin] RBAC: Authorization granted {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:03:18.336635 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 469/954] 10.222.0.82 () {60 vars in 1144 bytes} [Tue Aug 2 10:03:18 2022] POST /identity/v3/endpoints => generated 349 bytes in 328 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 02 10:03:18.336635 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62a4e200-8a32-47f3-b91c-56812c68a3f6 None None] REQUEST_METHOD: `GET` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:03:18.336635 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62a4e200-8a32-47f3-b91c-56812c68a3f6 None None] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:03:18.336635 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62a4e200-8a32-47f3-b91c-56812c68a3f6 None None] PATH_INFO: `/` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:03:18.338212 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 470/955] 10.222.0.82 () {54 vars in 926 bytes} [Tue Aug 2 10:03:18 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 02 10:03:18.339026 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62a4e200-8a32-47f3-b91c-56812c68a3f6 None None] Authenticating user token {{(pid=80378) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:03:18.429526 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22f37ef8-b5cd-4bc1-8651-12937caa80e0 None admin] Validating token access rules against request {{(pid=80377) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:03:18.510109 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aba741b9-be3c-4178-98cb-aae0251bf334 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '891005fa5ff74041b75747d16cc66798', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80377) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:03:18.511121 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aba741b9-be3c-4178-98cb-aae0251bf334 None admin] REQUEST_METHOD: `GET` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:03:18.511655 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aba741b9-be3c-4178-98cb-aae0251bf334 None admin] SCRIPT_NAME: `/identity` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:03:18.512093 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aba741b9-be3c-4178-98cb-aae0251bf334 None admin] PATH_INFO: `/v3/services` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:03:18.513200 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aba741b9-be3c-4178-98cb-aae0251bf334 None admin] RBAC: Adding query filter params (name=compute) {{(pid=80377) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 02 10:03:18.513526 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aba741b9-be3c-4178-98cb-aae0251bf334 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:03:18.515092 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aba741b9-be3c-4178-98cb-aae0251bf334 None admin] RBAC: Authorization granted {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:03:18.525842 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: [pid: 80377|app: 0|req: 486/956] 10.222.0.82 () {56 vars in 1112 bytes} [Tue Aug 2 10:03:18 2022] GET /identity/v3/services?name=compute => generated 124 bytes in 281 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 02 10:03:18.526144 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG dbcounter [-] [80378] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80378) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 10:03:18.526311 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68c52a35-43bf-48b5-9549-fbd8efc21007 None None] REQUEST_METHOD: `POST` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:03:18.526311 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68c52a35-43bf-48b5-9549-fbd8efc21007 None None] SCRIPT_NAME: `/identity` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:03:18.526559 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68c52a35-43bf-48b5-9549-fbd8efc21007 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:03:18.546840 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62a4e200-8a32-47f3-b91c-56812c68a3f6 None None] Validating token access rules against request {{(pid=80378) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:03:18.558149 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.auth.core [None req-68c52a35-43bf-48b5-9549-fbd8efc21007 None None] MFA Rules not processed for user `891005fa5ff74041b75747d16cc66798`. Rule list: `[]` (Enabled: `True`). {{(pid=80377) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 02 10:03:18.559477 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.fernet_utils [None req-68c52a35-43bf-48b5-9549-fbd8efc21007 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80377) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 10:03:18.594297 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-344e4821-551f-48e0-ae09-5eb323e24d8c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '891005fa5ff74041b75747d16cc66798', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80378) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:03:18.595029 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-344e4821-551f-48e0-ae09-5eb323e24d8c None admin] REQUEST_METHOD: `GET` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:03:18.596058 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-344e4821-551f-48e0-ae09-5eb323e24d8c None admin] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:03:18.598625 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-344e4821-551f-48e0-ae09-5eb323e24d8c None admin] PATH_INFO: `/v3/domains` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:03:18.598835 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG dbcounter [-] [80377] Writing DB stats keystone:SELECT=100 {{(pid=80377) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 10:03:18.599442 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-344e4821-551f-48e0-ae09-5eb323e24d8c None admin] RBAC: Adding query filter params (name=Default) {{(pid=80378) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 02 10:03:18.599764 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-344e4821-551f-48e0-ae09-5eb323e24d8c None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:03:18.601696 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-344e4821-551f-48e0-ae09-5eb323e24d8c None admin] RBAC: Authorization granted {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:03:18.622533 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 471/957] 10.222.0.82 () {56 vars in 1109 bytes} [Tue Aug 2 10:03:18 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 284 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 02 10:03:18.623410 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-344e4821-551f-48e0-ae09-5eb323e24d8c None admin] Authenticating user token {{(pid=80378) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:03:18.678969 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: [pid: 80377|app: 0|req: 487/958] 10.222.0.82 () {58 vars in 1023 bytes} [Tue Aug 2 10:03:18 2022] POST /identity/v3/auth/tokens => generated 1474 bytes in 155 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 02 10:03:18.680224 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68c52a35-43bf-48b5-9549-fbd8efc21007 None None] Authenticating user token {{(pid=80377) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:03:18.796916 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-344e4821-551f-48e0-ae09-5eb323e24d8c None admin] Validating token access rules against request {{(pid=80378) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:03:18.839170 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68c52a35-43bf-48b5-9549-fbd8efc21007 None None] Validating token access rules against request {{(pid=80377) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:03:18.851315 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76ea469c-1bb4-4910-8316-f2cde65045f8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '891005fa5ff74041b75747d16cc66798', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80378) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:03:18.855314 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76ea469c-1bb4-4910-8316-f2cde65045f8 None admin] REQUEST_METHOD: `GET` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:03:18.855954 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76ea469c-1bb4-4910-8316-f2cde65045f8 None admin] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:03:18.856227 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76ea469c-1bb4-4910-8316-f2cde65045f8 None admin] PATH_INFO: `/v3/services` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:03:18.857136 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76ea469c-1bb4-4910-8316-f2cde65045f8 None admin] RBAC: Adding query filter params (type=compute) {{(pid=80378) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 02 10:03:18.857451 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76ea469c-1bb4-4910-8316-f2cde65045f8 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:03:18.858895 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76ea469c-1bb4-4910-8316-f2cde65045f8 None admin] RBAC: Authorization granted {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:03:18.865745 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 472/959] 10.222.0.82 () {56 vars in 1112 bytes} [Tue Aug 2 10:03:18 2022] GET /identity/v3/services?type=compute => generated 124 bytes in 242 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 02 10:03:18.865745 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76ea469c-1bb4-4910-8316-f2cde65045f8 None admin] Authenticating user token {{(pid=80378) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:03:18.905272 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73ed48aa-60ee-472f-b186-399826f981c9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '891005fa5ff74041b75747d16cc66798', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80377) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:03:18.906019 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73ed48aa-60ee-472f-b186-399826f981c9 None admin] REQUEST_METHOD: `GET` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:03:18.906302 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73ed48aa-60ee-472f-b186-399826f981c9 None admin] SCRIPT_NAME: `/identity` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:03:18.906576 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73ed48aa-60ee-472f-b186-399826f981c9 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:03:18.913101 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG dbcounter [-] [80378] Writing DB stats keystone:SELECT=100 {{(pid=80378) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 10:03:18.920006 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73ed48aa-60ee-472f-b186-399826f981c9 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:03:18.922020 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73ed48aa-60ee-472f-b186-399826f981c9 None admin] RBAC: Authorization granted {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:03:18.940487 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application [None req-73ed48aa-60ee-472f-b186-399826f981c9 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 02 10:03:18.940487 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 02 10:03:18.940487 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 02 10:03:18.940487 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 02 10:03:18.940487 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 02 10:03:18.940487 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 02 10:03:18.940487 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 02 10:03:18.940487 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 02 10:03:18.940487 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 02 10:03:18.940487 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 02 10:03:18.940487 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 02 10:03:18.940487 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 02 10:03:18.940487 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 02 10:03:18.940487 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 02 10:03:18.940487 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 02 10:03:18.942193 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 02 10:03:18.942193 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 02 10:03:18.942193 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 02 10:03:18.942193 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 02 10:03:18.942193 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 02 10:03:18.942193 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 02 10:03:18.942193 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application return self.get_or_create( Aug 02 10:03:18.942193 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 02 10:03:18.942193 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application with Lock( Aug 02 10:03:18.942193 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 02 10:03:18.942193 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application return self._enter() Aug 02 10:03:18.942193 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 02 10:03:18.942193 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 02 10:03:18.942193 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 02 10:03:18.942193 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application return self.creator() Aug 02 10:03:18.942193 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 02 10:03:18.942193 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application created_value = creator( Aug 02 10:03:18.944488 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 02 10:03:18.944488 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 02 10:03:18.944488 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 02 10:03:18.944488 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 02 10:03:18.944488 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 02 10:03:18.944488 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 02 10:03:18.944488 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 02 10:03:18.944488 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application Aug 02 10:03:18.946417 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: [pid: 80377|app: 0|req: 488/960] 10.222.0.82 () {56 vars in 1111 bytes} [Tue Aug 2 10:03:18 2022] GET /identity/v3/projects/service => generated 88 bytes in 267 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 02 10:03:18.950170 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73ed48aa-60ee-472f-b186-399826f981c9 None admin] Authenticating user token {{(pid=80377) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:03:18.992698 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG dbcounter [-] [80377] Writing DB stats keystone:SELECT=100 {{(pid=80377) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 10:03:19.103272 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76ea469c-1bb4-4910-8316-f2cde65045f8 None admin] Validating token access rules against request {{(pid=80378) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:03:19.150310 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73ed48aa-60ee-472f-b186-399826f981c9 None admin] Validating token access rules against request {{(pid=80377) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:03:19.172633 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dcf72a9e-6611-4c49-be56-44df83befcde None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '891005fa5ff74041b75747d16cc66798', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80378) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:03:19.174235 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcf72a9e-6611-4c49-be56-44df83befcde None admin] REQUEST_METHOD: `POST` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:03:19.175323 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcf72a9e-6611-4c49-be56-44df83befcde None admin] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:03:19.176078 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcf72a9e-6611-4c49-be56-44df83befcde None admin] PATH_INFO: `/v3/users` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:03:19.178126 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dcf72a9e-6611-4c49-be56-44df83befcde None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:03:19.181663 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dcf72a9e-6611-4c49-be56-44df83befcde None admin] RBAC: Authorization granted {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:03:19.186983 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: WARNING keystone.server.flask.common [None req-dcf72a9e-6611-4c49-be56-44df83befcde None admin] Deprecated: Not specifying a domain during a create user, group or project call, and relying on falling back to the default domain, is deprecated as of Liberty. There is no plan to remove this compatibility, however, future API versions may remove this, so please specify the domain explicitly or use a domain-scoped token. Aug 02 10:03:19.212571 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.sql.core [None req-dcf72a9e-6611-4c49-be56-44df83befcde None admin] Conflict user: (pymysql.err.IntegrityError) (1062, "Duplicate entry 'default-cinder' for key 'local_user.domain_id'") Aug 02 10:03:19.212571 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: [SQL: INSERT INTO local_user (user_id, domain_id, name, failed_auth_count, failed_auth_at) VALUES (%(user_id)s, %(domain_id)s, %(name)s, %(failed_auth_count)s, %(failed_auth_at)s)] Aug 02 10:03:19.212571 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: [parameters: {'user_id': 'b21da98fd3604a9d974633cc1da2bae6', 'domain_id': 'default', 'name': 'cinder', 'failed_auth_count': 0, 'failed_auth_at': None}] Aug 02 10:03:19.212571 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: (Background on this error at: https://sqlalche.me/e/14/gkpj) {{(pid=80378) wrapper /opt/stack/keystone/keystone/common/sql/core.py:533}} Aug 02 10:03:19.214439 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24ec4d3f-9667-42c1-9d7a-c91b6631290a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '891005fa5ff74041b75747d16cc66798', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80377) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:03:19.215844 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24ec4d3f-9667-42c1-9d7a-c91b6631290a None admin] REQUEST_METHOD: `GET` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:03:19.216592 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24ec4d3f-9667-42c1-9d7a-c91b6631290a None admin] SCRIPT_NAME: `/identity` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:03:19.217392 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24ec4d3f-9667-42c1-9d7a-c91b6631290a None admin] PATH_INFO: `/v3/projects` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:03:19.219090 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24ec4d3f-9667-42c1-9d7a-c91b6631290a None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80377) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 02 10:03:19.219891 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24ec4d3f-9667-42c1-9d7a-c91b6631290a None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:03:19.223479 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24ec4d3f-9667-42c1-9d7a-c91b6631290a None admin] RBAC: Authorization granted {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:03:19.244318 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application [None req-dcf72a9e-6611-4c49-be56-44df83befcde None admin] Conflict occurred attempting to store user - Duplicate entry found with name cinder at domain ID default.: keystone.exception.Conflict: Conflict occurred attempting to store user - Duplicate entry found with name cinder at domain ID default. Aug 02 10:03:19.244318 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 02 10:03:19.244318 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1819, in _execute_context Aug 02 10:03:19.244318 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application self.dialect.do_execute( Aug 02 10:03:19.244318 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/default.py", line 732, in do_execute Aug 02 10:03:19.244318 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application cursor.execute(statement, parameters) Aug 02 10:03:19.244318 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 148, in execute Aug 02 10:03:19.244318 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application result = self._query(query) Aug 02 10:03:19.244318 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 310, in _query Aug 02 10:03:19.244318 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application conn.query(q) Aug 02 10:03:19.244318 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 548, in query Aug 02 10:03:19.244318 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application self._affected_rows = self._read_query_result(unbuffered=unbuffered) Aug 02 10:03:19.244318 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 775, in _read_query_result Aug 02 10:03:19.244318 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application result.read() Aug 02 10:03:19.244318 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 1156, in read Aug 02 10:03:19.246131 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application first_packet = self.connection._read_packet() Aug 02 10:03:19.246131 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 725, in _read_packet Aug 02 10:03:19.246131 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application packet.raise_for_error() Aug 02 10:03:19.246131 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/protocol.py", line 221, in raise_for_error Aug 02 10:03:19.246131 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application err.raise_mysql_exception(self._data) Aug 02 10:03:19.246131 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/err.py", line 143, in raise_mysql_exception Aug 02 10:03:19.246131 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application raise errorclass(errno, errval) Aug 02 10:03:19.246131 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application pymysql.err.IntegrityError: (1062, "Duplicate entry 'default-cinder' for key 'local_user.domain_id'") Aug 02 10:03:19.246131 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application Aug 02 10:03:19.246131 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application The above exception was the direct cause of the following exception: Aug 02 10:03:19.246131 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application Aug 02 10:03:19.246131 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 02 10:03:19.246131 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/sql/core.py", line 528, in wrapper Aug 02 10:03:19.246131 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application return method(*args, **kwargs) Aug 02 10:03:19.246131 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 141, in create_user Aug 02 10:03:19.246131 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application return base.filter_user(user_ref.to_dict()) Aug 02 10:03:19.246131 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ Aug 02 10:03:19.246131 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application next(self.gen) Aug 02 10:03:19.253382 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 1060, in _transaction_scope Aug 02 10:03:19.253382 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application yield resource Aug 02 10:03:19.253382 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ Aug 02 10:03:19.253382 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application next(self.gen) Aug 02 10:03:19.253382 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 662, in _session Aug 02 10:03:19.253382 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application self.session.rollback() Aug 02 10:03:19.253382 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 227, in __exit__ Aug 02 10:03:19.253382 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application self.force_reraise() Aug 02 10:03:19.253382 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 200, in force_reraise Aug 02 10:03:19.253382 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application raise self.value Aug 02 10:03:19.253382 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 659, in _session Aug 02 10:03:19.253382 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application self._end_session_transaction(self.session) Aug 02 10:03:19.253382 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 687, in _end_session_transaction Aug 02 10:03:19.253382 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application session.commit() Aug 02 10:03:19.253382 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 1451, in commit Aug 02 10:03:19.253382 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application self._transaction.commit(_to_root=self.future) Aug 02 10:03:19.253382 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 829, in commit Aug 02 10:03:19.255020 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application self._prepare_impl() Aug 02 10:03:19.255020 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 808, in _prepare_impl Aug 02 10:03:19.255020 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application self.session.flush() Aug 02 10:03:19.255020 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3383, in flush Aug 02 10:03:19.255020 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application self._flush(objects) Aug 02 10:03:19.255020 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3523, in _flush Aug 02 10:03:19.255020 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application transaction.rollback(_capture_exception=True) Aug 02 10:03:19.255020 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/langhelpers.py", line 70, in __exit__ Aug 02 10:03:19.255020 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application compat.raise_( Aug 02 10:03:19.255020 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/compat.py", line 208, in raise_ Aug 02 10:03:19.255020 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application raise exception Aug 02 10:03:19.255020 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3483, in _flush Aug 02 10:03:19.255020 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application flush_context.execute() Aug 02 10:03:19.255020 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/unitofwork.py", line 456, in execute Aug 02 10:03:19.255020 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application rec.execute(self) Aug 02 10:03:19.255020 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/unitofwork.py", line 630, in execute Aug 02 10:03:19.255020 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application util.preloaded.orm_persistence.save_obj( Aug 02 10:03:19.257589 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/persistence.py", line 245, in save_obj Aug 02 10:03:19.257589 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application _emit_insert_statements( Aug 02 10:03:19.257589 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/persistence.py", line 1238, in _emit_insert_statements Aug 02 10:03:19.257589 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application result = connection._execute_20( Aug 02 10:03:19.257589 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1631, in _execute_20 Aug 02 10:03:19.257589 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application return meth(self, args_10style, kwargs_10style, execution_options) Aug 02 10:03:19.257589 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/sql/elements.py", line 332, in _execute_on_connection Aug 02 10:03:19.257589 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application return connection._execute_clauseelement( Aug 02 10:03:19.257589 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1498, in _execute_clauseelement Aug 02 10:03:19.257589 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application ret = self._execute_context( Aug 02 10:03:19.257589 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1862, in _execute_context Aug 02 10:03:19.257589 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application self._handle_dbapi_exception( Aug 02 10:03:19.257589 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 2041, in _handle_dbapi_exception Aug 02 10:03:19.257589 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application util.raise_(newraise, with_traceback=exc_info[2], from_=e) Aug 02 10:03:19.257589 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/compat.py", line 208, in raise_ Aug 02 10:03:19.257589 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application raise exception Aug 02 10:03:19.261051 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1819, in _execute_context Aug 02 10:03:19.261051 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application self.dialect.do_execute( Aug 02 10:03:19.261051 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/default.py", line 732, in do_execute Aug 02 10:03:19.261051 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application cursor.execute(statement, parameters) Aug 02 10:03:19.261051 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 148, in execute Aug 02 10:03:19.261051 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application result = self._query(query) Aug 02 10:03:19.261051 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 310, in _query Aug 02 10:03:19.261051 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application conn.query(q) Aug 02 10:03:19.261051 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 548, in query Aug 02 10:03:19.261051 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application self._affected_rows = self._read_query_result(unbuffered=unbuffered) Aug 02 10:03:19.261051 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 775, in _read_query_result Aug 02 10:03:19.261051 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application result.read() Aug 02 10:03:19.261051 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 1156, in read Aug 02 10:03:19.261051 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application first_packet = self.connection._read_packet() Aug 02 10:03:19.261051 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 725, in _read_packet Aug 02 10:03:19.261051 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application packet.raise_for_error() Aug 02 10:03:19.264424 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/protocol.py", line 221, in raise_for_error Aug 02 10:03:19.264424 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application err.raise_mysql_exception(self._data) Aug 02 10:03:19.264424 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/err.py", line 143, in raise_mysql_exception Aug 02 10:03:19.264424 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application raise errorclass(errno, errval) Aug 02 10:03:19.264424 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application oslo_db.exception.DBDuplicateEntry: (pymysql.err.IntegrityError) (1062, "Duplicate entry 'default-cinder' for key 'local_user.domain_id'") Aug 02 10:03:19.264424 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application [SQL: INSERT INTO local_user (user_id, domain_id, name, failed_auth_count, failed_auth_at) VALUES (%(user_id)s, %(domain_id)s, %(name)s, %(failed_auth_count)s, %(failed_auth_at)s)] Aug 02 10:03:19.264424 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application [parameters: {'user_id': 'b21da98fd3604a9d974633cc1da2bae6', 'domain_id': 'default', 'name': 'cinder', 'failed_auth_count': 0, 'failed_auth_at': None}] Aug 02 10:03:19.264424 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application (Background on this error at: https://sqlalche.me/e/14/gkpj) Aug 02 10:03:19.264424 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application Aug 02 10:03:19.264424 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 02 10:03:19.264424 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application Aug 02 10:03:19.264424 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 02 10:03:19.264424 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 02 10:03:19.264424 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 02 10:03:19.264424 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 02 10:03:19.267806 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 02 10:03:19.267806 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 02 10:03:19.267806 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 02 10:03:19.267806 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 02 10:03:19.267806 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 02 10:03:19.267806 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 02 10:03:19.267806 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 02 10:03:19.267806 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 244, in post Aug 02 10:03:19.267806 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.create_user( Aug 02 10:03:19.267806 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 02 10:03:19.267806 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 02 10:03:19.267806 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 02 10:03:19.267806 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 02 10:03:19.267806 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 02 10:03:19.267806 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 02 10:03:19.267806 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1000, in create_user Aug 02 10:03:19.267806 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application ref = self._create_user_with_federated_objects(user, driver) Aug 02 10:03:19.270525 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 966, in _create_user_with_federated_objects Aug 02 10:03:19.270525 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application user = driver.create_user(user['id'], user) Aug 02 10:03:19.270525 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/sql/core.py", line 563, in wrapper Aug 02 10:03:19.270525 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application raise exception.Conflict(type=conflict_type, Aug 02 10:03:19.270525 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application keystone.exception.Conflict: Conflict occurred attempting to store user - Duplicate entry found with name cinder at domain ID default. Aug 02 10:03:19.270525 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application Aug 02 10:03:19.270525 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 473/961] 10.222.0.82 () {60 vars in 1131 bytes} [Tue Aug 2 10:03:18 2022] POST /identity/v3/users => generated 160 bytes in 382 msecs (HTTP/1.1 409) 5 headers in 183 bytes (1 switches on core 0) Aug 02 10:03:19.271528 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: [pid: 80377|app: 0|req: 489/962] 10.222.0.82 () {56 vars in 1148 bytes} [Tue Aug 2 10:03:18 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 307 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 02 10:03:19.271675 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dcf72a9e-6611-4c49-be56-44df83befcde None admin] Authenticating user token {{(pid=80378) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:03:19.271818 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24ec4d3f-9667-42c1-9d7a-c91b6631290a None admin] Authenticating user token {{(pid=80377) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:03:19.420924 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG dbcounter [-] [80377] Writing DB stats keystone:SELECT=100 {{(pid=80377) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 10:03:19.434521 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG dbcounter [-] [80378] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80378) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 10:03:19.464549 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dcf72a9e-6611-4c49-be56-44df83befcde None admin] Validating token access rules against request {{(pid=80378) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:03:19.471229 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24ec4d3f-9667-42c1-9d7a-c91b6631290a None admin] Validating token access rules against request {{(pid=80377) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:03:19.537138 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d2ec93e-354f-4d24-a24a-8ba58e96b6cf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '891005fa5ff74041b75747d16cc66798', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80378) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:03:19.538383 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d2ec93e-354f-4d24-a24a-8ba58e96b6cf None admin] REQUEST_METHOD: `GET` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:03:19.539002 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d2ec93e-354f-4d24-a24a-8ba58e96b6cf None admin] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:03:19.539619 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d2ec93e-354f-4d24-a24a-8ba58e96b6cf None admin] PATH_INFO: `/v3/users/cinder` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:03:19.553348 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d2ec93e-354f-4d24-a24a-8ba58e96b6cf None admin] RBAC: Authorizing `identity:get_user(user_id=cinder)` {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:03:19.556059 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d2ec93e-354f-4d24-a24a-8ba58e96b6cf None admin] RBAC: Authorization granted {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:03:19.562672 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6fcd602-8806-4b2d-9b8d-6b2cbc5b79f0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '891005fa5ff74041b75747d16cc66798', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80377) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:03:19.564361 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6fcd602-8806-4b2d-9b8d-6b2cbc5b79f0 None admin] REQUEST_METHOD: `PUT` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:03:19.565743 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6fcd602-8806-4b2d-9b8d-6b2cbc5b79f0 None admin] SCRIPT_NAME: `/identity` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:03:19.566480 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6fcd602-8806-4b2d-9b8d-6b2cbc5b79f0 None admin] PATH_INFO: `/v3/projects/f1b38ebb804c41c69772c998aae72c41/users/bd8dd72086db48b8aaac2ed0058872df/roles/f57a652471c8414597c5fdde3d6f4636` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:03:19.571200 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6fcd602-8806-4b2d-9b8d-6b2cbc5b79f0 None admin] RBAC: Authorizing `identity:create_grant(project_id=f1b38ebb804c41c69772c998aae72c41, user_id=bd8dd72086db48b8aaac2ed0058872df, role_id=f57a652471c8414597c5fdde3d6f4636)` {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:03:19.574227 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6fcd602-8806-4b2d-9b8d-6b2cbc5b79f0 None admin] RBAC: Authorization granted {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:03:19.577562 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application [None req-6d2ec93e-354f-4d24-a24a-8ba58e96b6cf None admin] Could not find user: cinder.: keystone.exception.UserNotFound: Could not find user: cinder. Aug 02 10:03:19.577562 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 02 10:03:19.577562 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 02 10:03:19.577562 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 02 10:03:19.577562 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 02 10:03:19.577562 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 02 10:03:19.577562 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 02 10:03:19.577562 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 02 10:03:19.577562 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 02 10:03:19.577562 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 02 10:03:19.577562 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 02 10:03:19.577562 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 02 10:03:19.577562 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 02 10:03:19.577562 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 02 10:03:19.577562 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 02 10:03:19.577562 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 02 10:03:19.579620 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 02 10:03:19.579620 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 02 10:03:19.579620 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 02 10:03:19.579620 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 02 10:03:19.579620 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 02 10:03:19.579620 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 02 10:03:19.579620 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 02 10:03:19.579620 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 02 10:03:19.579620 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 02 10:03:19.579620 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application return self.get_or_create( Aug 02 10:03:19.579620 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 02 10:03:19.579620 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application with Lock( Aug 02 10:03:19.579620 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 02 10:03:19.579620 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application return self._enter() Aug 02 10:03:19.579620 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 02 10:03:19.579620 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 02 10:03:19.579620 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 02 10:03:19.579620 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application return self.creator() Aug 02 10:03:19.581824 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 02 10:03:19.581824 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application created_value = creator( Aug 02 10:03:19.581824 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 02 10:03:19.581824 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 02 10:03:19.581824 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 02 10:03:19.581824 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 02 10:03:19.581824 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 02 10:03:19.581824 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 02 10:03:19.581824 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: cinder. Aug 02 10:03:19.581824 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application Aug 02 10:03:19.590099 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 474/963] 10.222.0.82 () {56 vars in 1099 bytes} [Tue Aug 2 10:03:19 2022] GET /identity/v3/users/cinder => generated 84 bytes in 326 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 02 10:03:19.595808 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d2ec93e-354f-4d24-a24a-8ba58e96b6cf None admin] Authenticating user token {{(pid=80378) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:03:19.624431 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: [pid: 80377|app: 0|req: 490/964] 10.222.0.82 () {58 vars in 1439 bytes} [Tue Aug 2 10:03:19 2022] PUT /identity/v3/projects/f1b38ebb804c41c69772c998aae72c41/users/bd8dd72086db48b8aaac2ed0058872df/roles/f57a652471c8414597c5fdde3d6f4636 => generated 0 bytes in 361 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 02 10:03:19.809543 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d2ec93e-354f-4d24-a24a-8ba58e96b6cf None admin] Validating token access rules against request {{(pid=80378) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:03:19.870011 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39b048ee-f192-4b1d-ac35-c9556aa0ca4f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '891005fa5ff74041b75747d16cc66798', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80378) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:03:19.871098 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39b048ee-f192-4b1d-ac35-c9556aa0ca4f None admin] REQUEST_METHOD: `GET` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:03:19.871594 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39b048ee-f192-4b1d-ac35-c9556aa0ca4f None admin] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:03:19.872091 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39b048ee-f192-4b1d-ac35-c9556aa0ca4f None admin] PATH_INFO: `/v3/users` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:03:19.873175 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39b048ee-f192-4b1d-ac35-c9556aa0ca4f None admin] RBAC: Adding query filter params (name=cinder) {{(pid=80378) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 02 10:03:19.873551 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39b048ee-f192-4b1d-ac35-c9556aa0ca4f None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:03:19.876216 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39b048ee-f192-4b1d-ac35-c9556aa0ca4f None admin] RBAC: Authorization granted {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:03:19.895160 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG dbcounter [-] [80378] Writing DB stats keystone:SELECT=100 {{(pid=80378) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 10:03:19.953995 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 475/965] 10.222.0.82 () {56 vars in 1101 bytes} [Tue Aug 2 10:03:19 2022] GET /identity/v3/users?name=cinder => generated 354 bytes in 353 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 02 10:03:21.118230 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff69be09-18fc-4edb-95d3-d1b1e16df3ab None None] REQUEST_METHOD: `GET` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:03:21.118923 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff69be09-18fc-4edb-95d3-d1b1e16df3ab None None] SCRIPT_NAME: `/identity` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:03:21.119249 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff69be09-18fc-4edb-95d3-d1b1e16df3ab None None] PATH_INFO: `/` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:03:21.121881 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: [pid: 80377|app: 0|req: 491/966] 10.222.0.82 () {54 vars in 926 bytes} [Tue Aug 2 10:03:21 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 02 10:03:21.131799 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a877aba0-6930-4072-b2ee-f246bea873b9 None None] REQUEST_METHOD: `POST` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:03:21.132343 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a877aba0-6930-4072-b2ee-f246bea873b9 None None] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:03:21.132798 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a877aba0-6930-4072-b2ee-f246bea873b9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:03:21.164829 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.auth.core [None req-a877aba0-6930-4072-b2ee-f246bea873b9 None None] MFA Rules not processed for user `891005fa5ff74041b75747d16cc66798`. Rule list: `[]` (Enabled: `True`). {{(pid=80378) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 02 10:03:21.166914 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.fernet_utils [None req-a877aba0-6930-4072-b2ee-f246bea873b9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80378) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 10:03:21.286930 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 476/967] 10.222.0.82 () {58 vars in 1023 bytes} [Tue Aug 2 10:03:21 2022] POST /identity/v3/auth/tokens => generated 1474 bytes in 157 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Aug 02 10:03:21.311005 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff69be09-18fc-4edb-95d3-d1b1e16df3ab None None] Authenticating user token {{(pid=80377) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:03:21.470270 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff69be09-18fc-4edb-95d3-d1b1e16df3ab None None] Validating token access rules against request {{(pid=80377) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:03:21.517944 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fda99bd2-c111-480a-85ff-bc16eacb60b2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '891005fa5ff74041b75747d16cc66798', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80377) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:03:21.518700 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fda99bd2-c111-480a-85ff-bc16eacb60b2 None admin] REQUEST_METHOD: `GET` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:03:21.519046 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fda99bd2-c111-480a-85ff-bc16eacb60b2 None admin] SCRIPT_NAME: `/identity` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:03:21.519366 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fda99bd2-c111-480a-85ff-bc16eacb60b2 None admin] PATH_INFO: `/v3/services/object-store` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:03:21.520203 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fda99bd2-c111-480a-85ff-bc16eacb60b2 None admin] RBAC: Authorizing `identity:get_service(service_id=object-store)` {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:03:21.521975 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fda99bd2-c111-480a-85ff-bc16eacb60b2 None admin] RBAC: Authorization granted {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:03:21.537343 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application [None req-fda99bd2-c111-480a-85ff-bc16eacb60b2 None admin] Could not find service: object-store.: keystone.exception.ServiceNotFound: Could not find service: object-store. Aug 02 10:03:21.537343 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 02 10:03:21.537343 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Aug 02 10:03:21.537343 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Aug 02 10:03:21.537343 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Aug 02 10:03:21.537343 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Aug 02 10:03:21.537343 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Aug 02 10:03:21.537343 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 02 10:03:21.537343 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Aug 02 10:03:21.537343 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application Aug 02 10:03:21.537343 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 02 10:03:21.537343 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application Aug 02 10:03:21.537343 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 02 10:03:21.537343 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 02 10:03:21.537343 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 02 10:03:21.537343 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 02 10:03:21.537343 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 02 10:03:21.539330 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 02 10:03:21.539330 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 02 10:03:21.539330 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 02 10:03:21.539330 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 02 10:03:21.539330 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 02 10:03:21.539330 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 02 10:03:21.539330 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Aug 02 10:03:21.539330 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application return self._get_service(service_id) Aug 02 10:03:21.539330 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Aug 02 10:03:21.539330 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Aug 02 10:03:21.539330 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 02 10:03:21.539330 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 02 10:03:21.539330 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 02 10:03:21.539330 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 02 10:03:21.539330 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 02 10:03:21.539330 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application return self.get_or_create( Aug 02 10:03:21.541343 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 02 10:03:21.541343 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application with Lock( Aug 02 10:03:21.541343 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 02 10:03:21.541343 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application return self._enter() Aug 02 10:03:21.541343 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 02 10:03:21.541343 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 02 10:03:21.541343 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 02 10:03:21.541343 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application return self.creator() Aug 02 10:03:21.541343 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 02 10:03:21.541343 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application created_value = creator( Aug 02 10:03:21.541343 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Aug 02 10:03:21.541343 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 02 10:03:21.541343 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Aug 02 10:03:21.541343 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application Aug 02 10:03:21.544157 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: [pid: 80377|app: 0|req: 492/968] 10.222.0.82 () {56 vars in 1126 bytes} [Tue Aug 2 10:03:21 2022] GET /identity/v3/services/object-store => generated 93 bytes in 233 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 02 10:03:21.545845 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a877aba0-6930-4072-b2ee-f246bea873b9 None None] Authenticating user token {{(pid=80378) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:03:21.617608 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b78171cf-f560-4681-9f91-ae4794577c02 None None] REQUEST_METHOD: `GET` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:03:21.618121 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b78171cf-f560-4681-9f91-ae4794577c02 None None] SCRIPT_NAME: `/identity` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:03:21.618567 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b78171cf-f560-4681-9f91-ae4794577c02 None None] PATH_INFO: `/` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:03:21.621067 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: [pid: 80377|app: 0|req: 493/969] 10.222.0.82 () {54 vars in 926 bytes} [Tue Aug 2 10:03:21 2022] GET /identity => generated 270 bytes in 7 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 02 10:03:21.630068 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a655c05-47e0-4b1a-8c45-7b6d553b3448 None None] REQUEST_METHOD: `POST` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:03:21.630520 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a655c05-47e0-4b1a-8c45-7b6d553b3448 None None] SCRIPT_NAME: `/identity` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:03:21.630995 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a655c05-47e0-4b1a-8c45-7b6d553b3448 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:03:21.656822 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG dbcounter [-] [80377] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80377) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 10:03:21.669646 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.auth.core [None req-3a655c05-47e0-4b1a-8c45-7b6d553b3448 None None] MFA Rules not processed for user `891005fa5ff74041b75747d16cc66798`. Rule list: `[]` (Enabled: `True`). {{(pid=80377) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 02 10:03:21.674405 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.fernet_utils [None req-3a655c05-47e0-4b1a-8c45-7b6d553b3448 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80377) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 10:03:21.714896 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a877aba0-6930-4072-b2ee-f246bea873b9 None None] Validating token access rules against request {{(pid=80378) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:03:21.755569 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG dbcounter [-] [80378] Writing DB stats keystone:SELECT=100 {{(pid=80378) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 10:03:21.773499 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05d4af64-7db1-4613-a400-afaccc37e422 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '891005fa5ff74041b75747d16cc66798', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80378) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:03:21.774549 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05d4af64-7db1-4613-a400-afaccc37e422 None admin] REQUEST_METHOD: `GET` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:03:21.775027 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05d4af64-7db1-4613-a400-afaccc37e422 None admin] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:03:21.775500 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05d4af64-7db1-4613-a400-afaccc37e422 None admin] PATH_INFO: `/v3/services` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:03:21.776915 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05d4af64-7db1-4613-a400-afaccc37e422 None admin] RBAC: Adding query filter params (name=object-store) {{(pid=80378) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 02 10:03:21.777438 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05d4af64-7db1-4613-a400-afaccc37e422 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:03:21.779648 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05d4af64-7db1-4613-a400-afaccc37e422 None admin] RBAC: Authorization granted {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:03:21.787536 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 477/970] 10.222.0.82 () {56 vars in 1122 bytes} [Tue Aug 2 10:03:21 2022] GET /identity/v3/services?name=object-store => generated 129 bytes in 243 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 02 10:03:21.795481 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05d4af64-7db1-4613-a400-afaccc37e422 None admin] Authenticating user token {{(pid=80378) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:03:21.804429 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: [pid: 80377|app: 0|req: 494/971] 10.222.0.82 () {58 vars in 1023 bytes} [Tue Aug 2 10:03:21 2022] POST /identity/v3/auth/tokens => generated 1474 bytes in 176 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Aug 02 10:03:21.830669 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a655c05-47e0-4b1a-8c45-7b6d553b3448 None None] Authenticating user token {{(pid=80377) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:03:21.965700 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05d4af64-7db1-4613-a400-afaccc37e422 None admin] Validating token access rules against request {{(pid=80378) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:03:21.992707 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a655c05-47e0-4b1a-8c45-7b6d553b3448 None None] Validating token access rules against request {{(pid=80377) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:03:22.020218 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3635782-e5f5-4088-b2a2-c87372c15609 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '891005fa5ff74041b75747d16cc66798', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80378) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:03:22.021125 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3635782-e5f5-4088-b2a2-c87372c15609 None admin] REQUEST_METHOD: `GET` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:03:22.021622 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3635782-e5f5-4088-b2a2-c87372c15609 None admin] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:03:22.022102 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3635782-e5f5-4088-b2a2-c87372c15609 None admin] PATH_INFO: `/v3/services` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:03:22.022935 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3635782-e5f5-4088-b2a2-c87372c15609 None admin] RBAC: Adding query filter params (type=object-store) {{(pid=80378) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 02 10:03:22.023374 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3635782-e5f5-4088-b2a2-c87372c15609 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:03:22.024944 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3635782-e5f5-4088-b2a2-c87372c15609 None admin] RBAC: Authorization granted {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:03:22.031871 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 478/972] 10.222.0.82 () {56 vars in 1122 bytes} [Tue Aug 2 10:03:21 2022] GET /identity/v3/services?type=object-store => generated 356 bytes in 238 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 02 10:03:22.039215 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3635782-e5f5-4088-b2a2-c87372c15609 None admin] Authenticating user token {{(pid=80378) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:03:22.060324 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffeb4d05-77cf-4e2c-b25a-a4a35a1f1199 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '891005fa5ff74041b75747d16cc66798', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80377) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:03:22.061381 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffeb4d05-77cf-4e2c-b25a-a4a35a1f1199 None admin] REQUEST_METHOD: `POST` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:03:22.061903 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffeb4d05-77cf-4e2c-b25a-a4a35a1f1199 None admin] SCRIPT_NAME: `/identity` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:03:22.062255 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffeb4d05-77cf-4e2c-b25a-a4a35a1f1199 None admin] PATH_INFO: `/v3/services` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:03:22.064070 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ffeb4d05-77cf-4e2c-b25a-a4a35a1f1199 None admin] RBAC: Authorizing `identity:create_service()` {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:03:22.064436 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ffeb4d05-77cf-4e2c-b25a-a4a35a1f1199 None admin] RBAC: Authorization granted {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:03:22.069522 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG dbcounter [-] [80377] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80377) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 10:03:22.089991 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: [pid: 80377|app: 0|req: 495/973] 10.222.0.82 () {60 vars in 1141 bytes} [Tue Aug 2 10:03:21 2022] POST /identity/v3/services => generated 242 bytes in 255 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 02 10:03:22.134633 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG dbcounter [-] [80378] Writing DB stats keystone:SELECT=100 {{(pid=80378) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 10:03:22.215907 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3635782-e5f5-4088-b2a2-c87372c15609 None admin] Validating token access rules against request {{(pid=80378) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:03:22.285526 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4939e735-dab3-443d-bac0-df38201b5d4f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '891005fa5ff74041b75747d16cc66798', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80378) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:03:22.286204 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4939e735-dab3-443d-bac0-df38201b5d4f None admin] REQUEST_METHOD: `GET` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:03:22.286518 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4939e735-dab3-443d-bac0-df38201b5d4f None admin] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:03:22.286800 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4939e735-dab3-443d-bac0-df38201b5d4f None admin] PATH_INFO: `/v3/endpoints` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:03:22.287671 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4939e735-dab3-443d-bac0-df38201b5d4f None admin] RBAC: Adding query filter params (interface=admin, service_id=afb73e70db0a46bd8e098a5c181d16cd, region_id=RegionOne) {{(pid=80378) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 02 10:03:22.287987 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4939e735-dab3-443d-bac0-df38201b5d4f None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:03:22.289133 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df5a7c9f-082e-4d88-b32f-2e22794df857 None None] REQUEST_METHOD: `GET` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:03:22.289875 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4939e735-dab3-443d-bac0-df38201b5d4f None admin] RBAC: Authorization granted {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:03:22.290061 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df5a7c9f-082e-4d88-b32f-2e22794df857 None None] SCRIPT_NAME: `/identity` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:03:22.290061 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df5a7c9f-082e-4d88-b32f-2e22794df857 None None] PATH_INFO: `/` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:03:22.291262 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: [pid: 80377|app: 0|req: 496/974] 10.222.0.82 () {54 vars in 926 bytes} [Tue Aug 2 10:03:22 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 02 10:03:22.304964 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3fc292e-b766-484b-9ec2-8ce16b5be97e None None] REQUEST_METHOD: `POST` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:03:22.305265 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3fc292e-b766-484b-9ec2-8ce16b5be97e None None] SCRIPT_NAME: `/identity` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:03:22.305351 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3fc292e-b766-484b-9ec2-8ce16b5be97e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:03:22.305666 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 479/975] 10.222.0.82 () {56 vars in 1249 bytes} [Tue Aug 2 10:03:22 2022] GET /identity/v3/endpoints?service_id=afb73e70db0a46bd8e098a5c181d16cd&interface=admin®ion_id=RegionOne => generated 193 bytes in 267 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 02 10:03:22.311397 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4939e735-dab3-443d-bac0-df38201b5d4f None admin] Authenticating user token {{(pid=80378) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:03:22.328007 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.auth.core [None req-d3fc292e-b766-484b-9ec2-8ce16b5be97e None None] MFA Rules not processed for user `891005fa5ff74041b75747d16cc66798`. Rule list: `[]` (Enabled: `True`). {{(pid=80377) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 02 10:03:22.329237 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.fernet_utils [None req-d3fc292e-b766-484b-9ec2-8ce16b5be97e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80377) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 10:03:22.434327 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: [pid: 80377|app: 0|req: 497/976] 10.222.0.82 () {58 vars in 1023 bytes} [Tue Aug 2 10:03:22 2022] POST /identity/v3/auth/tokens => generated 1570 bytes in 131 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 02 10:03:22.459897 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3fc292e-b766-484b-9ec2-8ce16b5be97e None None] Authenticating user token {{(pid=80377) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:03:22.482684 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4939e735-dab3-443d-bac0-df38201b5d4f None admin] Validating token access rules against request {{(pid=80378) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:03:22.505212 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG dbcounter [-] [80378] Writing DB stats keystone:SELECT=100 {{(pid=80378) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 10:03:22.540748 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cce56c2a-b80b-41ef-b69a-69a1355d8789 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '891005fa5ff74041b75747d16cc66798', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80378) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:03:22.541654 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cce56c2a-b80b-41ef-b69a-69a1355d8789 None admin] REQUEST_METHOD: `GET` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:03:22.542243 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cce56c2a-b80b-41ef-b69a-69a1355d8789 None admin] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:03:22.542730 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cce56c2a-b80b-41ef-b69a-69a1355d8789 None admin] PATH_INFO: `/v3/services` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:03:22.543800 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cce56c2a-b80b-41ef-b69a-69a1355d8789 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:03:22.546208 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cce56c2a-b80b-41ef-b69a-69a1355d8789 None admin] RBAC: Authorization granted {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:03:22.555797 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 480/977] 10.222.0.82 () {56 vars in 1087 bytes} [Tue Aug 2 10:03:22 2022] GET /identity/v3/services => generated 1727 bytes in 245 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 02 10:03:22.634923 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3fc292e-b766-484b-9ec2-8ce16b5be97e None None] Validating token access rules against request {{(pid=80377) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:03:22.681303 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG dbcounter [-] [80377] Writing DB stats keystone:SELECT=100 {{(pid=80377) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 10:03:22.700351 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa3b0f7b-dda8-4485-bd0d-5c0b6159d5e8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '891005fa5ff74041b75747d16cc66798', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80377) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:03:22.701979 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa3b0f7b-dda8-4485-bd0d-5c0b6159d5e8 None admin] REQUEST_METHOD: `GET` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:03:22.702420 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa3b0f7b-dda8-4485-bd0d-5c0b6159d5e8 None admin] SCRIPT_NAME: `/identity` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:03:22.702832 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa3b0f7b-dda8-4485-bd0d-5c0b6159d5e8 None admin] PATH_INFO: `/v3/roles/ResellerAdmin` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:03:22.728951 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa3b0f7b-dda8-4485-bd0d-5c0b6159d5e8 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 02 10:03:22.739933 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application [None req-aa3b0f7b-dda8-4485-bd0d-5c0b6159d5e8 None admin] Could not find role: ResellerAdmin.: keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Aug 02 10:03:22.739933 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 02 10:03:22.739933 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 02 10:03:22.739933 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 02 10:03:22.739933 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 02 10:03:22.739933 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 02 10:03:22.739933 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 02 10:03:22.739933 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 02 10:03:22.739933 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 02 10:03:22.739933 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 02 10:03:22.739933 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 02 10:03:22.739933 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 02 10:03:22.739933 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 02 10:03:22.739933 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 02 10:03:22.739933 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 02 10:03:22.739933 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 02 10:03:22.741541 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 02 10:03:22.741541 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 02 10:03:22.741541 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 02 10:03:22.741541 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 02 10:03:22.741541 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 02 10:03:22.741541 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 02 10:03:22.741541 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 02 10:03:22.741541 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 02 10:03:22.741541 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 02 10:03:22.741541 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application return self.get_or_create( Aug 02 10:03:22.741541 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 02 10:03:22.741541 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application with Lock( Aug 02 10:03:22.741541 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 02 10:03:22.741541 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application return self._enter() Aug 02 10:03:22.741541 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 02 10:03:22.741541 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 02 10:03:22.743753 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 02 10:03:22.743753 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application return self.creator() Aug 02 10:03:22.743753 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 02 10:03:22.743753 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application created_value = creator( Aug 02 10:03:22.743753 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 02 10:03:22.743753 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 02 10:03:22.743753 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 02 10:03:22.743753 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 02 10:03:22.743753 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 02 10:03:22.743753 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 02 10:03:22.743753 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Aug 02 10:03:22.743753 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application Aug 02 10:03:22.743753 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: [pid: 80377|app: 0|req: 498/978] 10.222.0.82 () {56 vars in 1120 bytes} [Tue Aug 2 10:03:22 2022] GET /identity/v3/roles/ResellerAdmin => generated 91 bytes in 283 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 02 10:03:22.753849 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cce56c2a-b80b-41ef-b69a-69a1355d8789 None admin] Authenticating user token {{(pid=80378) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:03:22.936913 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cce56c2a-b80b-41ef-b69a-69a1355d8789 None admin] Validating token access rules against request {{(pid=80378) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:03:22.996575 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-202fe6df-5ef1-44d4-a101-9e699875b44b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '891005fa5ff74041b75747d16cc66798', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80378) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:03:22.997217 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-202fe6df-5ef1-44d4-a101-9e699875b44b None admin] REQUEST_METHOD: `GET` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:03:22.997500 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-202fe6df-5ef1-44d4-a101-9e699875b44b None admin] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:03:22.997768 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-202fe6df-5ef1-44d4-a101-9e699875b44b None admin] PATH_INFO: `/v3/roles` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:03:22.998528 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-202fe6df-5ef1-44d4-a101-9e699875b44b None admin] RBAC: Adding query filter params (name=ResellerAdmin) {{(pid=80378) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 02 10:03:22.998831 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-202fe6df-5ef1-44d4-a101-9e699875b44b None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:03:23.000373 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-202fe6df-5ef1-44d4-a101-9e699875b44b None admin] RBAC: Authorization granted {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:03:23.016463 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 481/979] 10.222.0.82 () {56 vars in 1115 bytes} [Tue Aug 2 10:03:22 2022] GET /identity/v3/roles?name=ResellerAdmin => generated 338 bytes in 267 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 02 10:03:23.023201 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa3b0f7b-dda8-4485-bd0d-5c0b6159d5e8 None admin] Authenticating user token {{(pid=80377) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:03:23.181560 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa3b0f7b-dda8-4485-bd0d-5c0b6159d5e8 None admin] Validating token access rules against request {{(pid=80377) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:03:23.233069 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3e9f97c-403a-43a2-aec3-d3b39f6ab464 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '891005fa5ff74041b75747d16cc66798', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80377) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:03:23.234051 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3e9f97c-403a-43a2-aec3-d3b39f6ab464 None admin] REQUEST_METHOD: `GET` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:03:23.234377 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3e9f97c-403a-43a2-aec3-d3b39f6ab464 None admin] SCRIPT_NAME: `/identity` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:03:23.234635 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3e9f97c-403a-43a2-aec3-d3b39f6ab464 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:03:23.252249 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3e9f97c-403a-43a2-aec3-d3b39f6ab464 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:03:23.255068 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3e9f97c-403a-43a2-aec3-d3b39f6ab464 None admin] RBAC: Authorization granted {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:03:23.278757 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application [None req-a3e9f97c-403a-43a2-aec3-d3b39f6ab464 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 02 10:03:23.278757 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 02 10:03:23.278757 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 02 10:03:23.278757 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 02 10:03:23.278757 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 02 10:03:23.278757 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 02 10:03:23.278757 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 02 10:03:23.278757 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 02 10:03:23.278757 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 02 10:03:23.278757 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 02 10:03:23.278757 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 02 10:03:23.278757 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 02 10:03:23.278757 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 02 10:03:23.278757 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 02 10:03:23.278757 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 02 10:03:23.280109 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 02 10:03:23.280109 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 02 10:03:23.280109 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 02 10:03:23.280109 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 02 10:03:23.280109 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 02 10:03:23.280109 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 02 10:03:23.280109 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application return self.get_or_create( Aug 02 10:03:23.280109 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 02 10:03:23.280109 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application with Lock( Aug 02 10:03:23.280109 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 02 10:03:23.280109 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application return self._enter() Aug 02 10:03:23.280109 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 02 10:03:23.280109 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 02 10:03:23.280109 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 02 10:03:23.280109 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application return self.creator() Aug 02 10:03:23.280109 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 02 10:03:23.280109 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application created_value = creator( Aug 02 10:03:23.281561 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 02 10:03:23.281561 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 02 10:03:23.281561 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 02 10:03:23.281561 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application Aug 02 10:03:23.282707 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: [pid: 80377|app: 0|req: 499/980] 10.222.0.82 () {56 vars in 1108 bytes} [Tue Aug 2 10:03:23 2022] GET /identity/v3/domains/Default => generated 87 bytes in 260 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 02 10:03:23.289270 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-202fe6df-5ef1-44d4-a101-9e699875b44b None admin] Authenticating user token {{(pid=80378) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:03:23.360927 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG dbcounter [-] [80378] Writing DB stats keystone:SELECT=100 {{(pid=80378) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 10:03:23.479726 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-202fe6df-5ef1-44d4-a101-9e699875b44b None admin] Validating token access rules against request {{(pid=80378) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:03:23.544675 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f83e12b4-facf-4dfd-b74c-01b14fd23525 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '891005fa5ff74041b75747d16cc66798', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80378) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:03:23.545688 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f83e12b4-facf-4dfd-b74c-01b14fd23525 None admin] REQUEST_METHOD: `GET` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:03:23.546195 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f83e12b4-facf-4dfd-b74c-01b14fd23525 None admin] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:03:23.546680 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f83e12b4-facf-4dfd-b74c-01b14fd23525 None admin] PATH_INFO: `/v3/domains` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:03:23.547869 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f83e12b4-facf-4dfd-b74c-01b14fd23525 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80378) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 02 10:03:23.548396 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f83e12b4-facf-4dfd-b74c-01b14fd23525 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:03:23.551459 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f83e12b4-facf-4dfd-b74c-01b14fd23525 None admin] RBAC: Authorization granted {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:03:23.580977 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 482/981] 10.222.0.82 () {56 vars in 1109 bytes} [Tue Aug 2 10:03:23 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 287 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 02 10:03:23.583312 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3e9f97c-403a-43a2-aec3-d3b39f6ab464 None admin] Authenticating user token {{(pid=80377) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:03:23.653188 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG dbcounter [-] [80377] Writing DB stats keystone:SELECT=100 {{(pid=80377) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 10:03:23.769939 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3e9f97c-403a-43a2-aec3-d3b39f6ab464 None admin] Validating token access rules against request {{(pid=80377) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:03:23.828103 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-caae4d05-62f6-4511-94b5-1280fca66830 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '891005fa5ff74041b75747d16cc66798', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80377) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:03:23.828848 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-caae4d05-62f6-4511-94b5-1280fca66830 None admin] REQUEST_METHOD: `GET` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:03:23.829165 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-caae4d05-62f6-4511-94b5-1280fca66830 None admin] SCRIPT_NAME: `/identity` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:03:23.829441 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-caae4d05-62f6-4511-94b5-1280fca66830 None admin] PATH_INFO: `/v3/users/glance-swift` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:03:23.842104 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-caae4d05-62f6-4511-94b5-1280fca66830 None admin] RBAC: Authorizing `identity:get_user(user_id=glance-swift)` {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:03:23.842804 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-caae4d05-62f6-4511-94b5-1280fca66830 None admin] RBAC: Authorization granted {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:03:23.867982 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application [None req-caae4d05-62f6-4511-94b5-1280fca66830 None admin] Could not find user: glance-swift.: keystone.exception.UserNotFound: Could not find user: glance-swift. Aug 02 10:03:23.867982 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 02 10:03:23.867982 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 02 10:03:23.867982 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 02 10:03:23.867982 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 02 10:03:23.867982 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 02 10:03:23.867982 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 02 10:03:23.867982 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 02 10:03:23.867982 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 02 10:03:23.867982 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 02 10:03:23.867982 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 02 10:03:23.867982 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 02 10:03:23.867982 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 02 10:03:23.867982 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 02 10:03:23.867982 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 02 10:03:23.867982 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 02 10:03:23.874765 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 02 10:03:23.874765 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 02 10:03:23.874765 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 02 10:03:23.874765 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 02 10:03:23.874765 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 02 10:03:23.874765 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 02 10:03:23.874765 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 02 10:03:23.874765 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 02 10:03:23.874765 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 02 10:03:23.874765 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application return self.get_or_create( Aug 02 10:03:23.874765 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 02 10:03:23.874765 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application with Lock( Aug 02 10:03:23.874765 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 02 10:03:23.874765 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application return self._enter() Aug 02 10:03:23.874765 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 02 10:03:23.874765 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 02 10:03:23.874765 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 02 10:03:23.874765 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application return self.creator() Aug 02 10:03:23.876194 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 02 10:03:23.876194 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application created_value = creator( Aug 02 10:03:23.876194 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 02 10:03:23.876194 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 02 10:03:23.876194 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 02 10:03:23.876194 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 02 10:03:23.876194 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 02 10:03:23.876194 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 02 10:03:23.876194 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance-swift. Aug 02 10:03:23.876194 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application Aug 02 10:03:23.876194 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: [pid: 80377|app: 0|req: 500/982] 10.222.0.82 () {56 vars in 1117 bytes} [Tue Aug 2 10:03:23 2022] GET /identity/v3/users/glance-swift => generated 90 bytes in 288 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 02 10:03:23.881977 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f83e12b4-facf-4dfd-b74c-01b14fd23525 None admin] Authenticating user token {{(pid=80378) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:03:24.051079 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG dbcounter [-] [80378] Writing DB stats keystone:SELECT=100 {{(pid=80378) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 10:03:24.058941 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f83e12b4-facf-4dfd-b74c-01b14fd23525 None admin] Validating token access rules against request {{(pid=80378) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:03:24.122308 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-477582e4-5e85-4df9-9185-69076a495046 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '891005fa5ff74041b75747d16cc66798', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80378) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:03:24.123335 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-477582e4-5e85-4df9-9185-69076a495046 None admin] REQUEST_METHOD: `GET` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:03:24.123792 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-477582e4-5e85-4df9-9185-69076a495046 None admin] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:03:24.124247 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-477582e4-5e85-4df9-9185-69076a495046 None admin] PATH_INFO: `/v3/users` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:03:24.125675 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-477582e4-5e85-4df9-9185-69076a495046 None admin] RBAC: Adding query filter params (domain_id=default, name=glance-swift) {{(pid=80378) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 02 10:03:24.126259 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-477582e4-5e85-4df9-9185-69076a495046 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:03:24.128890 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-477582e4-5e85-4df9-9185-69076a495046 None admin] RBAC: Authorization granted {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:03:24.171379 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 483/983] 10.222.0.82 () {56 vars in 1149 bytes} [Tue Aug 2 10:03:23 2022] GET /identity/v3/users?domain_id=default&name=glance-swift => generated 384 bytes in 286 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 02 10:03:24.172931 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-caae4d05-62f6-4511-94b5-1280fca66830 None admin] Authenticating user token {{(pid=80377) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:03:24.326908 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG dbcounter [-] [80377] Writing DB stats keystone:SELECT=100 {{(pid=80377) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 10:03:24.335545 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-caae4d05-62f6-4511-94b5-1280fca66830 None admin] Validating token access rules against request {{(pid=80377) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:03:24.397695 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-593a5450-dc19-4caf-a920-9899a6039e64 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '891005fa5ff74041b75747d16cc66798', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80377) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:03:24.398564 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-593a5450-dc19-4caf-a920-9899a6039e64 None admin] REQUEST_METHOD: `GET` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:03:24.399011 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-593a5450-dc19-4caf-a920-9899a6039e64 None admin] SCRIPT_NAME: `/identity` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:03:24.399470 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-593a5450-dc19-4caf-a920-9899a6039e64 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:03:24.514358 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-593a5450-dc19-4caf-a920-9899a6039e64 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:03:24.517059 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-593a5450-dc19-4caf-a920-9899a6039e64 None admin] RBAC: Authorization granted {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:03:24.626395 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-523a566c-ebae-40db-b77a-5a6d61f5e779 None None] REQUEST_METHOD: `GET` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:03:24.627017 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-523a566c-ebae-40db-b77a-5a6d61f5e779 None None] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:03:24.627017 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-523a566c-ebae-40db-b77a-5a6d61f5e779 None None] PATH_INFO: `/` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:03:24.629844 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 484/984] 10.222.0.82 () {54 vars in 926 bytes} [Tue Aug 2 10:03:24 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 02 10:03:24.637919 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa4f1714-abde-4967-9151-a25db7f94553 None None] REQUEST_METHOD: `POST` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:03:24.638391 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa4f1714-abde-4967-9151-a25db7f94553 None None] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:03:24.639025 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa4f1714-abde-4967-9151-a25db7f94553 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:03:24.670838 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.auth.core [None req-fa4f1714-abde-4967-9151-a25db7f94553 None None] MFA Rules not processed for user `891005fa5ff74041b75747d16cc66798`. Rule list: `[]` (Enabled: `True`). {{(pid=80378) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 02 10:03:24.672644 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.fernet_utils [None req-fa4f1714-abde-4967-9151-a25db7f94553 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80378) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 10:03:24.695276 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG dbcounter [-] [80377] Writing DB stats keystone:SELECT=100 {{(pid=80377) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 10:03:24.710260 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: [pid: 80377|app: 0|req: 501/985] 10.222.0.82 () {58 vars in 1282 bytes} [Tue Aug 2 10:03:24 2022] GET /identity/v3/auth/tokens => generated 1570 bytes in 538 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Aug 02 10:03:24.712958 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-593a5450-dc19-4caf-a920-9899a6039e64 None admin] Authenticating user token {{(pid=80377) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:03:24.816895 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 485/986] 10.222.0.82 () {58 vars in 1023 bytes} [Tue Aug 2 10:03:24 2022] POST /identity/v3/auth/tokens => generated 1570 bytes in 180 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 02 10:03:24.847457 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa4f1714-abde-4967-9151-a25db7f94553 None None] Authenticating user token {{(pid=80378) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:03:24.910350 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-593a5450-dc19-4caf-a920-9899a6039e64 None admin] Validating token access rules against request {{(pid=80377) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:03:24.970196 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9924107e-2009-4c85-a1b8-64253b7cb37f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '891005fa5ff74041b75747d16cc66798', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80377) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:03:24.971408 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9924107e-2009-4c85-a1b8-64253b7cb37f None admin] REQUEST_METHOD: `GET` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:03:24.971866 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9924107e-2009-4c85-a1b8-64253b7cb37f None admin] SCRIPT_NAME: `/identity` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:03:24.972386 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9924107e-2009-4c85-a1b8-64253b7cb37f None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:03:24.991603 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9924107e-2009-4c85-a1b8-64253b7cb37f None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:03:24.993864 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9924107e-2009-4c85-a1b8-64253b7cb37f None admin] RBAC: Authorization granted {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:03:25.010399 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG dbcounter [-] [80378] Writing DB stats keystone:SELECT=100 {{(pid=80378) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 10:03:25.017935 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application [None req-9924107e-2009-4c85-a1b8-64253b7cb37f None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 02 10:03:25.017935 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 02 10:03:25.017935 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 02 10:03:25.017935 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 02 10:03:25.017935 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 02 10:03:25.017935 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 02 10:03:25.017935 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 02 10:03:25.017935 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 02 10:03:25.017935 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 02 10:03:25.017935 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 02 10:03:25.017935 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 02 10:03:25.017935 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 02 10:03:25.017935 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 02 10:03:25.017935 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 02 10:03:25.017935 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 02 10:03:25.019791 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 02 10:03:25.019791 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 02 10:03:25.019791 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 02 10:03:25.019791 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 02 10:03:25.019791 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 02 10:03:25.019791 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 02 10:03:25.019791 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application return self.get_or_create( Aug 02 10:03:25.019791 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 02 10:03:25.019791 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application with Lock( Aug 02 10:03:25.019791 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 02 10:03:25.019791 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application return self._enter() Aug 02 10:03:25.019791 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 02 10:03:25.019791 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 02 10:03:25.019791 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 02 10:03:25.019791 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application return self.creator() Aug 02 10:03:25.019791 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 02 10:03:25.019791 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application created_value = creator( Aug 02 10:03:25.021742 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 02 10:03:25.021742 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 02 10:03:25.021742 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 02 10:03:25.021742 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application Aug 02 10:03:25.021742 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: [pid: 80377|app: 0|req: 502/987] 10.222.0.82 () {56 vars in 1108 bytes} [Tue Aug 2 10:03:24 2022] GET /identity/v3/domains/Default => generated 87 bytes in 308 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 02 10:03:25.021742 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9536123-7b8d-4f4d-84d9-d9470763d66c None None] REQUEST_METHOD: `GET` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:03:25.021742 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9536123-7b8d-4f4d-84d9-d9470763d66c None None] SCRIPT_NAME: `/identity` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:03:25.021742 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9536123-7b8d-4f4d-84d9-d9470763d66c None None] PATH_INFO: `/` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:03:25.025971 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: [pid: 80377|app: 0|req: 503/988] 10.222.0.82 () {54 vars in 926 bytes} [Tue Aug 2 10:03:25 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 02 10:03:25.028252 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9536123-7b8d-4f4d-84d9-d9470763d66c None None] Authenticating user token {{(pid=80377) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:03:25.042525 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa4f1714-abde-4967-9151-a25db7f94553 None None] Validating token access rules against request {{(pid=80378) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:03:25.091165 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39d1d4a4-7714-4ca3-b66c-cf2394b3d732 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '891005fa5ff74041b75747d16cc66798', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80378) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:03:25.092185 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39d1d4a4-7714-4ca3-b66c-cf2394b3d732 None admin] REQUEST_METHOD: `GET` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:03:25.092524 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39d1d4a4-7714-4ca3-b66c-cf2394b3d732 None admin] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:03:25.092833 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39d1d4a4-7714-4ca3-b66c-cf2394b3d732 None admin] PATH_INFO: `/v3/services/compute` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:03:25.093683 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39d1d4a4-7714-4ca3-b66c-cf2394b3d732 None admin] RBAC: Authorizing `identity:get_service(service_id=compute)` {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:03:25.096147 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39d1d4a4-7714-4ca3-b66c-cf2394b3d732 None admin] RBAC: Authorization granted {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:03:25.102222 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG dbcounter [-] [80377] Writing DB stats keystone:SELECT=100 {{(pid=80377) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 10:03:25.106601 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application [None req-39d1d4a4-7714-4ca3-b66c-cf2394b3d732 None admin] Could not find service: compute.: keystone.exception.ServiceNotFound: Could not find service: compute. Aug 02 10:03:25.106601 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 02 10:03:25.106601 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Aug 02 10:03:25.106601 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Aug 02 10:03:25.106601 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Aug 02 10:03:25.106601 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Aug 02 10:03:25.106601 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Aug 02 10:03:25.106601 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 02 10:03:25.106601 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute. Aug 02 10:03:25.106601 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application Aug 02 10:03:25.106601 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 02 10:03:25.106601 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application Aug 02 10:03:25.106601 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 02 10:03:25.106601 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 02 10:03:25.106601 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 02 10:03:25.106601 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 02 10:03:25.106601 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 02 10:03:25.108405 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 02 10:03:25.108405 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 02 10:03:25.108405 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 02 10:03:25.108405 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 02 10:03:25.108405 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 02 10:03:25.108405 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 02 10:03:25.108405 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Aug 02 10:03:25.108405 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application return self._get_service(service_id) Aug 02 10:03:25.108405 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Aug 02 10:03:25.108405 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Aug 02 10:03:25.108405 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 02 10:03:25.108405 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 02 10:03:25.108405 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 02 10:03:25.108405 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 02 10:03:25.108405 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 02 10:03:25.108405 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application return self.get_or_create( Aug 02 10:03:25.110314 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 02 10:03:25.110314 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application with Lock( Aug 02 10:03:25.110314 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 02 10:03:25.110314 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application return self._enter() Aug 02 10:03:25.110314 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 02 10:03:25.110314 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 02 10:03:25.110314 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 02 10:03:25.110314 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application return self.creator() Aug 02 10:03:25.110314 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 02 10:03:25.110314 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application created_value = creator( Aug 02 10:03:25.110314 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Aug 02 10:03:25.110314 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 02 10:03:25.110314 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute. Aug 02 10:03:25.110314 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application Aug 02 10:03:25.110314 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 486/989] 10.222.0.82 () {56 vars in 1111 bytes} [Tue Aug 2 10:03:24 2022] GET /identity/v3/services/compute => generated 88 bytes in 262 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 02 10:03:25.112656 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d9a44bd-9907-43ff-be08-0ef0d4aa685c None None] REQUEST_METHOD: `POST` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:03:25.112656 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d9a44bd-9907-43ff-be08-0ef0d4aa685c None None] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:03:25.112656 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d9a44bd-9907-43ff-be08-0ef0d4aa685c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:03:25.131268 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.auth.core [None req-2d9a44bd-9907-43ff-be08-0ef0d4aa685c None None] MFA Rules not processed for user `891005fa5ff74041b75747d16cc66798`. Rule list: `[]` (Enabled: `True`). {{(pid=80378) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 02 10:03:25.132615 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.fernet_utils [None req-2d9a44bd-9907-43ff-be08-0ef0d4aa685c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80378) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 10:03:25.191330 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9536123-7b8d-4f4d-84d9-d9470763d66c None None] Validating token access rules against request {{(pid=80377) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:03:25.239326 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 487/990] 10.222.0.82 () {58 vars in 1023 bytes} [Tue Aug 2 10:03:25 2022] POST /identity/v3/auth/tokens => generated 1570 bytes in 131 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 02 10:03:25.240277 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d9a44bd-9907-43ff-be08-0ef0d4aa685c None None] Authenticating user token {{(pid=80378) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:03:25.242560 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6d1dd76-6df7-4757-9c8c-9c361b0a7fd6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '891005fa5ff74041b75747d16cc66798', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80377) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:03:25.243494 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6d1dd76-6df7-4757-9c8c-9c361b0a7fd6 None admin] REQUEST_METHOD: `GET` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:03:25.243907 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6d1dd76-6df7-4757-9c8c-9c361b0a7fd6 None admin] SCRIPT_NAME: `/identity` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:03:25.244323 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6d1dd76-6df7-4757-9c8c-9c361b0a7fd6 None admin] PATH_INFO: `/v3/domains` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:03:25.245227 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6d1dd76-6df7-4757-9c8c-9c361b0a7fd6 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80377) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 02 10:03:25.245727 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6d1dd76-6df7-4757-9c8c-9c361b0a7fd6 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:03:25.247362 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6d1dd76-6df7-4757-9c8c-9c361b0a7fd6 None admin] RBAC: Authorization granted {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:03:25.265642 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: [pid: 80377|app: 0|req: 504/991] 10.222.0.82 () {56 vars in 1109 bytes} [Tue Aug 2 10:03:25 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 237 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 02 10:03:25.272176 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6d1dd76-6df7-4757-9c8c-9c361b0a7fd6 None admin] Authenticating user token {{(pid=80377) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:03:25.364330 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG dbcounter [-] [80378] Writing DB stats keystone:SELECT=100 {{(pid=80378) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 10:03:25.395603 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d9a44bd-9907-43ff-be08-0ef0d4aa685c None None] Validating token access rules against request {{(pid=80378) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:03:25.419063 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6d1dd76-6df7-4757-9c8c-9c361b0a7fd6 None admin] Validating token access rules against request {{(pid=80377) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:03:25.433831 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG dbcounter [-] [80377] Writing DB stats keystone:SELECT=100 {{(pid=80377) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 10:03:25.455633 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2bd76d88-44b2-43c2-998d-9afb8c45b805 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '891005fa5ff74041b75747d16cc66798', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80378) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:03:25.459549 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bd76d88-44b2-43c2-998d-9afb8c45b805 None admin] REQUEST_METHOD: `GET` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:03:25.459849 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bd76d88-44b2-43c2-998d-9afb8c45b805 None admin] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:03:25.460131 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bd76d88-44b2-43c2-998d-9afb8c45b805 None admin] PATH_INFO: `/v3/services` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:03:25.460958 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2bd76d88-44b2-43c2-998d-9afb8c45b805 None admin] RBAC: Adding query filter params (name=compute) {{(pid=80378) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 02 10:03:25.461168 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2bd76d88-44b2-43c2-998d-9afb8c45b805 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:03:25.462632 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2bd76d88-44b2-43c2-998d-9afb8c45b805 None admin] RBAC: Authorization granted {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:03:25.470106 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 488/992] 10.222.0.82 () {56 vars in 1112 bytes} [Tue Aug 2 10:03:25 2022] GET /identity/v3/services?name=compute => generated 124 bytes in 229 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 02 10:03:25.470106 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2bd76d88-44b2-43c2-998d-9afb8c45b805 None admin] Authenticating user token {{(pid=80378) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:03:25.480341 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a813a6e-c25e-4093-8bb1-3f2f1e3a0b05 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '891005fa5ff74041b75747d16cc66798', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80377) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:03:25.481246 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a813a6e-c25e-4093-8bb1-3f2f1e3a0b05 None admin] REQUEST_METHOD: `GET` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:03:25.481653 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a813a6e-c25e-4093-8bb1-3f2f1e3a0b05 None admin] SCRIPT_NAME: `/identity` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:03:25.482229 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a813a6e-c25e-4093-8bb1-3f2f1e3a0b05 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:03:25.492962 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a813a6e-c25e-4093-8bb1-3f2f1e3a0b05 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:03:25.495657 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a813a6e-c25e-4093-8bb1-3f2f1e3a0b05 None admin] RBAC: Authorization granted {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:03:25.511035 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application [None req-0a813a6e-c25e-4093-8bb1-3f2f1e3a0b05 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 02 10:03:25.511035 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 02 10:03:25.511035 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 02 10:03:25.511035 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 02 10:03:25.511035 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 02 10:03:25.511035 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 02 10:03:25.511035 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 02 10:03:25.511035 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 02 10:03:25.511035 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 02 10:03:25.511035 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 02 10:03:25.511035 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 02 10:03:25.511035 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 02 10:03:25.511035 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 02 10:03:25.511035 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 02 10:03:25.511035 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 02 10:03:25.512058 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 02 10:03:25.512058 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 02 10:03:25.512058 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 02 10:03:25.512058 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 02 10:03:25.512058 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 02 10:03:25.512058 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 02 10:03:25.512058 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application return self.get_or_create( Aug 02 10:03:25.512058 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 02 10:03:25.512058 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application with Lock( Aug 02 10:03:25.512058 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 02 10:03:25.512058 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application return self._enter() Aug 02 10:03:25.512058 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 02 10:03:25.512058 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 02 10:03:25.512058 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 02 10:03:25.512058 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application return self.creator() Aug 02 10:03:25.512058 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 02 10:03:25.512058 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application created_value = creator( Aug 02 10:03:25.513000 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 02 10:03:25.513000 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 02 10:03:25.513000 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 02 10:03:25.513000 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 02 10:03:25.513000 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 02 10:03:25.513000 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 02 10:03:25.513000 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 02 10:03:25.513000 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application Aug 02 10:03:25.513000 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: [pid: 80377|app: 0|req: 505/993] 10.222.0.82 () {56 vars in 1111 bytes} [Tue Aug 2 10:03:25 2022] GET /identity/v3/projects/service => generated 88 bytes in 241 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 02 10:03:25.513664 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a813a6e-c25e-4093-8bb1-3f2f1e3a0b05 None admin] Authenticating user token {{(pid=80377) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:03:25.639623 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2bd76d88-44b2-43c2-998d-9afb8c45b805 None admin] Validating token access rules against request {{(pid=80378) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:03:25.658835 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a813a6e-c25e-4093-8bb1-3f2f1e3a0b05 None admin] Validating token access rules against request {{(pid=80377) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:03:25.687746 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7035599-dce3-4083-b9ce-13f566cef3b5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '891005fa5ff74041b75747d16cc66798', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80378) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:03:25.688675 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7035599-dce3-4083-b9ce-13f566cef3b5 None admin] REQUEST_METHOD: `GET` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:03:25.688955 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7035599-dce3-4083-b9ce-13f566cef3b5 None admin] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:03:25.689258 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7035599-dce3-4083-b9ce-13f566cef3b5 None admin] PATH_INFO: `/v3/services/object-store` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:03:25.691122 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7035599-dce3-4083-b9ce-13f566cef3b5 None admin] RBAC: Authorizing `identity:get_service(service_id=object-store)` {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:03:25.693325 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7035599-dce3-4083-b9ce-13f566cef3b5 None admin] RBAC: Authorization granted {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:03:25.705939 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-092a00d7-6ac9-4a0c-b06f-68655a61010c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '891005fa5ff74041b75747d16cc66798', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80377) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:03:25.706831 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application [None req-d7035599-dce3-4083-b9ce-13f566cef3b5 None admin] Could not find service: object-store.: keystone.exception.ServiceNotFound: Could not find service: object-store. Aug 02 10:03:25.706831 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 02 10:03:25.706831 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Aug 02 10:03:25.706831 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Aug 02 10:03:25.706831 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Aug 02 10:03:25.706831 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Aug 02 10:03:25.706831 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Aug 02 10:03:25.706831 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 02 10:03:25.706831 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Aug 02 10:03:25.706831 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application Aug 02 10:03:25.706831 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 02 10:03:25.706831 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application Aug 02 10:03:25.706831 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 02 10:03:25.706831 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 02 10:03:25.706831 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 02 10:03:25.706831 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 02 10:03:25.706831 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 02 10:03:25.708963 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 02 10:03:25.708963 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 02 10:03:25.708963 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 02 10:03:25.708963 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 02 10:03:25.708963 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 02 10:03:25.708963 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 02 10:03:25.708963 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Aug 02 10:03:25.708963 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application return self._get_service(service_id) Aug 02 10:03:25.708963 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Aug 02 10:03:25.708963 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Aug 02 10:03:25.708963 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 02 10:03:25.708963 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 02 10:03:25.708963 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 02 10:03:25.708963 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 02 10:03:25.708963 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 02 10:03:25.708963 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application return self.get_or_create( Aug 02 10:03:25.711058 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 02 10:03:25.711058 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application with Lock( Aug 02 10:03:25.711058 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 02 10:03:25.711058 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application return self._enter() Aug 02 10:03:25.711058 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 02 10:03:25.711058 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 02 10:03:25.711058 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 02 10:03:25.711058 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application return self.creator() Aug 02 10:03:25.711058 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 02 10:03:25.711058 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application created_value = creator( Aug 02 10:03:25.711058 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Aug 02 10:03:25.711058 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 02 10:03:25.711058 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Aug 02 10:03:25.711058 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application Aug 02 10:03:25.713017 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-092a00d7-6ac9-4a0c-b06f-68655a61010c None admin] REQUEST_METHOD: `GET` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:03:25.713017 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-092a00d7-6ac9-4a0c-b06f-68655a61010c None admin] SCRIPT_NAME: `/identity` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:03:25.713017 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-092a00d7-6ac9-4a0c-b06f-68655a61010c None admin] PATH_INFO: `/v3/services` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:03:25.713017 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-092a00d7-6ac9-4a0c-b06f-68655a61010c None admin] RBAC: Adding query filter params (type=compute) {{(pid=80377) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 02 10:03:25.713564 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 489/994] 10.222.0.82 () {56 vars in 1126 bytes} [Tue Aug 2 10:03:25 2022] GET /identity/v3/services/object-store => generated 93 bytes in 240 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 02 10:03:25.713564 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7035599-dce3-4083-b9ce-13f566cef3b5 None admin] Authenticating user token {{(pid=80378) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:03:25.714134 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-092a00d7-6ac9-4a0c-b06f-68655a61010c None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:03:25.717005 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-092a00d7-6ac9-4a0c-b06f-68655a61010c None admin] RBAC: Authorization granted {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:03:25.725881 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: [pid: 80377|app: 0|req: 506/995] 10.222.0.82 () {56 vars in 1112 bytes} [Tue Aug 2 10:03:25 2022] GET /identity/v3/services?type=compute => generated 352 bytes in 213 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 02 10:03:25.727085 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-092a00d7-6ac9-4a0c-b06f-68655a61010c None admin] Authenticating user token {{(pid=80377) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:03:25.733580 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG dbcounter [-] [80378] Writing DB stats keystone:SELECT=100 {{(pid=80378) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 10:03:25.776163 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG dbcounter [-] [80377] Writing DB stats keystone:SELECT=100 {{(pid=80377) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 10:03:25.860486 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-092a00d7-6ac9-4a0c-b06f-68655a61010c None admin] Validating token access rules against request {{(pid=80377) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:03:25.872379 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7035599-dce3-4083-b9ce-13f566cef3b5 None admin] Validating token access rules against request {{(pid=80378) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:03:25.917885 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dcfc3931-98e6-42d6-96c3-4039adb27687 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '891005fa5ff74041b75747d16cc66798', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80377) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:03:25.918535 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcfc3931-98e6-42d6-96c3-4039adb27687 None admin] REQUEST_METHOD: `GET` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:03:25.918823 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcfc3931-98e6-42d6-96c3-4039adb27687 None admin] SCRIPT_NAME: `/identity` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:03:25.919095 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcfc3931-98e6-42d6-96c3-4039adb27687 None admin] PATH_INFO: `/v3/services` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:03:25.919792 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dcfc3931-98e6-42d6-96c3-4039adb27687 None admin] RBAC: Adding query filter params (name=object-store) {{(pid=80377) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 02 10:03:25.920000 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-696e4252-b93e-4e8f-a785-12b8a483c46d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '891005fa5ff74041b75747d16cc66798', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80378) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:03:25.920126 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dcfc3931-98e6-42d6-96c3-4039adb27687 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:03:25.921095 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-696e4252-b93e-4e8f-a785-12b8a483c46d None admin] REQUEST_METHOD: `GET` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:03:25.921486 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dcfc3931-98e6-42d6-96c3-4039adb27687 None admin] RBAC: Authorization granted {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:03:25.921607 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-696e4252-b93e-4e8f-a785-12b8a483c46d None admin] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:03:25.922092 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-696e4252-b93e-4e8f-a785-12b8a483c46d None admin] PATH_INFO: `/v3/projects` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:03:25.925001 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-696e4252-b93e-4e8f-a785-12b8a483c46d None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80378) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 02 10:03:25.925001 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-696e4252-b93e-4e8f-a785-12b8a483c46d None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:03:25.927289 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-696e4252-b93e-4e8f-a785-12b8a483c46d None admin] RBAC: Authorization granted {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:03:25.934881 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: [pid: 80377|app: 0|req: 507/996] 10.222.0.82 () {56 vars in 1122 bytes} [Tue Aug 2 10:03:25 2022] GET /identity/v3/services?name=object-store => generated 129 bytes in 208 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 02 10:03:25.935741 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dcfc3931-98e6-42d6-96c3-4039adb27687 None admin] Authenticating user token {{(pid=80377) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:03:25.947254 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 490/997] 10.222.0.82 () {56 vars in 1148 bytes} [Tue Aug 2 10:03:25 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 237 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 02 10:03:25.948053 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-696e4252-b93e-4e8f-a785-12b8a483c46d None admin] Authenticating user token {{(pid=80378) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:03:26.033355 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG dbcounter [-] [80378] Writing DB stats keystone:SELECT=100 {{(pid=80378) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 10:03:26.057980 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG dbcounter [-] [80377] Writing DB stats keystone:SELECT=100 {{(pid=80377) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 10:03:26.063007 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dcfc3931-98e6-42d6-96c3-4039adb27687 None admin] Validating token access rules against request {{(pid=80377) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:03:26.081937 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-696e4252-b93e-4e8f-a785-12b8a483c46d None admin] Validating token access rules against request {{(pid=80378) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:03:26.113371 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4527736-3d4d-4f26-9b67-e36849d29f99 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '891005fa5ff74041b75747d16cc66798', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80377) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:03:26.114027 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4527736-3d4d-4f26-9b67-e36849d29f99 None admin] REQUEST_METHOD: `GET` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:03:26.114303 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4527736-3d4d-4f26-9b67-e36849d29f99 None admin] SCRIPT_NAME: `/identity` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:03:26.114582 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4527736-3d4d-4f26-9b67-e36849d29f99 None admin] PATH_INFO: `/v3/endpoints` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:03:26.115938 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4527736-3d4d-4f26-9b67-e36849d29f99 None admin] RBAC: Adding query filter params (interface=public, service_id=fd3d214dcf0f4e82b235168100afd730, region_id=RegionOne) {{(pid=80377) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 02 10:03:26.116433 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4527736-3d4d-4f26-9b67-e36849d29f99 None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:03:26.118445 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4527736-3d4d-4f26-9b67-e36849d29f99 None admin] RBAC: Authorization granted {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:03:26.120422 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3154798-2809-473c-b0d5-7847e2a1212e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '891005fa5ff74041b75747d16cc66798', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80378) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:03:26.121045 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3154798-2809-473c-b0d5-7847e2a1212e None admin] REQUEST_METHOD: `GET` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:03:26.121327 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3154798-2809-473c-b0d5-7847e2a1212e None admin] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:03:26.121608 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3154798-2809-473c-b0d5-7847e2a1212e None admin] PATH_INFO: `/v3/services` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:03:26.122323 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3154798-2809-473c-b0d5-7847e2a1212e None admin] RBAC: Adding query filter params (type=object-store) {{(pid=80378) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 02 10:03:26.122619 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3154798-2809-473c-b0d5-7847e2a1212e None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:03:26.123990 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3154798-2809-473c-b0d5-7847e2a1212e None admin] RBAC: Authorization granted {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:03:26.132355 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: [pid: 80377|app: 0|req: 508/998] 10.222.0.82 () {56 vars in 1251 bytes} [Tue Aug 2 10:03:25 2022] GET /identity/v3/endpoints?service_id=fd3d214dcf0f4e82b235168100afd730&interface=public®ion_id=RegionOne => generated 194 bytes in 191 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 02 10:03:26.132355 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4527736-3d4d-4f26-9b67-e36849d29f99 None admin] Authenticating user token {{(pid=80377) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:03:26.132629 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 491/999] 10.222.0.82 () {56 vars in 1122 bytes} [Tue Aug 2 10:03:25 2022] GET /identity/v3/services?type=object-store => generated 356 bytes in 182 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 02 10:03:26.133694 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3154798-2809-473c-b0d5-7847e2a1212e None admin] Authenticating user token {{(pid=80378) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:03:26.284086 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3154798-2809-473c-b0d5-7847e2a1212e None admin] Validating token access rules against request {{(pid=80378) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:03:26.303231 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4527736-3d4d-4f26-9b67-e36849d29f99 None admin] Validating token access rules against request {{(pid=80377) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:03:26.328011 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG dbcounter [-] [80378] Writing DB stats keystone:SELECT=100 {{(pid=80378) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 10:03:26.334995 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82a5700d-39b6-4ce2-a1f3-382be716f189 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '891005fa5ff74041b75747d16cc66798', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80378) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:03:26.335690 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82a5700d-39b6-4ce2-a1f3-382be716f189 None admin] REQUEST_METHOD: `GET` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:03:26.336060 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82a5700d-39b6-4ce2-a1f3-382be716f189 None admin] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:03:26.336249 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82a5700d-39b6-4ce2-a1f3-382be716f189 None admin] PATH_INFO: `/v3/services` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:03:26.337063 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82a5700d-39b6-4ce2-a1f3-382be716f189 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:03:26.338477 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82a5700d-39b6-4ce2-a1f3-382be716f189 None admin] RBAC: Authorization granted {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:03:26.345284 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 492/1000] 10.222.0.82 () {56 vars in 1087 bytes} [Tue Aug 2 10:03:26 2022] GET /identity/v3/services => generated 1727 bytes in 212 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 02 10:03:26.346836 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82a5700d-39b6-4ce2-a1f3-382be716f189 None admin] Authenticating user token {{(pid=80378) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:03:26.350078 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-338bd043-3d52-4d90-b224-a0f115eda87d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '891005fa5ff74041b75747d16cc66798', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80377) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:03:26.351703 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-338bd043-3d52-4d90-b224-a0f115eda87d None admin] REQUEST_METHOD: `GET` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:03:26.352004 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-338bd043-3d52-4d90-b224-a0f115eda87d None admin] SCRIPT_NAME: `/identity` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:03:26.352285 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-338bd043-3d52-4d90-b224-a0f115eda87d None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:03:26.353259 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-338bd043-3d52-4d90-b224-a0f115eda87d None admin] RBAC: Adding query filter params (role.id=f57a652471c8414597c5fdde3d6f4636, scope.project.id=f1b38ebb804c41c69772c998aae72c41, user.id=bd8dd72086db48b8aaac2ed0058872df) {{(pid=80377) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 02 10:03:26.353728 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-338bd043-3d52-4d90-b224-a0f115eda87d None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:03:26.355343 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-338bd043-3d52-4d90-b224-a0f115eda87d None admin] RBAC: Authorization granted {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:03:26.368416 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: [pid: 80377|app: 0|req: 509/1001] 10.222.0.82 () {56 vars in 1374 bytes} [Tue Aug 2 10:03:26 2022] GET /identity/v3/role_assignments?user.id=bd8dd72086db48b8aaac2ed0058872df&scope.project.id=f1b38ebb804c41c69772c998aae72c41&role.id=f57a652471c8414597c5fdde3d6f4636 => generated 608 bytes in 240 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 02 10:03:26.561186 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82a5700d-39b6-4ce2-a1f3-382be716f189 None admin] Validating token access rules against request {{(pid=80378) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:03:26.616348 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a63012c-cfb5-4658-8bb2-dc5693997c16 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '891005fa5ff74041b75747d16cc66798', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80378) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:03:26.617344 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a63012c-cfb5-4658-8bb2-dc5693997c16 None admin] REQUEST_METHOD: `POST` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:03:26.617641 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a63012c-cfb5-4658-8bb2-dc5693997c16 None admin] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:03:26.617979 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a63012c-cfb5-4658-8bb2-dc5693997c16 None admin] PATH_INFO: `/v3/endpoints` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:03:26.618866 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a63012c-cfb5-4658-8bb2-dc5693997c16 None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:03:26.621372 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a63012c-cfb5-4658-8bb2-dc5693997c16 None admin] RBAC: Authorization granted {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:03:26.652501 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 493/1002] 10.222.0.82 () {60 vars in 1144 bytes} [Tue Aug 2 10:03:26 2022] POST /identity/v3/endpoints => generated 325 bytes in 307 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 02 10:03:28.822331 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ccca982-22db-4b5e-9ac2-a6066290546d None None] REQUEST_METHOD: `GET` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:03:28.823250 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ccca982-22db-4b5e-9ac2-a6066290546d None None] SCRIPT_NAME: `/identity` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:03:28.823535 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ccca982-22db-4b5e-9ac2-a6066290546d None None] PATH_INFO: `/` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:03:28.825173 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: [pid: 80377|app: 0|req: 510/1003] 10.222.0.82 () {54 vars in 926 bytes} [Tue Aug 2 10:03:28 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 02 10:03:28.830663 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d26d49b-3c66-4c83-9a26-fddfa6c98d80 None None] REQUEST_METHOD: `POST` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:03:28.831504 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d26d49b-3c66-4c83-9a26-fddfa6c98d80 None None] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:03:28.831504 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d26d49b-3c66-4c83-9a26-fddfa6c98d80 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:03:28.852133 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.auth.core [None req-0d26d49b-3c66-4c83-9a26-fddfa6c98d80 None None] MFA Rules not processed for user `891005fa5ff74041b75747d16cc66798`. Rule list: `[]` (Enabled: `True`). {{(pid=80378) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 02 10:03:28.853355 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.fernet_utils [None req-0d26d49b-3c66-4c83-9a26-fddfa6c98d80 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80378) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 10:03:28.860360 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0df69d61-9dcb-4a9c-b60f-64b4a78c1e76 None None] REQUEST_METHOD: `GET` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:03:28.861844 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0df69d61-9dcb-4a9c-b60f-64b4a78c1e76 None None] SCRIPT_NAME: `/identity` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:03:28.862666 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0df69d61-9dcb-4a9c-b60f-64b4a78c1e76 None None] PATH_INFO: `/` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:03:28.865870 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: [pid: 80377|app: 0|req: 511/1004] 10.222.0.82 () {54 vars in 926 bytes} [Tue Aug 2 10:03:28 2022] GET /identity => generated 270 bytes in 7 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 02 10:03:28.872120 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c94af83-9208-42ac-9ab8-fe702949e3e2 None None] REQUEST_METHOD: `POST` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:03:28.872447 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c94af83-9208-42ac-9ab8-fe702949e3e2 None None] SCRIPT_NAME: `/identity` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:03:28.872862 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c94af83-9208-42ac-9ab8-fe702949e3e2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:03:28.903611 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.auth.core [None req-3c94af83-9208-42ac-9ab8-fe702949e3e2 None None] MFA Rules not processed for user `891005fa5ff74041b75747d16cc66798`. Rule list: `[]` (Enabled: `True`). {{(pid=80377) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 02 10:03:28.905167 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.fernet_utils [None req-3c94af83-9208-42ac-9ab8-fe702949e3e2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80377) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 10:03:28.921598 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG dbcounter [-] [80378] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80378) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 10:03:28.936740 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG dbcounter [-] [80377] Writing DB stats keystone:SELECT=100 {{(pid=80377) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 10:03:28.966032 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 494/1005] 10.222.0.82 () {58 vars in 1023 bytes} [Tue Aug 2 10:03:28 2022] POST /identity/v3/auth/tokens => generated 1717 bytes in 136 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 02 10:03:28.982364 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d26d49b-3c66-4c83-9a26-fddfa6c98d80 None None] Authenticating user token {{(pid=80378) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:03:29.022726 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: [pid: 80377|app: 0|req: 512/1006] 10.222.0.82 () {58 vars in 1023 bytes} [Tue Aug 2 10:03:28 2022] POST /identity/v3/auth/tokens => generated 1717 bytes in 151 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 02 10:03:29.046035 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c94af83-9208-42ac-9ab8-fe702949e3e2 None None] Authenticating user token {{(pid=80377) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:03:29.159498 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d26d49b-3c66-4c83-9a26-fddfa6c98d80 None None] Validating token access rules against request {{(pid=80378) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:03:29.201031 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e491ed20-e63d-4743-8320-7d6a55aaa900 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '891005fa5ff74041b75747d16cc66798', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80378) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:03:29.201707 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e491ed20-e63d-4743-8320-7d6a55aaa900 None admin] REQUEST_METHOD: `GET` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:03:29.202045 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e491ed20-e63d-4743-8320-7d6a55aaa900 None admin] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:03:29.202332 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e491ed20-e63d-4743-8320-7d6a55aaa900 None admin] PATH_INFO: `/v3/services/image` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:03:29.203064 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e491ed20-e63d-4743-8320-7d6a55aaa900 None admin] RBAC: Authorizing `identity:get_service(service_id=image)` {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:03:29.204790 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e491ed20-e63d-4743-8320-7d6a55aaa900 None admin] RBAC: Authorization granted {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:03:29.206120 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c94af83-9208-42ac-9ab8-fe702949e3e2 None None] Validating token access rules against request {{(pid=80377) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:03:29.215154 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application [None req-e491ed20-e63d-4743-8320-7d6a55aaa900 None admin] Could not find service: image.: keystone.exception.ServiceNotFound: Could not find service: image. Aug 02 10:03:29.215154 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 02 10:03:29.215154 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Aug 02 10:03:29.215154 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Aug 02 10:03:29.215154 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Aug 02 10:03:29.215154 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Aug 02 10:03:29.215154 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Aug 02 10:03:29.215154 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 02 10:03:29.215154 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: image. Aug 02 10:03:29.215154 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application Aug 02 10:03:29.215154 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 02 10:03:29.215154 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application Aug 02 10:03:29.215154 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 02 10:03:29.215154 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 02 10:03:29.215154 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 02 10:03:29.215154 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 02 10:03:29.215154 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 02 10:03:29.217177 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 02 10:03:29.217177 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 02 10:03:29.217177 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 02 10:03:29.217177 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 02 10:03:29.217177 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 02 10:03:29.217177 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 02 10:03:29.217177 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Aug 02 10:03:29.217177 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application return self._get_service(service_id) Aug 02 10:03:29.217177 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Aug 02 10:03:29.217177 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Aug 02 10:03:29.217177 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 02 10:03:29.217177 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 02 10:03:29.217177 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 02 10:03:29.217177 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 02 10:03:29.217177 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 02 10:03:29.217177 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application return self.get_or_create( Aug 02 10:03:29.219023 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 02 10:03:29.219023 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application with Lock( Aug 02 10:03:29.219023 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 02 10:03:29.219023 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application return self._enter() Aug 02 10:03:29.219023 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 02 10:03:29.219023 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 02 10:03:29.219023 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 02 10:03:29.219023 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application return self.creator() Aug 02 10:03:29.219023 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 02 10:03:29.219023 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application created_value = creator( Aug 02 10:03:29.219023 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Aug 02 10:03:29.219023 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 02 10:03:29.219023 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: image. Aug 02 10:03:29.219023 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application Aug 02 10:03:29.219023 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 495/1007] 10.222.0.82 () {56 vars in 1105 bytes} [Tue Aug 2 10:03:28 2022] GET /identity/v3/services/image => generated 86 bytes in 235 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 02 10:03:29.220180 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c23cfec-0c7a-40db-acdb-bce3e8eaf89e None None] REQUEST_METHOD: `GET` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:03:29.220180 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c23cfec-0c7a-40db-acdb-bce3e8eaf89e None None] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:03:29.220180 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c23cfec-0c7a-40db-acdb-bce3e8eaf89e None None] PATH_INFO: `/` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:03:29.220180 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 496/1008] 10.222.0.82 () {54 vars in 926 bytes} [Tue Aug 2 10:03:29 2022] GET /identity => generated 270 bytes in 2 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 02 10:03:29.226275 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c23cfec-0c7a-40db-acdb-bce3e8eaf89e None None] Authenticating user token {{(pid=80378) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:03:29.261649 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c51dd631-30c2-4ef6-812e-dff3bec9fe98 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '891005fa5ff74041b75747d16cc66798', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80377) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:03:29.262385 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c51dd631-30c2-4ef6-812e-dff3bec9fe98 None admin] REQUEST_METHOD: `GET` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:03:29.262711 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c51dd631-30c2-4ef6-812e-dff3bec9fe98 None admin] SCRIPT_NAME: `/identity` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:03:29.263033 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c51dd631-30c2-4ef6-812e-dff3bec9fe98 None admin] PATH_INFO: `/v3/services/compute` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:03:29.263735 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c51dd631-30c2-4ef6-812e-dff3bec9fe98 None admin] RBAC: Authorizing `identity:get_service(service_id=compute)` {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:03:29.265221 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c51dd631-30c2-4ef6-812e-dff3bec9fe98 None admin] RBAC: Authorization granted {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:03:29.275131 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application [None req-c51dd631-30c2-4ef6-812e-dff3bec9fe98 None admin] Could not find service: compute.: keystone.exception.ServiceNotFound: Could not find service: compute. Aug 02 10:03:29.275131 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 02 10:03:29.275131 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Aug 02 10:03:29.275131 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Aug 02 10:03:29.275131 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Aug 02 10:03:29.275131 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Aug 02 10:03:29.275131 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Aug 02 10:03:29.275131 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 02 10:03:29.275131 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute. Aug 02 10:03:29.275131 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application Aug 02 10:03:29.275131 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 02 10:03:29.275131 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application Aug 02 10:03:29.275131 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 02 10:03:29.275131 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 02 10:03:29.275131 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 02 10:03:29.275131 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 02 10:03:29.275131 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 02 10:03:29.276784 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 02 10:03:29.276784 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 02 10:03:29.276784 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 02 10:03:29.276784 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 02 10:03:29.276784 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 02 10:03:29.276784 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 02 10:03:29.276784 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Aug 02 10:03:29.276784 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application return self._get_service(service_id) Aug 02 10:03:29.276784 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Aug 02 10:03:29.276784 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Aug 02 10:03:29.276784 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 02 10:03:29.276784 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 02 10:03:29.276784 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 02 10:03:29.276784 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 02 10:03:29.276784 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 02 10:03:29.276784 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application return self.get_or_create( Aug 02 10:03:29.278176 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 02 10:03:29.278176 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application with Lock( Aug 02 10:03:29.278176 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 02 10:03:29.278176 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application return self._enter() Aug 02 10:03:29.278176 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 02 10:03:29.278176 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 02 10:03:29.278176 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 02 10:03:29.278176 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application return self.creator() Aug 02 10:03:29.278176 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 02 10:03:29.278176 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application created_value = creator( Aug 02 10:03:29.278176 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Aug 02 10:03:29.278176 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 02 10:03:29.278176 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute. Aug 02 10:03:29.278176 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application Aug 02 10:03:29.278176 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: [pid: 80377|app: 0|req: 513/1009] 10.222.0.82 () {56 vars in 1111 bytes} [Tue Aug 2 10:03:29 2022] GET /identity/v3/services/compute => generated 88 bytes in 231 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 02 10:03:29.281021 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-014ddc66-e754-44fd-8e62-2ebf502f689a None None] REQUEST_METHOD: `POST` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:03:29.281021 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-014ddc66-e754-44fd-8e62-2ebf502f689a None None] SCRIPT_NAME: `/identity` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:03:29.281021 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-014ddc66-e754-44fd-8e62-2ebf502f689a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:03:29.298000 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.auth.core [None req-014ddc66-e754-44fd-8e62-2ebf502f689a None None] MFA Rules not processed for user `891005fa5ff74041b75747d16cc66798`. Rule list: `[]` (Enabled: `True`). {{(pid=80377) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 02 10:03:29.301637 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.fernet_utils [None req-014ddc66-e754-44fd-8e62-2ebf502f689a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80377) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 10:03:29.307631 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG dbcounter [-] [80378] Writing DB stats keystone:SELECT=100 {{(pid=80378) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 10:03:29.324101 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG dbcounter [-] [80377] Writing DB stats keystone:SELECT=100 {{(pid=80377) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 10:03:29.384705 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c23cfec-0c7a-40db-acdb-bce3e8eaf89e None None] Validating token access rules against request {{(pid=80378) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:03:29.398359 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: [pid: 80377|app: 0|req: 514/1010] 10.222.0.82 () {58 vars in 1023 bytes} [Tue Aug 2 10:03:29 2022] POST /identity/v3/auth/tokens => generated 1717 bytes in 121 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 02 10:03:29.399705 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-014ddc66-e754-44fd-8e62-2ebf502f689a None None] Authenticating user token {{(pid=80377) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:03:29.438124 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fb0f38f-5a68-4227-b3fc-b99ec48d89cd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '891005fa5ff74041b75747d16cc66798', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80378) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:03:29.439157 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fb0f38f-5a68-4227-b3fc-b99ec48d89cd None admin] REQUEST_METHOD: `GET` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:03:29.439708 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fb0f38f-5a68-4227-b3fc-b99ec48d89cd None admin] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:03:29.440297 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fb0f38f-5a68-4227-b3fc-b99ec48d89cd None admin] PATH_INFO: `/v3/services` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:03:29.441842 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4fb0f38f-5a68-4227-b3fc-b99ec48d89cd None admin] RBAC: Adding query filter params (name=image) {{(pid=80378) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 02 10:03:29.442504 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4fb0f38f-5a68-4227-b3fc-b99ec48d89cd None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:03:29.445429 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4fb0f38f-5a68-4227-b3fc-b99ec48d89cd None admin] RBAC: Authorization granted {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:03:29.463860 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 497/1011] 10.222.0.82 () {56 vars in 1108 bytes} [Tue Aug 2 10:03:29 2022] GET /identity/v3/services?name=image => generated 122 bytes in 231 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 02 10:03:29.463860 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fb0f38f-5a68-4227-b3fc-b99ec48d89cd None admin] Authenticating user token {{(pid=80378) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:03:29.543473 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-014ddc66-e754-44fd-8e62-2ebf502f689a None None] Validating token access rules against request {{(pid=80377) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:03:29.593482 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e0de909-f77d-487f-92ef-d19990250509 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '891005fa5ff74041b75747d16cc66798', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80377) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:03:29.594329 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e0de909-f77d-487f-92ef-d19990250509 None admin] REQUEST_METHOD: `GET` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:03:29.594788 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e0de909-f77d-487f-92ef-d19990250509 None admin] SCRIPT_NAME: `/identity` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:03:29.594973 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fb0f38f-5a68-4227-b3fc-b99ec48d89cd None admin] Validating token access rules against request {{(pid=80378) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:03:29.595256 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e0de909-f77d-487f-92ef-d19990250509 None admin] PATH_INFO: `/v3/services` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:03:29.596688 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e0de909-f77d-487f-92ef-d19990250509 None admin] RBAC: Adding query filter params (name=compute) {{(pid=80377) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 02 10:03:29.597002 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e0de909-f77d-487f-92ef-d19990250509 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:03:29.598474 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e0de909-f77d-487f-92ef-d19990250509 None admin] RBAC: Authorization granted {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:03:29.612226 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: [pid: 80377|app: 0|req: 515/1012] 10.222.0.82 () {56 vars in 1112 bytes} [Tue Aug 2 10:03:29 2022] GET /identity/v3/services?name=compute => generated 124 bytes in 205 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 02 10:03:29.612226 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e0de909-f77d-487f-92ef-d19990250509 None admin] Authenticating user token {{(pid=80377) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:03:29.612609 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG dbcounter [-] [80378] Writing DB stats keystone:SELECT=100 {{(pid=80378) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 10:03:29.633449 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG dbcounter [-] [80377] Writing DB stats keystone:SELECT=100 {{(pid=80377) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 10:03:29.640496 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ffbff52-6726-405f-8d95-61697f8c4da3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '891005fa5ff74041b75747d16cc66798', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80378) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:03:29.641158 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ffbff52-6726-405f-8d95-61697f8c4da3 None admin] REQUEST_METHOD: `GET` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:03:29.641432 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ffbff52-6726-405f-8d95-61697f8c4da3 None admin] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:03:29.641698 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ffbff52-6726-405f-8d95-61697f8c4da3 None admin] PATH_INFO: `/v3/domains/default` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:03:29.643615 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ffbff52-6726-405f-8d95-61697f8c4da3 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:03:29.645577 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ffbff52-6726-405f-8d95-61697f8c4da3 None admin] RBAC: Authorization granted {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:03:29.653870 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 498/1013] 10.222.0.82 () {56 vars in 1108 bytes} [Tue Aug 2 10:03:29 2022] GET /identity/v3/domains/default => generated 199 bytes in 191 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 02 10:03:29.653870 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ffbff52-6726-405f-8d95-61697f8c4da3 None admin] Authenticating user token {{(pid=80378) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:03:29.749704 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e0de909-f77d-487f-92ef-d19990250509 None admin] Validating token access rules against request {{(pid=80377) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:03:29.802592 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e50f303-c78a-4075-a775-0f4705771a7c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '891005fa5ff74041b75747d16cc66798', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80377) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:03:29.803225 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e50f303-c78a-4075-a775-0f4705771a7c None admin] REQUEST_METHOD: `GET` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:03:29.803508 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e50f303-c78a-4075-a775-0f4705771a7c None admin] SCRIPT_NAME: `/identity` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:03:29.803790 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e50f303-c78a-4075-a775-0f4705771a7c None admin] PATH_INFO: `/v3/services` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:03:29.804582 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e50f303-c78a-4075-a775-0f4705771a7c None admin] RBAC: Adding query filter params (type=image) {{(pid=80377) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 02 10:03:29.804895 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e50f303-c78a-4075-a775-0f4705771a7c None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:03:29.806674 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e50f303-c78a-4075-a775-0f4705771a7c None admin] RBAC: Authorization granted {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:03:29.813116 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: [pid: 80377|app: 0|req: 516/1014] 10.222.0.82 () {56 vars in 1108 bytes} [Tue Aug 2 10:03:29 2022] GET /identity/v3/services?type=image => generated 122 bytes in 208 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 02 10:03:29.814484 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e50f303-c78a-4075-a775-0f4705771a7c None admin] Authenticating user token {{(pid=80377) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:03:29.829702 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ffbff52-6726-405f-8d95-61697f8c4da3 None admin] Validating token access rules against request {{(pid=80378) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:03:29.901105 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ecf46ad-eda3-4c58-8a41-c6c4f326bb5c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '891005fa5ff74041b75747d16cc66798', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80378) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:03:29.902034 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ecf46ad-eda3-4c58-8a41-c6c4f326bb5c None admin] REQUEST_METHOD: `GET` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:03:29.902131 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ecf46ad-eda3-4c58-8a41-c6c4f326bb5c None admin] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:03:29.902393 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ecf46ad-eda3-4c58-8a41-c6c4f326bb5c None admin] PATH_INFO: `/v3/services` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:03:29.903114 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ecf46ad-eda3-4c58-8a41-c6c4f326bb5c None admin] RBAC: Adding query filter params (type=compute) {{(pid=80378) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 02 10:03:29.903419 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ecf46ad-eda3-4c58-8a41-c6c4f326bb5c None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:03:29.905028 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ecf46ad-eda3-4c58-8a41-c6c4f326bb5c None admin] RBAC: Authorization granted {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:03:29.913030 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 499/1015] 10.222.0.82 () {56 vars in 1112 bytes} [Tue Aug 2 10:03:29 2022] GET /identity/v3/services?type=compute => generated 352 bytes in 265 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 02 10:03:29.920011 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ecf46ad-eda3-4c58-8a41-c6c4f326bb5c None admin] Authenticating user token {{(pid=80378) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:03:29.966708 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG dbcounter [-] [80377] Writing DB stats keystone:SELECT=100 {{(pid=80377) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 10:03:29.998317 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG dbcounter [-] [80378] Writing DB stats keystone:SELECT=100 {{(pid=80378) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 10:03:30.011660 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e50f303-c78a-4075-a775-0f4705771a7c None admin] Validating token access rules against request {{(pid=80377) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:03:30.080008 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1bac905-d2dd-40ce-87e2-80ea1db1ee6a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '891005fa5ff74041b75747d16cc66798', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80377) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:03:30.081629 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1bac905-d2dd-40ce-87e2-80ea1db1ee6a None admin] REQUEST_METHOD: `POST` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:03:30.082179 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1bac905-d2dd-40ce-87e2-80ea1db1ee6a None admin] SCRIPT_NAME: `/identity` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:03:30.082494 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1bac905-d2dd-40ce-87e2-80ea1db1ee6a None admin] PATH_INFO: `/v3/projects` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:03:30.083206 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1bac905-d2dd-40ce-87e2-80ea1db1ee6a None admin] RBAC: Authorizing `identity:create_project()` {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:03:30.084717 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1bac905-d2dd-40ce-87e2-80ea1db1ee6a None admin] RBAC: Authorization granted {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:03:30.097646 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ecf46ad-eda3-4c58-8a41-c6c4f326bb5c None admin] Validating token access rules against request {{(pid=80378) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:03:30.126764 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: [pid: 80377|app: 0|req: 517/1016] 10.222.0.82 () {60 vars in 1141 bytes} [Tue Aug 2 10:03:29 2022] POST /identity/v3/projects => generated 311 bytes in 313 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 02 10:03:30.165448 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e408902-2331-4009-81ec-9c37146fe4ba None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '891005fa5ff74041b75747d16cc66798', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80378) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:03:30.166264 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e408902-2331-4009-81ec-9c37146fe4ba None admin] REQUEST_METHOD: `POST` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:03:30.166621 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e408902-2331-4009-81ec-9c37146fe4ba None admin] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:03:30.166977 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e408902-2331-4009-81ec-9c37146fe4ba None admin] PATH_INFO: `/v3/endpoints` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:03:30.167890 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e408902-2331-4009-81ec-9c37146fe4ba None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:03:30.169612 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e408902-2331-4009-81ec-9c37146fe4ba None admin] RBAC: Authorization granted {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:03:30.216904 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 500/1017] 10.222.0.82 () {60 vars in 1144 bytes} [Tue Aug 2 10:03:29 2022] POST /identity/v3/endpoints => generated 334 bytes in 297 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 02 10:03:32.192208 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a99f53ee-a998-4a94-9a25-4976256fa119 None None] REQUEST_METHOD: `GET` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:03:32.192713 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a99f53ee-a998-4a94-9a25-4976256fa119 None None] SCRIPT_NAME: `/identity` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:03:32.192713 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a99f53ee-a998-4a94-9a25-4976256fa119 None None] PATH_INFO: `/` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:03:32.194169 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: [pid: 80377|app: 0|req: 518/1018] 10.222.0.82 () {54 vars in 926 bytes} [Tue Aug 2 10:03:32 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 02 10:03:32.200427 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b45b56b6-0f54-4712-9155-3bacfe846768 None None] REQUEST_METHOD: `POST` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:03:32.200964 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b45b56b6-0f54-4712-9155-3bacfe846768 None None] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:03:32.200964 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b45b56b6-0f54-4712-9155-3bacfe846768 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:03:32.224012 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.auth.core [None req-b45b56b6-0f54-4712-9155-3bacfe846768 None None] MFA Rules not processed for user `891005fa5ff74041b75747d16cc66798`. Rule list: `[]` (Enabled: `True`). {{(pid=80378) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 02 10:03:32.225224 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.fernet_utils [None req-b45b56b6-0f54-4712-9155-3bacfe846768 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80378) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 10:03:32.339967 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 501/1019] 10.222.0.82 () {58 vars in 1023 bytes} [Tue Aug 2 10:03:32 2022] POST /identity/v3/auth/tokens => generated 1873 bytes in 141 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 02 10:03:32.372574 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a99f53ee-a998-4a94-9a25-4976256fa119 None None] Authenticating user token {{(pid=80377) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:03:32.518895 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a99f53ee-a998-4a94-9a25-4976256fa119 None None] Validating token access rules against request {{(pid=80377) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:03:32.566824 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG dbcounter [-] [80377] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80377) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 10:03:32.582468 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a0ffa2e-60cc-40ed-a63c-ca118771ec00 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '891005fa5ff74041b75747d16cc66798', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80377) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:03:32.583117 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a0ffa2e-60cc-40ed-a63c-ca118771ec00 None admin] REQUEST_METHOD: `POST` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:03:32.583404 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a0ffa2e-60cc-40ed-a63c-ca118771ec00 None admin] SCRIPT_NAME: `/identity` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:03:32.583680 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a0ffa2e-60cc-40ed-a63c-ca118771ec00 None admin] PATH_INFO: `/v3/services` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:03:32.584418 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a0ffa2e-60cc-40ed-a63c-ca118771ec00 None admin] RBAC: Authorizing `identity:create_service()` {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:03:32.585895 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a0ffa2e-60cc-40ed-a63c-ca118771ec00 None admin] RBAC: Authorization granted {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:03:32.596788 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41fb4093-cff9-4753-9f12-e5ee12504bf9 None None] REQUEST_METHOD: `GET` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:03:32.597556 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41fb4093-cff9-4753-9f12-e5ee12504bf9 None None] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:03:32.598255 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41fb4093-cff9-4753-9f12-e5ee12504bf9 None None] PATH_INFO: `/` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:03:32.600612 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 502/1020] 10.222.0.82 () {54 vars in 926 bytes} [Tue Aug 2 10:03:32 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 02 10:03:32.610250 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee314dfb-4a25-4210-a758-14cab9f1d295 None None] REQUEST_METHOD: `POST` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:03:32.610525 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: [pid: 80377|app: 0|req: 519/1021] 10.222.0.82 () {60 vars in 1141 bytes} [Tue Aug 2 10:03:32 2022] POST /identity/v3/services => generated 242 bytes in 238 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 02 10:03:32.611897 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee314dfb-4a25-4210-a758-14cab9f1d295 None None] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:03:32.612558 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee314dfb-4a25-4210-a758-14cab9f1d295 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:03:32.635942 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.auth.core [None req-ee314dfb-4a25-4210-a758-14cab9f1d295 None None] MFA Rules not processed for user `891005fa5ff74041b75747d16cc66798`. Rule list: `[]` (Enabled: `True`). {{(pid=80378) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 02 10:03:32.637986 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.fernet_utils [None req-ee314dfb-4a25-4210-a758-14cab9f1d295 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80378) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 10:03:32.651481 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da088f4c-ce5e-44f3-88f0-67e38243390c None None] REQUEST_METHOD: `GET` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:03:32.651481 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da088f4c-ce5e-44f3-88f0-67e38243390c None None] SCRIPT_NAME: `/identity` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:03:32.652014 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da088f4c-ce5e-44f3-88f0-67e38243390c None None] PATH_INFO: `/` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:03:32.657596 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: [pid: 80377|app: 0|req: 520/1022] 10.222.0.82 () {54 vars in 926 bytes} [Tue Aug 2 10:03:32 2022] GET /identity => generated 270 bytes in 7 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 02 10:03:32.665652 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG dbcounter [-] [80378] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80378) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 10:03:32.670449 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f13cc655-e1de-444c-9e85-6b8f630972f5 None None] REQUEST_METHOD: `POST` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:03:32.671345 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f13cc655-e1de-444c-9e85-6b8f630972f5 None None] SCRIPT_NAME: `/identity` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:03:32.671827 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f13cc655-e1de-444c-9e85-6b8f630972f5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:03:32.696703 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.auth.core [None req-f13cc655-e1de-444c-9e85-6b8f630972f5 None None] MFA Rules not processed for user `891005fa5ff74041b75747d16cc66798`. Rule list: `[]` (Enabled: `True`). {{(pid=80377) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 02 10:03:32.700184 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.fernet_utils [None req-f13cc655-e1de-444c-9e85-6b8f630972f5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80377) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 10:03:32.771727 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 503/1023] 10.222.0.82 () {58 vars in 1023 bytes} [Tue Aug 2 10:03:32 2022] POST /identity/v3/auth/tokens => generated 1969 bytes in 163 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 02 10:03:32.796763 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee314dfb-4a25-4210-a758-14cab9f1d295 None None] Authenticating user token {{(pid=80378) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:03:32.814211 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: [pid: 80377|app: 0|req: 521/1024] 10.222.0.82 () {58 vars in 1023 bytes} [Tue Aug 2 10:03:32 2022] POST /identity/v3/auth/tokens => generated 1969 bytes in 147 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Aug 02 10:03:32.840513 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f13cc655-e1de-444c-9e85-6b8f630972f5 None None] Authenticating user token {{(pid=80377) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:03:32.963019 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee314dfb-4a25-4210-a758-14cab9f1d295 None None] Validating token access rules against request {{(pid=80378) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:03:33.006508 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-791b00b8-0941-4e5c-9f2e-5f0488b18c6d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '891005fa5ff74041b75747d16cc66798', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80378) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:03:33.007610 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-791b00b8-0941-4e5c-9f2e-5f0488b18c6d None admin] REQUEST_METHOD: `GET` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:03:33.007987 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-791b00b8-0941-4e5c-9f2e-5f0488b18c6d None admin] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:03:33.008463 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-791b00b8-0941-4e5c-9f2e-5f0488b18c6d None admin] PATH_INFO: `/v3/roles/ResellerAdmin` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:03:33.008836 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f13cc655-e1de-444c-9e85-6b8f630972f5 None None] Validating token access rules against request {{(pid=80377) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:03:33.024832 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-791b00b8-0941-4e5c-9f2e-5f0488b18c6d None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 02 10:03:33.030326 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application [None req-791b00b8-0941-4e5c-9f2e-5f0488b18c6d None admin] Could not find role: ResellerAdmin.: keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Aug 02 10:03:33.030326 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 02 10:03:33.030326 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 02 10:03:33.030326 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 02 10:03:33.030326 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 02 10:03:33.030326 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 02 10:03:33.030326 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 02 10:03:33.030326 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 02 10:03:33.030326 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 02 10:03:33.030326 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 02 10:03:33.030326 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 02 10:03:33.030326 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 02 10:03:33.030326 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 02 10:03:33.030326 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 02 10:03:33.030326 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 02 10:03:33.030326 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 02 10:03:33.031352 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 02 10:03:33.031352 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 02 10:03:33.031352 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 02 10:03:33.031352 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 02 10:03:33.031352 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 02 10:03:33.031352 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 02 10:03:33.031352 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 02 10:03:33.031352 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 02 10:03:33.031352 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 02 10:03:33.031352 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application return self.get_or_create( Aug 02 10:03:33.031352 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 02 10:03:33.031352 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application with Lock( Aug 02 10:03:33.031352 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 02 10:03:33.031352 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application return self._enter() Aug 02 10:03:33.031352 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 02 10:03:33.031352 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 02 10:03:33.032510 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 02 10:03:33.032510 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application return self.creator() Aug 02 10:03:33.032510 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 02 10:03:33.032510 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application created_value = creator( Aug 02 10:03:33.032510 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 02 10:03:33.032510 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 02 10:03:33.032510 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 02 10:03:33.032510 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 02 10:03:33.032510 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 02 10:03:33.032510 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 02 10:03:33.032510 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Aug 02 10:03:33.032510 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application Aug 02 10:03:33.032510 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 504/1025] 10.222.0.82 () {56 vars in 1120 bytes} [Tue Aug 2 10:03:32 2022] GET /identity/v3/roles/ResellerAdmin => generated 91 bytes in 236 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 02 10:03:33.040017 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-791b00b8-0941-4e5c-9f2e-5f0488b18c6d None admin] Authenticating user token {{(pid=80378) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:03:33.045320 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG dbcounter [-] [80377] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80377) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 10:03:33.056079 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG dbcounter [-] [80378] Writing DB stats keystone:SELECT=100 {{(pid=80378) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 10:03:33.071260 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c55e99e-693a-46aa-bb0d-48a37d20e5f5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '891005fa5ff74041b75747d16cc66798', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80377) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:03:33.071915 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c55e99e-693a-46aa-bb0d-48a37d20e5f5 None admin] REQUEST_METHOD: `GET` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:03:33.072302 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c55e99e-693a-46aa-bb0d-48a37d20e5f5 None admin] SCRIPT_NAME: `/identity` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:03:33.072487 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c55e99e-693a-46aa-bb0d-48a37d20e5f5 None admin] PATH_INFO: `/v3/domains/default` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:03:33.073608 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c55e99e-693a-46aa-bb0d-48a37d20e5f5 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:03:33.075081 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c55e99e-693a-46aa-bb0d-48a37d20e5f5 None admin] RBAC: Authorization granted {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:03:33.076690 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: [pid: 80377|app: 0|req: 522/1026] 10.222.0.82 () {56 vars in 1108 bytes} [Tue Aug 2 10:03:32 2022] GET /identity/v3/domains/default => generated 199 bytes in 237 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 02 10:03:33.083150 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c55e99e-693a-46aa-bb0d-48a37d20e5f5 None admin] Authenticating user token {{(pid=80377) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:03:33.191979 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-791b00b8-0941-4e5c-9f2e-5f0488b18c6d None admin] Validating token access rules against request {{(pid=80378) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:03:33.241820 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20a809f7-9022-4471-b1b1-d55c132bba97 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '891005fa5ff74041b75747d16cc66798', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80378) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:03:33.242850 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20a809f7-9022-4471-b1b1-d55c132bba97 None admin] REQUEST_METHOD: `GET` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:03:33.243255 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20a809f7-9022-4471-b1b1-d55c132bba97 None admin] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:03:33.243743 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20a809f7-9022-4471-b1b1-d55c132bba97 None admin] PATH_INFO: `/v3/roles` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:03:33.244494 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20a809f7-9022-4471-b1b1-d55c132bba97 None admin] RBAC: Adding query filter params (name=ResellerAdmin) {{(pid=80378) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 02 10:03:33.244969 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20a809f7-9022-4471-b1b1-d55c132bba97 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:03:33.247259 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20a809f7-9022-4471-b1b1-d55c132bba97 None admin] RBAC: Authorization granted {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:03:33.262343 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 505/1027] 10.222.0.82 () {56 vars in 1115 bytes} [Tue Aug 2 10:03:33 2022] GET /identity/v3/roles?name=ResellerAdmin => generated 338 bytes in 223 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 02 10:03:33.265431 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c55e99e-693a-46aa-bb0d-48a37d20e5f5 None admin] Validating token access rules against request {{(pid=80377) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:03:33.272896 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20a809f7-9022-4471-b1b1-d55c132bba97 None admin] Authenticating user token {{(pid=80378) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:03:33.320763 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b103c66-d6ac-4c69-afd7-84d0c405d8d9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '891005fa5ff74041b75747d16cc66798', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80377) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:03:33.322171 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b103c66-d6ac-4c69-afd7-84d0c405d8d9 None admin] REQUEST_METHOD: `POST` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:03:33.322615 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b103c66-d6ac-4c69-afd7-84d0c405d8d9 None admin] SCRIPT_NAME: `/identity` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:03:33.323041 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b103c66-d6ac-4c69-afd7-84d0c405d8d9 None admin] PATH_INFO: `/v3/users` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:03:33.324101 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b103c66-d6ac-4c69-afd7-84d0c405d8d9 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:03:33.326639 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b103c66-d6ac-4c69-afd7-84d0c405d8d9 None admin] RBAC: Authorization granted {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:03:33.370086 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: [pid: 80377|app: 0|req: 523/1028] 10.222.0.82 () {60 vars in 1132 bytes} [Tue Aug 2 10:03:33 2022] POST /identity/v3/users => generated 285 bytes in 287 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 02 10:03:33.378205 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG dbcounter [-] [80378] Writing DB stats keystone:SELECT=100 {{(pid=80378) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 10:03:33.426622 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20a809f7-9022-4471-b1b1-d55c132bba97 None admin] Validating token access rules against request {{(pid=80378) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:03:33.480805 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-059e2804-832d-48ba-b5f7-e4c37d419ed5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '891005fa5ff74041b75747d16cc66798', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80378) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:03:33.481485 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-059e2804-832d-48ba-b5f7-e4c37d419ed5 None admin] REQUEST_METHOD: `GET` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:03:33.481770 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-059e2804-832d-48ba-b5f7-e4c37d419ed5 None admin] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:03:33.482164 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-059e2804-832d-48ba-b5f7-e4c37d419ed5 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:03:33.498629 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-059e2804-832d-48ba-b5f7-e4c37d419ed5 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:03:33.501159 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-059e2804-832d-48ba-b5f7-e4c37d419ed5 None admin] RBAC: Authorization granted {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:03:33.527497 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application [None req-059e2804-832d-48ba-b5f7-e4c37d419ed5 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 02 10:03:33.527497 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 02 10:03:33.527497 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 02 10:03:33.527497 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 02 10:03:33.527497 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 02 10:03:33.527497 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 02 10:03:33.527497 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 02 10:03:33.527497 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 02 10:03:33.527497 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 02 10:03:33.527497 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 02 10:03:33.527497 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 02 10:03:33.527497 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 02 10:03:33.527497 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 02 10:03:33.527497 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 02 10:03:33.527497 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 02 10:03:33.529315 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 02 10:03:33.529315 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 02 10:03:33.529315 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 02 10:03:33.529315 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 02 10:03:33.529315 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 02 10:03:33.529315 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 02 10:03:33.529315 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application return self.get_or_create( Aug 02 10:03:33.529315 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 02 10:03:33.529315 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application with Lock( Aug 02 10:03:33.529315 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 02 10:03:33.529315 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application return self._enter() Aug 02 10:03:33.529315 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 02 10:03:33.529315 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 02 10:03:33.529315 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 02 10:03:33.529315 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application return self.creator() Aug 02 10:03:33.529315 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 02 10:03:33.529315 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application created_value = creator( Aug 02 10:03:33.531441 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 02 10:03:33.531441 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 02 10:03:33.531441 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 02 10:03:33.531441 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application Aug 02 10:03:33.534438 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 506/1029] 10.222.0.82 () {56 vars in 1108 bytes} [Tue Aug 2 10:03:33 2022] GET /identity/v3/domains/Default => generated 87 bytes in 262 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 02 10:03:33.538900 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b103c66-d6ac-4c69-afd7-84d0c405d8d9 None admin] Authenticating user token {{(pid=80377) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:03:33.613631 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG dbcounter [-] [80377] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80377) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 10:03:33.700454 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b103c66-d6ac-4c69-afd7-84d0c405d8d9 None admin] Validating token access rules against request {{(pid=80377) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:03:33.754841 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abba1969-ff86-4c5a-8179-467b5cb9b550 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '891005fa5ff74041b75747d16cc66798', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80377) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:03:33.755476 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abba1969-ff86-4c5a-8179-467b5cb9b550 None admin] REQUEST_METHOD: `GET` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:03:33.755763 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abba1969-ff86-4c5a-8179-467b5cb9b550 None admin] SCRIPT_NAME: `/identity` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:03:33.756226 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abba1969-ff86-4c5a-8179-467b5cb9b550 None admin] PATH_INFO: `/v3/domains` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:03:33.757367 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-abba1969-ff86-4c5a-8179-467b5cb9b550 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80377) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 02 10:03:33.757974 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-abba1969-ff86-4c5a-8179-467b5cb9b550 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:03:33.760860 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-abba1969-ff86-4c5a-8179-467b5cb9b550 None admin] RBAC: Authorization granted {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:03:33.789669 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: [pid: 80377|app: 0|req: 524/1030] 10.222.0.82 () {56 vars in 1109 bytes} [Tue Aug 2 10:03:33 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 245 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 02 10:03:33.791556 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-059e2804-832d-48ba-b5f7-e4c37d419ed5 None admin] Authenticating user token {{(pid=80378) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:03:33.958459 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-059e2804-832d-48ba-b5f7-e4c37d419ed5 None admin] Validating token access rules against request {{(pid=80378) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:03:33.992480 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG dbcounter [-] [80378] Writing DB stats keystone:SELECT=100 {{(pid=80378) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 10:03:34.015945 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ddef63c-b45f-43f8-bedd-98383cbe62c5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '891005fa5ff74041b75747d16cc66798', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80378) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:03:34.015945 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ddef63c-b45f-43f8-bedd-98383cbe62c5 None admin] REQUEST_METHOD: `GET` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:03:34.015945 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ddef63c-b45f-43f8-bedd-98383cbe62c5 None admin] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:03:34.017413 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ddef63c-b45f-43f8-bedd-98383cbe62c5 None admin] PATH_INFO: `/v3/users/nova` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:03:34.027087 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ddef63c-b45f-43f8-bedd-98383cbe62c5 None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:03:34.028601 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ddef63c-b45f-43f8-bedd-98383cbe62c5 None admin] RBAC: Authorization granted {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:03:34.044144 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application [None req-6ddef63c-b45f-43f8-bedd-98383cbe62c5 None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Aug 02 10:03:34.044144 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 02 10:03:34.044144 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 02 10:03:34.044144 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 02 10:03:34.044144 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 02 10:03:34.044144 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 02 10:03:34.044144 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 02 10:03:34.044144 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 02 10:03:34.044144 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 02 10:03:34.044144 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 02 10:03:34.044144 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 02 10:03:34.044144 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 02 10:03:34.044144 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 02 10:03:34.044144 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 02 10:03:34.044144 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 02 10:03:34.044144 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 02 10:03:34.046149 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 02 10:03:34.046149 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 02 10:03:34.046149 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 02 10:03:34.046149 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 02 10:03:34.046149 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 02 10:03:34.046149 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 02 10:03:34.046149 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 02 10:03:34.046149 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 02 10:03:34.046149 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 02 10:03:34.046149 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application return self.get_or_create( Aug 02 10:03:34.046149 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 02 10:03:34.046149 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application with Lock( Aug 02 10:03:34.046149 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 02 10:03:34.046149 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application return self._enter() Aug 02 10:03:34.046149 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 02 10:03:34.046149 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 02 10:03:34.046149 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 02 10:03:34.046149 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application return self.creator() Aug 02 10:03:34.048118 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 02 10:03:34.048118 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application created_value = creator( Aug 02 10:03:34.048118 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 02 10:03:34.048118 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 02 10:03:34.048118 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 02 10:03:34.048118 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 02 10:03:34.048118 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 02 10:03:34.048118 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 02 10:03:34.048118 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Aug 02 10:03:34.048118 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application Aug 02 10:03:34.048118 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 507/1031] 10.222.0.82 () {56 vars in 1093 bytes} [Tue Aug 2 10:03:33 2022] GET /identity/v3/users/nova => generated 82 bytes in 255 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 02 10:03:34.061859 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abba1969-ff86-4c5a-8179-467b5cb9b550 None admin] Authenticating user token {{(pid=80377) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:03:34.235635 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abba1969-ff86-4c5a-8179-467b5cb9b550 None admin] Validating token access rules against request {{(pid=80377) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:03:34.238273 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG dbcounter [-] [80377] Writing DB stats keystone:SELECT=100 {{(pid=80377) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 10:03:34.290126 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fdd4e5ea-57b4-465b-9715-3e1502b4f4f7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '891005fa5ff74041b75747d16cc66798', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80377) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:03:34.291151 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdd4e5ea-57b4-465b-9715-3e1502b4f4f7 None admin] REQUEST_METHOD: `GET` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:03:34.291151 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdd4e5ea-57b4-465b-9715-3e1502b4f4f7 None admin] SCRIPT_NAME: `/identity` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:03:34.291404 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdd4e5ea-57b4-465b-9715-3e1502b4f4f7 None admin] PATH_INFO: `/v3/users` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:03:34.292133 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fdd4e5ea-57b4-465b-9715-3e1502b4f4f7 None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=80377) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 02 10:03:34.292443 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fdd4e5ea-57b4-465b-9715-3e1502b4f4f7 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:03:34.293951 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fdd4e5ea-57b4-465b-9715-3e1502b4f4f7 None admin] RBAC: Authorization granted {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:03:34.324240 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: [pid: 80377|app: 0|req: 525/1032] 10.222.0.82 () {56 vars in 1133 bytes} [Tue Aug 2 10:03:34 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 263 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 02 10:03:34.330968 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ddef63c-b45f-43f8-bedd-98383cbe62c5 None admin] Authenticating user token {{(pid=80378) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:03:34.468842 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ddef63c-b45f-43f8-bedd-98383cbe62c5 None admin] Validating token access rules against request {{(pid=80378) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:03:34.526635 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8959112-eac8-422f-b3ff-eb5081fadfcd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '891005fa5ff74041b75747d16cc66798', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80378) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:03:34.527578 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8959112-eac8-422f-b3ff-eb5081fadfcd None admin] REQUEST_METHOD: `GET` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:03:34.528069 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8959112-eac8-422f-b3ff-eb5081fadfcd None admin] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:03:34.528546 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8959112-eac8-422f-b3ff-eb5081fadfcd None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:03:34.630753 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG dbcounter [-] [80378] Writing DB stats keystone:SELECT=100 {{(pid=80378) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 10:03:34.651607 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8959112-eac8-422f-b3ff-eb5081fadfcd None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:03:34.653088 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8959112-eac8-422f-b3ff-eb5081fadfcd None admin] RBAC: Authorization granted {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:03:34.782888 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 508/1033] 10.222.0.82 () {58 vars in 1282 bytes} [Tue Aug 2 10:03:34 2022] GET /identity/v3/auth/tokens => generated 1969 bytes in 453 msecs (HTTP/1.1 200) 6 headers in 359 bytes (2 switches on core 0) Aug 02 10:03:34.790550 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fdd4e5ea-57b4-465b-9715-3e1502b4f4f7 None admin] Authenticating user token {{(pid=80377) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:03:34.942079 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fdd4e5ea-57b4-465b-9715-3e1502b4f4f7 None admin] Validating token access rules against request {{(pid=80377) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:03:34.993677 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56b5b121-0da9-45e1-9b65-67f12c93b0a0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '891005fa5ff74041b75747d16cc66798', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80377) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:03:34.994858 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56b5b121-0da9-45e1-9b65-67f12c93b0a0 None admin] REQUEST_METHOD: `GET` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:03:34.995588 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56b5b121-0da9-45e1-9b65-67f12c93b0a0 None admin] SCRIPT_NAME: `/identity` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:03:34.995870 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56b5b121-0da9-45e1-9b65-67f12c93b0a0 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:03:35.016654 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56b5b121-0da9-45e1-9b65-67f12c93b0a0 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:03:35.020055 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56b5b121-0da9-45e1-9b65-67f12c93b0a0 None admin] RBAC: Authorization granted {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:03:35.049830 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application [None req-56b5b121-0da9-45e1-9b65-67f12c93b0a0 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 02 10:03:35.049830 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 02 10:03:35.049830 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 02 10:03:35.049830 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 02 10:03:35.049830 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 02 10:03:35.049830 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 02 10:03:35.049830 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 02 10:03:35.049830 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 02 10:03:35.049830 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 02 10:03:35.049830 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 02 10:03:35.049830 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 02 10:03:35.049830 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 02 10:03:35.049830 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 02 10:03:35.049830 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 02 10:03:35.049830 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 02 10:03:35.050973 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 02 10:03:35.050973 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 02 10:03:35.050973 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 02 10:03:35.050973 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 02 10:03:35.050973 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 02 10:03:35.050973 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 02 10:03:35.050973 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application return self.get_or_create( Aug 02 10:03:35.050973 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 02 10:03:35.050973 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application with Lock( Aug 02 10:03:35.050973 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 02 10:03:35.050973 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application return self._enter() Aug 02 10:03:35.050973 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 02 10:03:35.050973 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 02 10:03:35.050973 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 02 10:03:35.050973 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application return self.creator() Aug 02 10:03:35.050973 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 02 10:03:35.050973 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application created_value = creator( Aug 02 10:03:35.052851 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 02 10:03:35.052851 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 02 10:03:35.052851 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 02 10:03:35.052851 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application Aug 02 10:03:35.054028 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: [pid: 80377|app: 0|req: 526/1034] 10.222.0.82 () {56 vars in 1108 bytes} [Tue Aug 2 10:03:34 2022] GET /identity/v3/domains/Default => generated 87 bytes in 264 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 02 10:03:35.061407 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8959112-eac8-422f-b3ff-eb5081fadfcd None admin] Authenticating user token {{(pid=80378) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:03:35.196302 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG dbcounter [-] [80378] Writing DB stats keystone:SELECT=100 {{(pid=80378) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 10:03:35.217718 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8959112-eac8-422f-b3ff-eb5081fadfcd None admin] Validating token access rules against request {{(pid=80378) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:03:35.228772 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-158bd884-92a8-4bcb-9a73-1f477b3f0315 None None] REQUEST_METHOD: `GET` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:03:35.229149 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-158bd884-92a8-4bcb-9a73-1f477b3f0315 None None] SCRIPT_NAME: `/identity` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:03:35.229560 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-158bd884-92a8-4bcb-9a73-1f477b3f0315 None None] PATH_INFO: `/` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:03:35.231315 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: [pid: 80377|app: 0|req: 527/1035] 10.222.0.82 () {54 vars in 926 bytes} [Tue Aug 2 10:03:35 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 02 10:03:35.238046 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-926deb38-e66c-4d4d-a9a8-05c6c3e08d57 None None] REQUEST_METHOD: `POST` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:03:35.238771 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-926deb38-e66c-4d4d-a9a8-05c6c3e08d57 None None] SCRIPT_NAME: `/identity` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:03:35.239066 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-926deb38-e66c-4d4d-a9a8-05c6c3e08d57 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:03:35.270490 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.auth.core [None req-926deb38-e66c-4d4d-a9a8-05c6c3e08d57 None None] MFA Rules not processed for user `891005fa5ff74041b75747d16cc66798`. Rule list: `[]` (Enabled: `True`). {{(pid=80377) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 02 10:03:35.271734 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.fernet_utils [None req-926deb38-e66c-4d4d-a9a8-05c6c3e08d57 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80377) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 10:03:35.276366 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91c1d6e5-9fb3-4f1a-9005-742b2ec45fa2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '891005fa5ff74041b75747d16cc66798', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80378) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:03:35.277012 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91c1d6e5-9fb3-4f1a-9005-742b2ec45fa2 None admin] REQUEST_METHOD: `GET` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:03:35.277293 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91c1d6e5-9fb3-4f1a-9005-742b2ec45fa2 None admin] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:03:35.277565 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91c1d6e5-9fb3-4f1a-9005-742b2ec45fa2 None admin] PATH_INFO: `/v3/domains` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:03:35.278294 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91c1d6e5-9fb3-4f1a-9005-742b2ec45fa2 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80378) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 02 10:03:35.278600 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91c1d6e5-9fb3-4f1a-9005-742b2ec45fa2 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:03:35.280493 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91c1d6e5-9fb3-4f1a-9005-742b2ec45fa2 None admin] RBAC: Authorization granted {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:03:35.283003 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG dbcounter [-] [80377] Writing DB stats keystone:SELECT=100 {{(pid=80377) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 10:03:35.309032 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 509/1036] 10.222.0.82 () {56 vars in 1109 bytes} [Tue Aug 2 10:03:35 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 241 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 02 10:03:35.310599 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91c1d6e5-9fb3-4f1a-9005-742b2ec45fa2 None admin] Authenticating user token {{(pid=80378) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:03:35.374689 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: [pid: 80377|app: 0|req: 528/1037] 10.222.0.82 () {58 vars in 1023 bytes} [Tue Aug 2 10:03:35 2022] POST /identity/v3/auth/tokens => generated 1969 bytes in 138 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 02 10:03:35.415520 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-926deb38-e66c-4d4d-a9a8-05c6c3e08d57 None None] Authenticating user token {{(pid=80377) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:03:35.476271 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91c1d6e5-9fb3-4f1a-9005-742b2ec45fa2 None admin] Validating token access rules against request {{(pid=80378) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:03:35.532111 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a5d06b4-3c4a-47d9-a38c-6f763e7c3388 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '891005fa5ff74041b75747d16cc66798', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80378) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:03:35.533281 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a5d06b4-3c4a-47d9-a38c-6f763e7c3388 None admin] REQUEST_METHOD: `GET` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:03:35.534333 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a5d06b4-3c4a-47d9-a38c-6f763e7c3388 None admin] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:03:35.534703 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a5d06b4-3c4a-47d9-a38c-6f763e7c3388 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:03:35.545558 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a5d06b4-3c4a-47d9-a38c-6f763e7c3388 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:03:35.547887 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a5d06b4-3c4a-47d9-a38c-6f763e7c3388 None admin] RBAC: Authorization granted {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:03:35.562412 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-926deb38-e66c-4d4d-a9a8-05c6c3e08d57 None None] Validating token access rules against request {{(pid=80377) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:03:35.569868 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application [None req-0a5d06b4-3c4a-47d9-a38c-6f763e7c3388 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 02 10:03:35.569868 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 02 10:03:35.569868 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 02 10:03:35.569868 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 02 10:03:35.569868 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 02 10:03:35.569868 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 02 10:03:35.569868 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 02 10:03:35.569868 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 02 10:03:35.569868 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 02 10:03:35.569868 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 02 10:03:35.569868 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 02 10:03:35.569868 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 02 10:03:35.569868 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 02 10:03:35.569868 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 02 10:03:35.569868 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 02 10:03:35.570838 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 02 10:03:35.570838 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 02 10:03:35.570838 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 02 10:03:35.570838 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 02 10:03:35.570838 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 02 10:03:35.570838 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 02 10:03:35.570838 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application return self.get_or_create( Aug 02 10:03:35.570838 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 02 10:03:35.570838 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application with Lock( Aug 02 10:03:35.570838 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 02 10:03:35.570838 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application return self._enter() Aug 02 10:03:35.570838 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 02 10:03:35.570838 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 02 10:03:35.570838 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 02 10:03:35.570838 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application return self.creator() Aug 02 10:03:35.570838 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 02 10:03:35.570838 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application created_value = creator( Aug 02 10:03:35.577318 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 02 10:03:35.577318 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 02 10:03:35.577318 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 02 10:03:35.577318 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 02 10:03:35.577318 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 02 10:03:35.577318 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 02 10:03:35.577318 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 02 10:03:35.577318 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application Aug 02 10:03:35.577318 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 510/1038] 10.222.0.82 () {56 vars in 1111 bytes} [Tue Aug 2 10:03:35 2022] GET /identity/v3/projects/service => generated 88 bytes in 262 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 02 10:03:35.578037 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a5d06b4-3c4a-47d9-a38c-6f763e7c3388 None admin] Authenticating user token {{(pid=80378) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:03:35.609582 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG dbcounter [-] [80378] Writing DB stats keystone:SELECT=100 {{(pid=80378) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 10:03:35.629695 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16063521-2024-4cbc-8dcd-dec48186c55e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '891005fa5ff74041b75747d16cc66798', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80377) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:03:35.630902 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16063521-2024-4cbc-8dcd-dec48186c55e None admin] REQUEST_METHOD: `GET` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:03:35.631345 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16063521-2024-4cbc-8dcd-dec48186c55e None admin] SCRIPT_NAME: `/identity` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:03:35.631822 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16063521-2024-4cbc-8dcd-dec48186c55e None admin] PATH_INFO: `/v3/services/image` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:03:35.632992 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16063521-2024-4cbc-8dcd-dec48186c55e None admin] RBAC: Authorizing `identity:get_service(service_id=image)` {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:03:35.635681 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16063521-2024-4cbc-8dcd-dec48186c55e None admin] RBAC: Authorization granted {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:03:35.651357 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application [None req-16063521-2024-4cbc-8dcd-dec48186c55e None admin] Could not find service: image.: keystone.exception.ServiceNotFound: Could not find service: image. Aug 02 10:03:35.651357 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 02 10:03:35.651357 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Aug 02 10:03:35.651357 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Aug 02 10:03:35.651357 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Aug 02 10:03:35.651357 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Aug 02 10:03:35.651357 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Aug 02 10:03:35.651357 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 02 10:03:35.651357 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: image. Aug 02 10:03:35.651357 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application Aug 02 10:03:35.651357 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 02 10:03:35.651357 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application Aug 02 10:03:35.651357 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 02 10:03:35.651357 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 02 10:03:35.651357 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 02 10:03:35.651357 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 02 10:03:35.651357 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 02 10:03:35.653201 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 02 10:03:35.653201 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 02 10:03:35.653201 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 02 10:03:35.653201 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 02 10:03:35.653201 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 02 10:03:35.653201 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 02 10:03:35.653201 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Aug 02 10:03:35.653201 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application return self._get_service(service_id) Aug 02 10:03:35.653201 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Aug 02 10:03:35.653201 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Aug 02 10:03:35.653201 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 02 10:03:35.653201 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 02 10:03:35.653201 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 02 10:03:35.653201 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 02 10:03:35.653201 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 02 10:03:35.653201 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application return self.get_or_create( Aug 02 10:03:35.654838 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 02 10:03:35.654838 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application with Lock( Aug 02 10:03:35.654838 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 02 10:03:35.654838 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application return self._enter() Aug 02 10:03:35.654838 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 02 10:03:35.654838 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 02 10:03:35.654838 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 02 10:03:35.654838 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application return self.creator() Aug 02 10:03:35.654838 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 02 10:03:35.654838 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application created_value = creator( Aug 02 10:03:35.654838 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Aug 02 10:03:35.654838 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 02 10:03:35.654838 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: image. Aug 02 10:03:35.654838 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application Aug 02 10:03:35.654838 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: [pid: 80377|app: 0|req: 529/1039] 10.222.0.82 () {56 vars in 1105 bytes} [Tue Aug 2 10:03:35 2022] GET /identity/v3/services/image => generated 86 bytes in 239 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 02 10:03:35.662993 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16063521-2024-4cbc-8dcd-dec48186c55e None admin] Authenticating user token {{(pid=80377) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:03:35.678592 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG dbcounter [-] [80377] Writing DB stats keystone:SELECT=100 {{(pid=80377) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 10:03:35.764365 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a5d06b4-3c4a-47d9-a38c-6f763e7c3388 None admin] Validating token access rules against request {{(pid=80378) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:03:35.811943 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16063521-2024-4cbc-8dcd-dec48186c55e None admin] Validating token access rules against request {{(pid=80377) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:03:35.831486 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e0316db-1c8b-4ee7-8d20-ae67e5d49f52 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '891005fa5ff74041b75747d16cc66798', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80378) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:03:35.832478 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e0316db-1c8b-4ee7-8d20-ae67e5d49f52 None admin] REQUEST_METHOD: `GET` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:03:35.832760 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e0316db-1c8b-4ee7-8d20-ae67e5d49f52 None admin] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:03:35.833034 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e0316db-1c8b-4ee7-8d20-ae67e5d49f52 None admin] PATH_INFO: `/v3/projects` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:03:35.833827 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e0316db-1c8b-4ee7-8d20-ae67e5d49f52 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80378) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 02 10:03:35.834140 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e0316db-1c8b-4ee7-8d20-ae67e5d49f52 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:03:35.835608 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e0316db-1c8b-4ee7-8d20-ae67e5d49f52 None admin] RBAC: Authorization granted {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:03:35.864795 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 511/1040] 10.222.0.82 () {56 vars in 1148 bytes} [Tue Aug 2 10:03:35 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 280 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 02 10:03:35.866155 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e0316db-1c8b-4ee7-8d20-ae67e5d49f52 None admin] Authenticating user token {{(pid=80378) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:03:35.866594 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9efa605b-b23c-41c9-8bbf-5a4e721bc35a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '891005fa5ff74041b75747d16cc66798', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80377) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:03:35.867543 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9efa605b-b23c-41c9-8bbf-5a4e721bc35a None admin] REQUEST_METHOD: `GET` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:03:35.868012 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9efa605b-b23c-41c9-8bbf-5a4e721bc35a None admin] SCRIPT_NAME: `/identity` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:03:35.868517 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9efa605b-b23c-41c9-8bbf-5a4e721bc35a None admin] PATH_INFO: `/v3/services` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:03:35.869578 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9efa605b-b23c-41c9-8bbf-5a4e721bc35a None admin] RBAC: Adding query filter params (name=image) {{(pid=80377) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 02 10:03:35.870168 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9efa605b-b23c-41c9-8bbf-5a4e721bc35a None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:03:35.872111 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9efa605b-b23c-41c9-8bbf-5a4e721bc35a None admin] RBAC: Authorization granted {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:03:35.885765 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: [pid: 80377|app: 0|req: 530/1041] 10.222.0.82 () {56 vars in 1108 bytes} [Tue Aug 2 10:03:35 2022] GET /identity/v3/services?name=image => generated 122 bytes in 223 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 02 10:03:35.891699 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9efa605b-b23c-41c9-8bbf-5a4e721bc35a None admin] Authenticating user token {{(pid=80377) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:03:35.966115 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG dbcounter [-] [80378] Writing DB stats keystone:SELECT=100 {{(pid=80378) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 10:03:36.007723 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG dbcounter [-] [80377] Writing DB stats keystone:SELECT=100 {{(pid=80377) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 10:03:36.040094 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e0316db-1c8b-4ee7-8d20-ae67e5d49f52 None admin] Validating token access rules against request {{(pid=80378) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:03:36.072799 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9efa605b-b23c-41c9-8bbf-5a4e721bc35a None admin] Validating token access rules against request {{(pid=80377) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:03:36.093884 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec2db9da-bba5-46f9-906e-e56d032610cf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '891005fa5ff74041b75747d16cc66798', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80378) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:03:36.094799 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec2db9da-bba5-46f9-906e-e56d032610cf None admin] REQUEST_METHOD: `GET` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:03:36.095201 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec2db9da-bba5-46f9-906e-e56d032610cf None admin] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:03:36.095561 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec2db9da-bba5-46f9-906e-e56d032610cf None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:03:36.097246 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec2db9da-bba5-46f9-906e-e56d032610cf None admin] RBAC: Adding query filter params (role.id=f57a652471c8414597c5fdde3d6f4636, scope.project.id=f1b38ebb804c41c69772c998aae72c41, user.id=88af518196674fe593ad88d30111fb7d) {{(pid=80378) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 02 10:03:36.097867 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec2db9da-bba5-46f9-906e-e56d032610cf None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:03:36.100616 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec2db9da-bba5-46f9-906e-e56d032610cf None admin] RBAC: Authorization granted {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:03:36.111082 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 512/1042] 10.222.0.82 () {56 vars in 1374 bytes} [Tue Aug 2 10:03:35 2022] GET /identity/v3/role_assignments?user.id=88af518196674fe593ad88d30111fb7d&scope.project.id=f1b38ebb804c41c69772c998aae72c41&role.id=f57a652471c8414597c5fdde3d6f4636 => generated 259 bytes in 245 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 02 10:03:36.113250 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c56fefd6-1f78-45e0-a117-f4c624179a80 None None] REQUEST_METHOD: `GET` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:03:36.113725 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c56fefd6-1f78-45e0-a117-f4c624179a80 None None] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:03:36.114125 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c56fefd6-1f78-45e0-a117-f4c624179a80 None None] PATH_INFO: `/` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:03:36.120342 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 513/1043] 10.222.0.82 () {54 vars in 926 bytes} [Tue Aug 2 10:03:36 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 02 10:03:36.125986 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f65d4e4e-8cf2-4081-b85b-b48ec654fcf6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '891005fa5ff74041b75747d16cc66798', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80377) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:03:36.127158 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f65d4e4e-8cf2-4081-b85b-b48ec654fcf6 None admin] REQUEST_METHOD: `GET` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:03:36.128377 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f65d4e4e-8cf2-4081-b85b-b48ec654fcf6 None admin] SCRIPT_NAME: `/identity` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:03:36.128843 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f65d4e4e-8cf2-4081-b85b-b48ec654fcf6 None admin] PATH_INFO: `/v3/services` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:03:36.129756 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f65d4e4e-8cf2-4081-b85b-b48ec654fcf6 None admin] RBAC: Adding query filter params (type=image) {{(pid=80377) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 02 10:03:36.129925 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a016bdd9-ff7b-438f-b2f2-210020f9dd92 None None] REQUEST_METHOD: `POST` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:03:36.130314 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a016bdd9-ff7b-438f-b2f2-210020f9dd92 None None] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:03:36.130448 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f65d4e4e-8cf2-4081-b85b-b48ec654fcf6 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:03:36.130772 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a016bdd9-ff7b-438f-b2f2-210020f9dd92 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:03:36.131816 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f65d4e4e-8cf2-4081-b85b-b48ec654fcf6 None admin] RBAC: Authorization granted {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:03:36.143693 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: [pid: 80377|app: 0|req: 531/1044] 10.222.0.82 () {56 vars in 1108 bytes} [Tue Aug 2 10:03:35 2022] GET /identity/v3/services?type=image => generated 350 bytes in 248 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 02 10:03:36.145287 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f65d4e4e-8cf2-4081-b85b-b48ec654fcf6 None admin] Authenticating user token {{(pid=80377) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:03:36.167888 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.auth.core [None req-a016bdd9-ff7b-438f-b2f2-210020f9dd92 None None] MFA Rules not processed for user `891005fa5ff74041b75747d16cc66798`. Rule list: `[]` (Enabled: `True`). {{(pid=80378) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 02 10:03:36.169150 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.fernet_utils [None req-a016bdd9-ff7b-438f-b2f2-210020f9dd92 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80378) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 10:03:36.273893 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 514/1045] 10.222.0.82 () {58 vars in 1023 bytes} [Tue Aug 2 10:03:36 2022] POST /identity/v3/auth/tokens => generated 1969 bytes in 149 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 02 10:03:36.295797 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a016bdd9-ff7b-438f-b2f2-210020f9dd92 None None] Authenticating user token {{(pid=80378) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:03:36.298141 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f65d4e4e-8cf2-4081-b85b-b48ec654fcf6 None admin] Validating token access rules against request {{(pid=80377) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:03:36.355895 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6489d83-9e3f-45f1-8990-327ad2a6b64e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '891005fa5ff74041b75747d16cc66798', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80377) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:03:36.356478 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6489d83-9e3f-45f1-8990-327ad2a6b64e None admin] REQUEST_METHOD: `GET` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:03:36.356761 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6489d83-9e3f-45f1-8990-327ad2a6b64e None admin] SCRIPT_NAME: `/identity` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:03:36.357038 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6489d83-9e3f-45f1-8990-327ad2a6b64e None admin] PATH_INFO: `/v3/endpoints` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:03:36.357809 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6489d83-9e3f-45f1-8990-327ad2a6b64e None admin] RBAC: Adding query filter params (interface=public, service_id=cbb3bc07a76342b2894a08da05090269, region_id=RegionOne) {{(pid=80377) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 02 10:03:36.358833 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6489d83-9e3f-45f1-8990-327ad2a6b64e None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:03:36.359598 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6489d83-9e3f-45f1-8990-327ad2a6b64e None admin] RBAC: Authorization granted {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:03:36.363464 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG dbcounter [-] [80377] Writing DB stats keystone:SELECT=100 {{(pid=80377) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 10:03:36.368264 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: [pid: 80377|app: 0|req: 532/1046] 10.222.0.82 () {56 vars in 1251 bytes} [Tue Aug 2 10:03:36 2022] GET /identity/v3/endpoints?service_id=cbb3bc07a76342b2894a08da05090269&interface=public®ion_id=RegionOne => generated 194 bytes in 224 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 02 10:03:36.373436 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6489d83-9e3f-45f1-8990-327ad2a6b64e None admin] Authenticating user token {{(pid=80377) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:03:36.391215 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG dbcounter [-] [80378] Writing DB stats keystone:SELECT=100 {{(pid=80378) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 10:03:36.461419 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a016bdd9-ff7b-438f-b2f2-210020f9dd92 None None] Validating token access rules against request {{(pid=80378) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:03:36.510973 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ae018c0-ac1f-4ea5-a2a8-c26f5014a62b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '891005fa5ff74041b75747d16cc66798', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80378) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:03:36.512476 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ae018c0-ac1f-4ea5-a2a8-c26f5014a62b None admin] REQUEST_METHOD: `GET` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:03:36.512476 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ae018c0-ac1f-4ea5-a2a8-c26f5014a62b None admin] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:03:36.512741 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ae018c0-ac1f-4ea5-a2a8-c26f5014a62b None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:03:36.529074 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ae018c0-ac1f-4ea5-a2a8-c26f5014a62b None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 02 10:03:36.534081 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6489d83-9e3f-45f1-8990-327ad2a6b64e None admin] Validating token access rules against request {{(pid=80377) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:03:36.535804 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application [None req-5ae018c0-ac1f-4ea5-a2a8-c26f5014a62b None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 02 10:03:36.535804 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 02 10:03:36.535804 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 02 10:03:36.535804 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 02 10:03:36.535804 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 02 10:03:36.535804 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 02 10:03:36.535804 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 02 10:03:36.535804 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 02 10:03:36.535804 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 02 10:03:36.535804 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 02 10:03:36.535804 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 02 10:03:36.535804 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 02 10:03:36.535804 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 02 10:03:36.535804 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 02 10:03:36.535804 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 02 10:03:36.535804 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 02 10:03:36.536731 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 02 10:03:36.536731 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 02 10:03:36.536731 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 02 10:03:36.536731 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 02 10:03:36.536731 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 02 10:03:36.536731 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 02 10:03:36.536731 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 02 10:03:36.536731 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 02 10:03:36.536731 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 02 10:03:36.536731 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application return self.get_or_create( Aug 02 10:03:36.536731 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 02 10:03:36.536731 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application with Lock( Aug 02 10:03:36.536731 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 02 10:03:36.536731 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application return self._enter() Aug 02 10:03:36.536731 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 02 10:03:36.536731 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 02 10:03:36.537594 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 02 10:03:36.537594 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application return self.creator() Aug 02 10:03:36.537594 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 02 10:03:36.537594 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application created_value = creator( Aug 02 10:03:36.537594 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 02 10:03:36.537594 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 02 10:03:36.537594 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 02 10:03:36.537594 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 02 10:03:36.537594 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 02 10:03:36.537594 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 02 10:03:36.537594 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 02 10:03:36.537594 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application Aug 02 10:03:36.537594 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 515/1047] 10.222.0.82 () {56 vars in 1096 bytes} [Tue Aug 2 10:03:36 2022] GET /identity/v3/roles/admin => generated 83 bytes in 242 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 02 10:03:36.543348 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ae018c0-ac1f-4ea5-a2a8-c26f5014a62b None admin] Authenticating user token {{(pid=80378) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:03:36.594596 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ce6b61c-daa5-4234-869c-171cf7af4cf7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '891005fa5ff74041b75747d16cc66798', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80377) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:03:36.595373 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ce6b61c-daa5-4234-869c-171cf7af4cf7 None admin] REQUEST_METHOD: `GET` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:03:36.596318 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ce6b61c-daa5-4234-869c-171cf7af4cf7 None admin] SCRIPT_NAME: `/identity` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:03:36.596657 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ce6b61c-daa5-4234-869c-171cf7af4cf7 None admin] PATH_INFO: `/v3/services` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:03:36.597406 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ce6b61c-daa5-4234-869c-171cf7af4cf7 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:03:36.599001 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ce6b61c-daa5-4234-869c-171cf7af4cf7 None admin] RBAC: Authorization granted {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:03:36.623197 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: [pid: 80377|app: 0|req: 533/1048] 10.222.0.82 () {56 vars in 1087 bytes} [Tue Aug 2 10:03:36 2022] GET /identity/v3/services => generated 1957 bytes in 249 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 02 10:03:36.716433 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ae018c0-ac1f-4ea5-a2a8-c26f5014a62b None admin] Validating token access rules against request {{(pid=80378) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:03:36.741860 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG dbcounter [-] [80378] Writing DB stats keystone:SELECT=100 {{(pid=80378) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 10:03:36.770768 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12d86f93-91b5-4eb1-a882-b4080ed7d140 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '891005fa5ff74041b75747d16cc66798', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80378) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:03:36.771465 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12d86f93-91b5-4eb1-a882-b4080ed7d140 None admin] REQUEST_METHOD: `GET` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:03:36.771745 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12d86f93-91b5-4eb1-a882-b4080ed7d140 None admin] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:03:36.772014 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12d86f93-91b5-4eb1-a882-b4080ed7d140 None admin] PATH_INFO: `/v3/roles` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:03:36.772749 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12d86f93-91b5-4eb1-a882-b4080ed7d140 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80378) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 02 10:03:36.773073 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12d86f93-91b5-4eb1-a882-b4080ed7d140 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:03:36.774715 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12d86f93-91b5-4eb1-a882-b4080ed7d140 None admin] RBAC: Authorization granted {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:03:36.786193 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 516/1049] 10.222.0.82 () {56 vars in 1099 bytes} [Tue Aug 2 10:03:36 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 243 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 02 10:03:36.791599 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ce6b61c-daa5-4234-869c-171cf7af4cf7 None admin] Authenticating user token {{(pid=80377) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:03:36.901766 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG dbcounter [-] [80377] Writing DB stats keystone:SELECT=100 {{(pid=80377) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 10:03:36.966280 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ce6b61c-daa5-4234-869c-171cf7af4cf7 None admin] Validating token access rules against request {{(pid=80377) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:03:37.015344 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb8948c4-8f7b-4f48-a3ba-8e352319e294 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '891005fa5ff74041b75747d16cc66798', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80377) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:03:37.016013 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb8948c4-8f7b-4f48-a3ba-8e352319e294 None admin] REQUEST_METHOD: `GET` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:03:37.016371 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb8948c4-8f7b-4f48-a3ba-8e352319e294 None admin] SCRIPT_NAME: `/identity` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:03:37.016652 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb8948c4-8f7b-4f48-a3ba-8e352319e294 None admin] PATH_INFO: `/v3/users/b0299b32194741659e74f6d38f78aca7` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:03:37.040843 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb8948c4-8f7b-4f48-a3ba-8e352319e294 None admin] RBAC: Authorizing `identity:get_user(user_id=b0299b32194741659e74f6d38f78aca7)` {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:03:37.042550 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb8948c4-8f7b-4f48-a3ba-8e352319e294 None admin] RBAC: Authorization granted {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:03:37.044359 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: [pid: 80377|app: 0|req: 534/1050] 10.222.0.82 () {56 vars in 1177 bytes} [Tue Aug 2 10:03:36 2022] GET /identity/v3/users/b0299b32194741659e74f6d38f78aca7 => generated 285 bytes in 253 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 02 10:03:37.052911 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12d86f93-91b5-4eb1-a882-b4080ed7d140 None admin] Authenticating user token {{(pid=80378) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:03:37.211707 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12d86f93-91b5-4eb1-a882-b4080ed7d140 None admin] Validating token access rules against request {{(pid=80378) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:03:37.255094 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa6f2e7a-4c9e-4e97-8dbe-4044023a95f5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '891005fa5ff74041b75747d16cc66798', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80378) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:03:37.255730 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa6f2e7a-4c9e-4e97-8dbe-4044023a95f5 None admin] REQUEST_METHOD: `GET` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:03:37.256063 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa6f2e7a-4c9e-4e97-8dbe-4044023a95f5 None admin] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:03:37.256355 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa6f2e7a-4c9e-4e97-8dbe-4044023a95f5 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:03:37.322402 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG dbcounter [-] [80378] Writing DB stats keystone:SELECT=100 {{(pid=80378) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 10:03:37.354787 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa6f2e7a-4c9e-4e97-8dbe-4044023a95f5 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:03:37.356320 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa6f2e7a-4c9e-4e97-8dbe-4044023a95f5 None admin] RBAC: Authorization granted {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:03:37.541822 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 517/1051] 10.222.0.82 () {58 vars in 1282 bytes} [Tue Aug 2 10:03:37 2022] GET /identity/v3/auth/tokens => generated 1969 bytes in 485 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Aug 02 10:03:37.545720 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb8948c4-8f7b-4f48-a3ba-8e352319e294 None admin] Authenticating user token {{(pid=80377) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:03:37.691494 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb8948c4-8f7b-4f48-a3ba-8e352319e294 None admin] Validating token access rules against request {{(pid=80377) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:03:37.735906 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG dbcounter [-] [80377] Writing DB stats keystone:SELECT=100 {{(pid=80377) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 10:03:37.757150 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f240e53-3361-4e23-b261-10d0ab1b7942 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '891005fa5ff74041b75747d16cc66798', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80377) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:03:37.758308 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f240e53-3361-4e23-b261-10d0ab1b7942 None admin] REQUEST_METHOD: `GET` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:03:37.758851 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f240e53-3361-4e23-b261-10d0ab1b7942 None admin] SCRIPT_NAME: `/identity` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:03:37.759605 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f240e53-3361-4e23-b261-10d0ab1b7942 None admin] PATH_INFO: `/v3/projects/edf242cc36af4db9b5d76d897711bb6c` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:03:37.761589 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f240e53-3361-4e23-b261-10d0ab1b7942 None admin] RBAC: Authorizing `identity:get_project(project_id=edf242cc36af4db9b5d76d897711bb6c)` {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:03:37.763366 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f240e53-3361-4e23-b261-10d0ab1b7942 None admin] RBAC: Authorization granted {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:03:37.765580 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: [pid: 80377|app: 0|req: 535/1052] 10.222.0.82 () {56 vars in 1186 bytes} [Tue Aug 2 10:03:37 2022] GET /identity/v3/projects/edf242cc36af4db9b5d76d897711bb6c => generated 311 bytes in 221 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 02 10:03:37.775308 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa6f2e7a-4c9e-4e97-8dbe-4044023a95f5 None admin] Authenticating user token {{(pid=80378) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:03:37.943108 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG dbcounter [-] [80378] Writing DB stats keystone:SELECT=100 {{(pid=80378) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 10:03:37.988508 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa6f2e7a-4c9e-4e97-8dbe-4044023a95f5 None admin] Validating token access rules against request {{(pid=80378) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:03:38.051559 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e39d86c-d0f4-4b30-b766-4d26dac72fa7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '891005fa5ff74041b75747d16cc66798', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80378) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:03:38.052472 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e39d86c-d0f4-4b30-b766-4d26dac72fa7 None admin] REQUEST_METHOD: `GET` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:03:38.052822 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e39d86c-d0f4-4b30-b766-4d26dac72fa7 None admin] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:03:38.053306 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e39d86c-d0f4-4b30-b766-4d26dac72fa7 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:03:38.054445 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e39d86c-d0f4-4b30-b766-4d26dac72fa7 None admin] RBAC: Adding query filter params (role.id=237966442f05411191105bf336bc14f3, scope.project.id=edf242cc36af4db9b5d76d897711bb6c, user.id=b0299b32194741659e74f6d38f78aca7) {{(pid=80378) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 02 10:03:38.054875 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e39d86c-d0f4-4b30-b766-4d26dac72fa7 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:03:38.057088 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e39d86c-d0f4-4b30-b766-4d26dac72fa7 None admin] RBAC: Authorization granted {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:03:38.066668 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 518/1053] 10.222.0.82 () {56 vars in 1374 bytes} [Tue Aug 2 10:03:37 2022] GET /identity/v3/role_assignments?user.id=b0299b32194741659e74f6d38f78aca7&scope.project.id=edf242cc36af4db9b5d76d897711bb6c&role.id=237966442f05411191105bf336bc14f3 => generated 259 bytes in 292 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 02 10:03:38.684284 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25b970bd-33a1-40a5-acdc-fc6b1f023c4b None None] REQUEST_METHOD: `GET` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:03:38.685058 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25b970bd-33a1-40a5-acdc-fc6b1f023c4b None None] SCRIPT_NAME: `/identity` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:03:38.685209 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25b970bd-33a1-40a5-acdc-fc6b1f023c4b None None] PATH_INFO: `/` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:03:38.687338 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: [pid: 80377|app: 0|req: 536/1054] 10.222.0.82 () {54 vars in 926 bytes} [Tue Aug 2 10:03:38 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 02 10:03:38.696898 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a4e6dc2-76d0-4bd3-91ac-30c94ded4d94 None None] REQUEST_METHOD: `POST` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:03:38.699565 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a4e6dc2-76d0-4bd3-91ac-30c94ded4d94 None None] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:03:38.699565 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a4e6dc2-76d0-4bd3-91ac-30c94ded4d94 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:03:38.725565 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.auth.core [None req-3a4e6dc2-76d0-4bd3-91ac-30c94ded4d94 None None] MFA Rules not processed for user `891005fa5ff74041b75747d16cc66798`. Rule list: `[]` (Enabled: `True`). {{(pid=80378) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 02 10:03:38.727509 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.fernet_utils [None req-3a4e6dc2-76d0-4bd3-91ac-30c94ded4d94 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80378) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 10:03:38.824035 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 519/1055] 10.222.0.82 () {58 vars in 1023 bytes} [Tue Aug 2 10:03:38 2022] POST /identity/v3/auth/tokens => generated 1969 bytes in 129 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Aug 02 10:03:38.849198 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25b970bd-33a1-40a5-acdc-fc6b1f023c4b None None] Authenticating user token {{(pid=80377) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:03:38.993210 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b513be7-39d6-4678-9db6-5e81533cd2ed None None] REQUEST_METHOD: `GET` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:03:38.993547 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b513be7-39d6-4678-9db6-5e81533cd2ed None None] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:03:38.993921 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b513be7-39d6-4678-9db6-5e81533cd2ed None None] PATH_INFO: `/` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:03:38.996752 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 520/1056] 10.222.0.82 () {54 vars in 926 bytes} [Tue Aug 2 10:03:38 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 02 10:03:39.000992 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25b970bd-33a1-40a5-acdc-fc6b1f023c4b None None] Validating token access rules against request {{(pid=80377) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:03:39.003660 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf75ee58-e928-4e26-acb4-27336aeafaa4 None None] REQUEST_METHOD: `POST` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:03:39.003947 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf75ee58-e928-4e26-acb4-27336aeafaa4 None None] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:03:39.004222 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf75ee58-e928-4e26-acb4-27336aeafaa4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:03:39.036001 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.auth.core [None req-bf75ee58-e928-4e26-acb4-27336aeafaa4 None None] MFA Rules not processed for user `891005fa5ff74041b75747d16cc66798`. Rule list: `[]` (Enabled: `True`). {{(pid=80378) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 02 10:03:39.037335 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.fernet_utils [None req-bf75ee58-e928-4e26-acb4-27336aeafaa4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80378) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 10:03:39.061204 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c30f3296-30a2-48bf-b60c-5a8d615dde1a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '891005fa5ff74041b75747d16cc66798', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80377) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:03:39.062328 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c30f3296-30a2-48bf-b60c-5a8d615dde1a None admin] REQUEST_METHOD: `GET` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:03:39.062709 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c30f3296-30a2-48bf-b60c-5a8d615dde1a None admin] SCRIPT_NAME: `/identity` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:03:39.064522 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c30f3296-30a2-48bf-b60c-5a8d615dde1a None admin] PATH_INFO: `/v3/roles/ResellerAdmin` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:03:39.087500 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c30f3296-30a2-48bf-b60c-5a8d615dde1a None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 02 10:03:39.093659 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application [None req-c30f3296-30a2-48bf-b60c-5a8d615dde1a None admin] Could not find role: ResellerAdmin.: keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Aug 02 10:03:39.093659 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 02 10:03:39.093659 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 02 10:03:39.093659 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 02 10:03:39.093659 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 02 10:03:39.093659 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 02 10:03:39.093659 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 02 10:03:39.093659 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 02 10:03:39.093659 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 02 10:03:39.093659 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 02 10:03:39.093659 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 02 10:03:39.093659 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 02 10:03:39.093659 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 02 10:03:39.093659 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 02 10:03:39.093659 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 02 10:03:39.093659 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 02 10:03:39.094680 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 02 10:03:39.094680 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 02 10:03:39.094680 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 02 10:03:39.094680 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 02 10:03:39.094680 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 02 10:03:39.094680 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 02 10:03:39.094680 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 02 10:03:39.094680 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 02 10:03:39.094680 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 02 10:03:39.094680 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application return self.get_or_create( Aug 02 10:03:39.094680 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 02 10:03:39.094680 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application with Lock( Aug 02 10:03:39.094680 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 02 10:03:39.094680 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application return self._enter() Aug 02 10:03:39.094680 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 02 10:03:39.094680 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 02 10:03:39.095545 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 02 10:03:39.095545 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application return self.creator() Aug 02 10:03:39.095545 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 02 10:03:39.095545 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application created_value = creator( Aug 02 10:03:39.095545 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 02 10:03:39.095545 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 02 10:03:39.095545 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 02 10:03:39.095545 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 02 10:03:39.095545 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 02 10:03:39.095545 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 02 10:03:39.095545 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Aug 02 10:03:39.095545 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application Aug 02 10:03:39.102704 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: [pid: 80377|app: 0|req: 537/1057] 10.222.0.82 () {56 vars in 1120 bytes} [Tue Aug 2 10:03:38 2022] GET /identity/v3/roles/ResellerAdmin => generated 91 bytes in 254 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 02 10:03:39.103991 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c30f3296-30a2-48bf-b60c-5a8d615dde1a None admin] Authenticating user token {{(pid=80377) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:03:39.147275 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 521/1058] 10.222.0.82 () {58 vars in 1023 bytes} [Tue Aug 2 10:03:39 2022] POST /identity/v3/auth/tokens => generated 1969 bytes in 146 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 02 10:03:39.170010 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf75ee58-e928-4e26-acb4-27336aeafaa4 None None] Authenticating user token {{(pid=80378) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:03:39.173095 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG dbcounter [-] [80378] Writing DB stats keystone:SELECT=100 {{(pid=80378) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 10:03:39.199033 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG dbcounter [-] [80377] Writing DB stats keystone:SELECT=100 {{(pid=80377) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 10:03:39.269951 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c30f3296-30a2-48bf-b60c-5a8d615dde1a None admin] Validating token access rules against request {{(pid=80377) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:03:39.323515 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf75ee58-e928-4e26-acb4-27336aeafaa4 None None] Validating token access rules against request {{(pid=80378) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:03:39.324572 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9767c56b-ecbc-47c7-9b16-6ddf4b31c0c5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '891005fa5ff74041b75747d16cc66798', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80377) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:03:39.325355 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9767c56b-ecbc-47c7-9b16-6ddf4b31c0c5 None admin] REQUEST_METHOD: `GET` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:03:39.325899 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9767c56b-ecbc-47c7-9b16-6ddf4b31c0c5 None admin] SCRIPT_NAME: `/identity` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:03:39.326316 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9767c56b-ecbc-47c7-9b16-6ddf4b31c0c5 None admin] PATH_INFO: `/v3/roles` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:03:39.327830 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9767c56b-ecbc-47c7-9b16-6ddf4b31c0c5 None admin] RBAC: Adding query filter params (name=ResellerAdmin) {{(pid=80377) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 02 10:03:39.328128 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9767c56b-ecbc-47c7-9b16-6ddf4b31c0c5 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:03:39.329941 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9767c56b-ecbc-47c7-9b16-6ddf4b31c0c5 None admin] RBAC: Authorization granted {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:03:39.341005 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: [pid: 80377|app: 0|req: 538/1059] 10.222.0.82 () {56 vars in 1115 bytes} [Tue Aug 2 10:03:39 2022] GET /identity/v3/roles?name=ResellerAdmin => generated 338 bytes in 238 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 02 10:03:39.347917 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9767c56b-ecbc-47c7-9b16-6ddf4b31c0c5 None admin] Authenticating user token {{(pid=80377) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:03:39.376811 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2302de5-4dba-40b2-8460-0ecd654352bd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '891005fa5ff74041b75747d16cc66798', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80378) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:03:39.377461 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2302de5-4dba-40b2-8460-0ecd654352bd None admin] REQUEST_METHOD: `GET` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:03:39.377821 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2302de5-4dba-40b2-8460-0ecd654352bd None admin] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:03:39.378097 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2302de5-4dba-40b2-8460-0ecd654352bd None admin] PATH_INFO: `/v3/services/image` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:03:39.379073 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2302de5-4dba-40b2-8460-0ecd654352bd None admin] RBAC: Authorizing `identity:get_service(service_id=image)` {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:03:39.380867 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2302de5-4dba-40b2-8460-0ecd654352bd None admin] RBAC: Authorization granted {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:03:39.393115 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application [None req-b2302de5-4dba-40b2-8460-0ecd654352bd None admin] Could not find service: image.: keystone.exception.ServiceNotFound: Could not find service: image. Aug 02 10:03:39.393115 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 02 10:03:39.393115 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Aug 02 10:03:39.393115 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Aug 02 10:03:39.393115 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Aug 02 10:03:39.393115 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Aug 02 10:03:39.393115 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Aug 02 10:03:39.393115 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 02 10:03:39.393115 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: image. Aug 02 10:03:39.393115 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application Aug 02 10:03:39.393115 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 02 10:03:39.393115 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application Aug 02 10:03:39.393115 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 02 10:03:39.393115 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 02 10:03:39.393115 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 02 10:03:39.393115 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 02 10:03:39.393115 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 02 10:03:39.398712 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 02 10:03:39.398712 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 02 10:03:39.398712 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 02 10:03:39.398712 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 02 10:03:39.398712 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 02 10:03:39.398712 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 02 10:03:39.398712 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Aug 02 10:03:39.398712 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application return self._get_service(service_id) Aug 02 10:03:39.398712 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Aug 02 10:03:39.398712 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Aug 02 10:03:39.398712 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 02 10:03:39.398712 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 02 10:03:39.398712 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 02 10:03:39.398712 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 02 10:03:39.398712 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 02 10:03:39.398712 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application return self.get_or_create( Aug 02 10:03:39.401110 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 02 10:03:39.401110 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application with Lock( Aug 02 10:03:39.401110 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 02 10:03:39.401110 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application return self._enter() Aug 02 10:03:39.401110 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 02 10:03:39.401110 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 02 10:03:39.401110 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 02 10:03:39.401110 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application return self.creator() Aug 02 10:03:39.401110 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 02 10:03:39.401110 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application created_value = creator( Aug 02 10:03:39.401110 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Aug 02 10:03:39.401110 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 02 10:03:39.401110 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: image. Aug 02 10:03:39.401110 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application Aug 02 10:03:39.401110 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 522/1060] 10.222.0.82 () {56 vars in 1105 bytes} [Tue Aug 2 10:03:39 2022] GET /identity/v3/services/image => generated 86 bytes in 229 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 02 10:03:39.404076 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2302de5-4dba-40b2-8460-0ecd654352bd None admin] Authenticating user token {{(pid=80378) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:03:39.484606 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9767c56b-ecbc-47c7-9b16-6ddf4b31c0c5 None admin] Validating token access rules against request {{(pid=80377) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:03:39.490059 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG dbcounter [-] [80378] Writing DB stats keystone:SELECT=100 {{(pid=80378) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 10:03:39.504196 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG dbcounter [-] [80377] Writing DB stats keystone:SELECT=100 {{(pid=80377) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 10:03:39.534284 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33eff540-97a3-4285-8450-fcca7f2e0b65 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '891005fa5ff74041b75747d16cc66798', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80377) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:03:39.534922 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33eff540-97a3-4285-8450-fcca7f2e0b65 None admin] REQUEST_METHOD: `GET` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:03:39.535142 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33eff540-97a3-4285-8450-fcca7f2e0b65 None admin] SCRIPT_NAME: `/identity` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:03:39.535421 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33eff540-97a3-4285-8450-fcca7f2e0b65 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:03:39.537503 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2302de5-4dba-40b2-8460-0ecd654352bd None admin] Validating token access rules against request {{(pid=80378) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:03:39.550061 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33eff540-97a3-4285-8450-fcca7f2e0b65 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:03:39.551539 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33eff540-97a3-4285-8450-fcca7f2e0b65 None admin] RBAC: Authorization granted {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:03:39.571759 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application [None req-33eff540-97a3-4285-8450-fcca7f2e0b65 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 02 10:03:39.571759 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 02 10:03:39.571759 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 02 10:03:39.571759 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 02 10:03:39.571759 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 02 10:03:39.571759 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 02 10:03:39.571759 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 02 10:03:39.571759 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 02 10:03:39.571759 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 02 10:03:39.571759 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 02 10:03:39.571759 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 02 10:03:39.571759 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 02 10:03:39.571759 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 02 10:03:39.571759 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 02 10:03:39.571759 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 02 10:03:39.573587 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 02 10:03:39.573587 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 02 10:03:39.573587 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 02 10:03:39.573587 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 02 10:03:39.573587 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 02 10:03:39.573587 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 02 10:03:39.573587 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application return self.get_or_create( Aug 02 10:03:39.573587 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 02 10:03:39.573587 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application with Lock( Aug 02 10:03:39.573587 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 02 10:03:39.573587 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application return self._enter() Aug 02 10:03:39.573587 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 02 10:03:39.573587 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 02 10:03:39.573587 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 02 10:03:39.573587 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application return self.creator() Aug 02 10:03:39.573587 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 02 10:03:39.573587 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application created_value = creator( Aug 02 10:03:39.576066 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 02 10:03:39.576066 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 02 10:03:39.576066 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 02 10:03:39.576066 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application Aug 02 10:03:39.576066 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: [pid: 80377|app: 0|req: 539/1061] 10.222.0.82 () {56 vars in 1108 bytes} [Tue Aug 2 10:03:39 2022] GET /identity/v3/domains/Default => generated 87 bytes in 226 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 02 10:03:39.581672 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33eff540-97a3-4285-8450-fcca7f2e0b65 None admin] Authenticating user token {{(pid=80377) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:03:39.593460 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbc8709e-2e3e-4a76-a364-a9ac95c3dc1c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '891005fa5ff74041b75747d16cc66798', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80378) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:03:39.594430 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbc8709e-2e3e-4a76-a364-a9ac95c3dc1c None admin] REQUEST_METHOD: `GET` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:03:39.594947 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbc8709e-2e3e-4a76-a364-a9ac95c3dc1c None admin] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:03:39.595405 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbc8709e-2e3e-4a76-a364-a9ac95c3dc1c None admin] PATH_INFO: `/v3/services` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:03:39.597145 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dbc8709e-2e3e-4a76-a364-a9ac95c3dc1c None admin] RBAC: Adding query filter params (name=image) {{(pid=80378) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 02 10:03:39.597711 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dbc8709e-2e3e-4a76-a364-a9ac95c3dc1c None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:03:39.600191 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dbc8709e-2e3e-4a76-a364-a9ac95c3dc1c None admin] RBAC: Authorization granted {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:03:39.613827 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 523/1062] 10.222.0.82 () {56 vars in 1108 bytes} [Tue Aug 2 10:03:39 2022] GET /identity/v3/services?name=image => generated 122 bytes in 205 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 02 10:03:39.619665 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbc8709e-2e3e-4a76-a364-a9ac95c3dc1c None admin] Authenticating user token {{(pid=80378) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:03:39.751291 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33eff540-97a3-4285-8450-fcca7f2e0b65 None admin] Validating token access rules against request {{(pid=80377) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:03:39.792407 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbc8709e-2e3e-4a76-a364-a9ac95c3dc1c None admin] Validating token access rules against request {{(pid=80378) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:03:39.809581 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9ccae86-5a79-4332-9e59-e3fe5a8766a5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '891005fa5ff74041b75747d16cc66798', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80377) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:03:39.810414 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9ccae86-5a79-4332-9e59-e3fe5a8766a5 None admin] REQUEST_METHOD: `GET` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:03:39.810703 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9ccae86-5a79-4332-9e59-e3fe5a8766a5 None admin] SCRIPT_NAME: `/identity` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:03:39.810980 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9ccae86-5a79-4332-9e59-e3fe5a8766a5 None admin] PATH_INFO: `/v3/domains` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:03:39.811696 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9ccae86-5a79-4332-9e59-e3fe5a8766a5 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80377) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 02 10:03:39.812001 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9ccae86-5a79-4332-9e59-e3fe5a8766a5 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:03:39.813433 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9ccae86-5a79-4332-9e59-e3fe5a8766a5 None admin] RBAC: Authorization granted {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:03:39.831825 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG dbcounter [-] [80378] Writing DB stats keystone:SELECT=100 {{(pid=80378) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 10:03:39.832076 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: [pid: 80377|app: 0|req: 540/1063] 10.222.0.82 () {56 vars in 1109 bytes} [Tue Aug 2 10:03:39 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 253 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 02 10:03:39.832919 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9ccae86-5a79-4332-9e59-e3fe5a8766a5 None admin] Authenticating user token {{(pid=80377) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:03:39.840128 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4368dd1-806e-42b4-bedf-024bf08c065c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '891005fa5ff74041b75747d16cc66798', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80378) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:03:39.840939 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4368dd1-806e-42b4-bedf-024bf08c065c None admin] REQUEST_METHOD: `GET` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:03:39.841344 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4368dd1-806e-42b4-bedf-024bf08c065c None admin] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:03:39.841744 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4368dd1-806e-42b4-bedf-024bf08c065c None admin] PATH_INFO: `/v3/services` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:03:39.843023 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4368dd1-806e-42b4-bedf-024bf08c065c None admin] RBAC: Adding query filter params (type=image) {{(pid=80378) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 02 10:03:39.843356 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4368dd1-806e-42b4-bedf-024bf08c065c None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:03:39.844982 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4368dd1-806e-42b4-bedf-024bf08c065c None admin] RBAC: Authorization granted {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:03:39.858383 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 524/1064] 10.222.0.82 () {56 vars in 1108 bytes} [Tue Aug 2 10:03:39 2022] GET /identity/v3/services?type=image => generated 350 bytes in 236 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 02 10:03:39.860410 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4368dd1-806e-42b4-bedf-024bf08c065c None admin] Authenticating user token {{(pid=80378) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:03:39.887892 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG dbcounter [-] [80377] Writing DB stats keystone:SELECT=100 {{(pid=80377) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 10:03:39.992601 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9ccae86-5a79-4332-9e59-e3fe5a8766a5 None admin] Validating token access rules against request {{(pid=80377) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:03:40.042753 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4368dd1-806e-42b4-bedf-024bf08c065c None admin] Validating token access rules against request {{(pid=80378) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:03:40.050050 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e25df9a-9996-419e-bd11-bffbc90564bb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '891005fa5ff74041b75747d16cc66798', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80377) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:03:40.051345 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e25df9a-9996-419e-bd11-bffbc90564bb None admin] REQUEST_METHOD: `GET` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:03:40.051846 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e25df9a-9996-419e-bd11-bffbc90564bb None admin] SCRIPT_NAME: `/identity` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:03:40.052358 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e25df9a-9996-419e-bd11-bffbc90564bb None admin] PATH_INFO: `/v3/users/nova` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:03:40.064191 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e25df9a-9996-419e-bd11-bffbc90564bb None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:03:40.068016 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e25df9a-9996-419e-bd11-bffbc90564bb None admin] RBAC: Authorization granted {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:03:40.082039 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application [None req-1e25df9a-9996-419e-bd11-bffbc90564bb None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Aug 02 10:03:40.082039 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 02 10:03:40.082039 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 02 10:03:40.082039 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 02 10:03:40.082039 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 02 10:03:40.082039 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 02 10:03:40.082039 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 02 10:03:40.082039 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 02 10:03:40.082039 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 02 10:03:40.082039 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 02 10:03:40.082039 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 02 10:03:40.082039 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 02 10:03:40.082039 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 02 10:03:40.082039 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 02 10:03:40.082039 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 02 10:03:40.082039 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 02 10:03:40.084102 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 02 10:03:40.084102 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 02 10:03:40.084102 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 02 10:03:40.084102 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 02 10:03:40.084102 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 02 10:03:40.084102 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 02 10:03:40.084102 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 02 10:03:40.084102 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 02 10:03:40.084102 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 02 10:03:40.084102 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application return self.get_or_create( Aug 02 10:03:40.084102 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 02 10:03:40.084102 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application with Lock( Aug 02 10:03:40.084102 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 02 10:03:40.084102 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application return self._enter() Aug 02 10:03:40.084102 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 02 10:03:40.084102 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 02 10:03:40.084102 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 02 10:03:40.084102 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application return self.creator() Aug 02 10:03:40.085282 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 02 10:03:40.085282 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application created_value = creator( Aug 02 10:03:40.085282 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 02 10:03:40.085282 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 02 10:03:40.085282 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 02 10:03:40.085282 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 02 10:03:40.085282 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 02 10:03:40.085282 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 02 10:03:40.085282 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Aug 02 10:03:40.085282 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application Aug 02 10:03:40.088201 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: [pid: 80377|app: 0|req: 541/1065] 10.222.0.82 () {56 vars in 1093 bytes} [Tue Aug 2 10:03:39 2022] GET /identity/v3/users/nova => generated 82 bytes in 255 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 02 10:03:40.089301 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e25df9a-9996-419e-bd11-bffbc90564bb None admin] Authenticating user token {{(pid=80377) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:03:40.100919 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-670f7b63-847c-4315-88f4-87c5836c49f3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '891005fa5ff74041b75747d16cc66798', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80378) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:03:40.101556 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-670f7b63-847c-4315-88f4-87c5836c49f3 None admin] REQUEST_METHOD: `POST` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:03:40.101862 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-670f7b63-847c-4315-88f4-87c5836c49f3 None admin] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:03:40.102132 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-670f7b63-847c-4315-88f4-87c5836c49f3 None admin] PATH_INFO: `/v3/endpoints` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:03:40.102853 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-670f7b63-847c-4315-88f4-87c5836c49f3 None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:03:40.104245 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-670f7b63-847c-4315-88f4-87c5836c49f3 None admin] RBAC: Authorization granted {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:03:40.137459 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 525/1066] 10.222.0.82 () {60 vars in 1144 bytes} [Tue Aug 2 10:03:39 2022] POST /identity/v3/endpoints => generated 327 bytes in 278 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 02 10:03:40.213730 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG dbcounter [-] [80377] Writing DB stats keystone:SELECT=100 {{(pid=80377) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 10:03:40.238785 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e25df9a-9996-419e-bd11-bffbc90564bb None admin] Validating token access rules against request {{(pid=80377) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:03:40.310772 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45eb2ea0-5e1d-4dd1-9066-fa5686a1c749 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '891005fa5ff74041b75747d16cc66798', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80377) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:03:40.311715 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45eb2ea0-5e1d-4dd1-9066-fa5686a1c749 None admin] REQUEST_METHOD: `GET` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:03:40.312175 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45eb2ea0-5e1d-4dd1-9066-fa5686a1c749 None admin] SCRIPT_NAME: `/identity` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:03:40.312656 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45eb2ea0-5e1d-4dd1-9066-fa5686a1c749 None admin] PATH_INFO: `/v3/users` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:03:40.314229 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45eb2ea0-5e1d-4dd1-9066-fa5686a1c749 None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=80377) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 02 10:03:40.314826 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45eb2ea0-5e1d-4dd1-9066-fa5686a1c749 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:03:40.317479 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45eb2ea0-5e1d-4dd1-9066-fa5686a1c749 None admin] RBAC: Authorization granted {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:03:40.347068 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: [pid: 80377|app: 0|req: 542/1067] 10.222.0.82 () {56 vars in 1133 bytes} [Tue Aug 2 10:03:40 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 258 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 02 10:03:40.353289 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-670f7b63-847c-4315-88f4-87c5836c49f3 None admin] Authenticating user token {{(pid=80378) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:03:40.438715 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG dbcounter [-] [80378] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80378) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 10:03:40.451591 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79c34571-687c-48d8-b9e6-fe078942e3a7 None None] REQUEST_METHOD: `GET` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:03:40.451974 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79c34571-687c-48d8-b9e6-fe078942e3a7 None None] SCRIPT_NAME: `/identity` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:03:40.452261 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79c34571-687c-48d8-b9e6-fe078942e3a7 None None] PATH_INFO: `/` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:03:40.453683 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: [pid: 80377|app: 0|req: 543/1068] 10.222.0.82 () {54 vars in 926 bytes} [Tue Aug 2 10:03:40 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 02 10:03:40.462987 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ea44f6c-54ca-4d61-8032-aaaea03daeb7 None None] REQUEST_METHOD: `POST` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:03:40.463383 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ea44f6c-54ca-4d61-8032-aaaea03daeb7 None None] SCRIPT_NAME: `/identity` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:03:40.464132 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ea44f6c-54ca-4d61-8032-aaaea03daeb7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:03:40.486991 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.auth.core [None req-6ea44f6c-54ca-4d61-8032-aaaea03daeb7 None None] MFA Rules not processed for user `891005fa5ff74041b75747d16cc66798`. Rule list: `[]` (Enabled: `True`). {{(pid=80377) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 02 10:03:40.488303 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.fernet_utils [None req-6ea44f6c-54ca-4d61-8032-aaaea03daeb7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80377) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 10:03:40.525424 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-670f7b63-847c-4315-88f4-87c5836c49f3 None admin] Validating token access rules against request {{(pid=80378) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:03:40.583312 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2de6758-6328-4a8e-9172-8fa1b4e23ffa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '891005fa5ff74041b75747d16cc66798', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80378) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:03:40.583869 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2de6758-6328-4a8e-9172-8fa1b4e23ffa None admin] REQUEST_METHOD: `GET` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:03:40.585253 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2de6758-6328-4a8e-9172-8fa1b4e23ffa None admin] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:03:40.585253 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2de6758-6328-4a8e-9172-8fa1b4e23ffa None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:03:40.604468 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2de6758-6328-4a8e-9172-8fa1b4e23ffa None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:03:40.606713 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2de6758-6328-4a8e-9172-8fa1b4e23ffa None admin] RBAC: Authorization granted {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:03:40.608609 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: [pid: 80377|app: 0|req: 544/1069] 10.222.0.82 () {58 vars in 1023 bytes} [Tue Aug 2 10:03:40 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 146 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Aug 02 10:03:40.631789 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application [None req-b2de6758-6328-4a8e-9172-8fa1b4e23ffa None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 02 10:03:40.631789 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 02 10:03:40.631789 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 02 10:03:40.631789 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 02 10:03:40.631789 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 02 10:03:40.631789 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 02 10:03:40.631789 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 02 10:03:40.631789 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 02 10:03:40.631789 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 02 10:03:40.631789 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 02 10:03:40.631789 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 02 10:03:40.631789 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 02 10:03:40.631789 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 02 10:03:40.631789 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 02 10:03:40.631789 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 02 10:03:40.632811 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 02 10:03:40.632811 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 02 10:03:40.632811 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 02 10:03:40.632811 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 02 10:03:40.632811 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 02 10:03:40.632811 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 02 10:03:40.632811 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application return self.get_or_create( Aug 02 10:03:40.632811 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 02 10:03:40.632811 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application with Lock( Aug 02 10:03:40.632811 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 02 10:03:40.632811 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application return self._enter() Aug 02 10:03:40.632811 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 02 10:03:40.632811 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 02 10:03:40.632811 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 02 10:03:40.632811 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application return self.creator() Aug 02 10:03:40.632811 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 02 10:03:40.632811 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application created_value = creator( Aug 02 10:03:40.634376 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 02 10:03:40.634376 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 02 10:03:40.634376 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 02 10:03:40.634376 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application Aug 02 10:03:40.634376 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 526/1070] 10.222.0.82 () {56 vars in 1108 bytes} [Tue Aug 2 10:03:40 2022] GET /identity/v3/domains/Default => generated 87 bytes in 281 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 02 10:03:40.637885 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ea44f6c-54ca-4d61-8032-aaaea03daeb7 None None] Authenticating user token {{(pid=80377) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:03:40.638696 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2de6758-6328-4a8e-9172-8fa1b4e23ffa None admin] Authenticating user token {{(pid=80378) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:03:40.732956 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG dbcounter [-] [80377] Writing DB stats keystone:SELECT=100 {{(pid=80377) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 10:03:40.780117 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ea44f6c-54ca-4d61-8032-aaaea03daeb7 None None] Validating token access rules against request {{(pid=80377) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:03:40.800953 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2de6758-6328-4a8e-9172-8fa1b4e23ffa None admin] Validating token access rules against request {{(pid=80378) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:03:40.806219 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG dbcounter [-] [80378] Writing DB stats keystone:SELECT=100 {{(pid=80378) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 10:03:40.836053 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54463183-a437-4fe9-957d-93a759a8fdc7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '891005fa5ff74041b75747d16cc66798', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80377) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:03:40.837469 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54463183-a437-4fe9-957d-93a759a8fdc7 None admin] REQUEST_METHOD: `GET` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:03:40.838272 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54463183-a437-4fe9-957d-93a759a8fdc7 None admin] SCRIPT_NAME: `/identity` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:03:40.838735 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54463183-a437-4fe9-957d-93a759a8fdc7 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:03:40.855360 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54463183-a437-4fe9-957d-93a759a8fdc7 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 02 10:03:40.857715 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e8507fb-f8e9-4455-a57e-0ef0154d8617 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '891005fa5ff74041b75747d16cc66798', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80378) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:03:40.858474 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e8507fb-f8e9-4455-a57e-0ef0154d8617 None admin] REQUEST_METHOD: `GET` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:03:40.858711 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e8507fb-f8e9-4455-a57e-0ef0154d8617 None admin] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:03:40.858981 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e8507fb-f8e9-4455-a57e-0ef0154d8617 None admin] PATH_INFO: `/v3/domains` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:03:40.859778 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e8507fb-f8e9-4455-a57e-0ef0154d8617 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80378) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 02 10:03:40.860086 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e8507fb-f8e9-4455-a57e-0ef0154d8617 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:03:40.861560 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application [None req-54463183-a437-4fe9-957d-93a759a8fdc7 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 02 10:03:40.861560 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 02 10:03:40.861560 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 02 10:03:40.861560 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 02 10:03:40.861560 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 02 10:03:40.861560 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 02 10:03:40.861560 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 02 10:03:40.861560 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 02 10:03:40.861560 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 02 10:03:40.861560 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 02 10:03:40.861560 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 02 10:03:40.861560 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 02 10:03:40.861560 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 02 10:03:40.861560 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 02 10:03:40.861560 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 02 10:03:40.861560 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 02 10:03:40.867006 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 02 10:03:40.867006 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 02 10:03:40.867006 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 02 10:03:40.867006 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 02 10:03:40.867006 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 02 10:03:40.867006 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 02 10:03:40.867006 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 02 10:03:40.867006 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 02 10:03:40.867006 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 02 10:03:40.867006 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application return self.get_or_create( Aug 02 10:03:40.867006 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 02 10:03:40.867006 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application with Lock( Aug 02 10:03:40.867006 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 02 10:03:40.867006 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application return self._enter() Aug 02 10:03:40.867006 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 02 10:03:40.867006 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 02 10:03:40.868026 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 02 10:03:40.868026 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application return self.creator() Aug 02 10:03:40.868026 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 02 10:03:40.868026 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application created_value = creator( Aug 02 10:03:40.868026 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 02 10:03:40.868026 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 02 10:03:40.868026 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 02 10:03:40.868026 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 02 10:03:40.868026 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 02 10:03:40.868026 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 02 10:03:40.868026 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 02 10:03:40.868026 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application Aug 02 10:03:40.868826 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e8507fb-f8e9-4455-a57e-0ef0154d8617 None admin] RBAC: Authorization granted {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:03:40.868925 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: [pid: 80377|app: 0|req: 545/1071] 10.222.0.82 () {56 vars in 1096 bytes} [Tue Aug 2 10:03:40 2022] GET /identity/v3/roles/admin => generated 83 bytes in 226 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 02 10:03:40.874449 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54463183-a437-4fe9-957d-93a759a8fdc7 None admin] Authenticating user token {{(pid=80377) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:03:40.882431 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 527/1072] 10.222.0.82 () {56 vars in 1109 bytes} [Tue Aug 2 10:03:40 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 244 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 02 10:03:40.888376 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e8507fb-f8e9-4455-a57e-0ef0154d8617 None admin] Authenticating user token {{(pid=80378) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:03:41.031938 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54463183-a437-4fe9-957d-93a759a8fdc7 None admin] Validating token access rules against request {{(pid=80377) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:03:41.046422 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e8507fb-f8e9-4455-a57e-0ef0154d8617 None admin] Validating token access rules against request {{(pid=80378) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:03:41.073410 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG dbcounter [-] [80377] Writing DB stats keystone:SELECT=100 {{(pid=80377) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 10:03:41.087149 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b4fa70e-af15-4277-af42-514534f4aae1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '891005fa5ff74041b75747d16cc66798', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80377) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:03:41.088014 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b4fa70e-af15-4277-af42-514534f4aae1 None admin] REQUEST_METHOD: `GET` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:03:41.088362 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b4fa70e-af15-4277-af42-514534f4aae1 None admin] SCRIPT_NAME: `/identity` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:03:41.088640 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b4fa70e-af15-4277-af42-514534f4aae1 None admin] PATH_INFO: `/v3/roles` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:03:41.089377 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b4fa70e-af15-4277-af42-514534f4aae1 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80377) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 02 10:03:41.089689 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b4fa70e-af15-4277-af42-514534f4aae1 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:03:41.091140 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b4fa70e-af15-4277-af42-514534f4aae1 None admin] RBAC: Authorization granted {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:03:41.102099 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: [pid: 80377|app: 0|req: 546/1073] 10.222.0.82 () {56 vars in 1099 bytes} [Tue Aug 2 10:03:40 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 228 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 02 10:03:41.107957 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6a482a2-a772-4ed2-9ddf-f568f0996ef1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '891005fa5ff74041b75747d16cc66798', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80378) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:03:41.108621 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b4fa70e-af15-4277-af42-514534f4aae1 None admin] Authenticating user token {{(pid=80377) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:03:41.108733 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6a482a2-a772-4ed2-9ddf-f568f0996ef1 None admin] REQUEST_METHOD: `GET` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:03:41.109113 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6a482a2-a772-4ed2-9ddf-f568f0996ef1 None admin] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:03:41.109334 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6a482a2-a772-4ed2-9ddf-f568f0996ef1 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:03:41.117850 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6a482a2-a772-4ed2-9ddf-f568f0996ef1 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:03:41.119500 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6a482a2-a772-4ed2-9ddf-f568f0996ef1 None admin] RBAC: Authorization granted {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:03:41.134336 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application [None req-c6a482a2-a772-4ed2-9ddf-f568f0996ef1 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 02 10:03:41.134336 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 02 10:03:41.134336 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 02 10:03:41.134336 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 02 10:03:41.134336 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 02 10:03:41.134336 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 02 10:03:41.134336 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 02 10:03:41.134336 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 02 10:03:41.134336 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 02 10:03:41.134336 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 02 10:03:41.134336 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 02 10:03:41.134336 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 02 10:03:41.134336 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 02 10:03:41.134336 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 02 10:03:41.134336 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 02 10:03:41.136202 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 02 10:03:41.136202 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 02 10:03:41.136202 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 02 10:03:41.136202 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 02 10:03:41.136202 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 02 10:03:41.136202 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 02 10:03:41.136202 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application return self.get_or_create( Aug 02 10:03:41.136202 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 02 10:03:41.136202 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application with Lock( Aug 02 10:03:41.136202 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 02 10:03:41.136202 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application return self._enter() Aug 02 10:03:41.136202 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 02 10:03:41.136202 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 02 10:03:41.136202 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 02 10:03:41.136202 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application return self.creator() Aug 02 10:03:41.136202 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 02 10:03:41.136202 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application created_value = creator( Aug 02 10:03:41.138006 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 02 10:03:41.138006 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 02 10:03:41.138006 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 02 10:03:41.138006 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 02 10:03:41.138006 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 02 10:03:41.138006 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 02 10:03:41.138006 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 02 10:03:41.138006 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application Aug 02 10:03:41.138006 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 528/1074] 10.222.0.82 () {56 vars in 1111 bytes} [Tue Aug 2 10:03:40 2022] GET /identity/v3/projects/service => generated 88 bytes in 248 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 02 10:03:41.141330 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6a482a2-a772-4ed2-9ddf-f568f0996ef1 None admin] Authenticating user token {{(pid=80378) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:03:41.175200 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG dbcounter [-] [80378] Writing DB stats keystone:SELECT=100 {{(pid=80378) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 10:03:41.277907 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6a482a2-a772-4ed2-9ddf-f568f0996ef1 None admin] Validating token access rules against request {{(pid=80378) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:03:41.279610 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b4fa70e-af15-4277-af42-514534f4aae1 None admin] Validating token access rules against request {{(pid=80377) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:03:41.326066 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90d05892-c15a-4595-8b81-5d3bf9f440f0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '891005fa5ff74041b75747d16cc66798', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80377) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:03:41.326731 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90d05892-c15a-4595-8b81-5d3bf9f440f0 None admin] REQUEST_METHOD: `GET` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:03:41.327010 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90d05892-c15a-4595-8b81-5d3bf9f440f0 None admin] SCRIPT_NAME: `/identity` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:03:41.327286 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90d05892-c15a-4595-8b81-5d3bf9f440f0 None admin] PATH_INFO: `/v3/users/b0299b32194741659e74f6d38f78aca7` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:03:41.328586 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b4a8dfc-c56f-413b-947f-5c5f7faf69e2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '891005fa5ff74041b75747d16cc66798', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80378) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:03:41.328847 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90d05892-c15a-4595-8b81-5d3bf9f440f0 None admin] RBAC: Authorizing `identity:get_user(user_id=b0299b32194741659e74f6d38f78aca7)` {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:03:41.329343 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b4a8dfc-c56f-413b-947f-5c5f7faf69e2 None admin] REQUEST_METHOD: `GET` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:03:41.329834 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b4a8dfc-c56f-413b-947f-5c5f7faf69e2 None admin] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:03:41.330779 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b4a8dfc-c56f-413b-947f-5c5f7faf69e2 None admin] PATH_INFO: `/v3/projects` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:03:41.331338 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90d05892-c15a-4595-8b81-5d3bf9f440f0 None admin] RBAC: Authorization granted {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:03:41.332023 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b4a8dfc-c56f-413b-947f-5c5f7faf69e2 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80378) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 02 10:03:41.332387 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b4a8dfc-c56f-413b-947f-5c5f7faf69e2 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:03:41.338346 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: [pid: 80377|app: 0|req: 547/1075] 10.222.0.82 () {56 vars in 1177 bytes} [Tue Aug 2 10:03:41 2022] GET /identity/v3/users/b0299b32194741659e74f6d38f78aca7 => generated 285 bytes in 226 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 02 10:03:41.338518 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b4a8dfc-c56f-413b-947f-5c5f7faf69e2 None admin] RBAC: Authorization granted {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:03:41.339295 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90d05892-c15a-4595-8b81-5d3bf9f440f0 None admin] Authenticating user token {{(pid=80377) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:03:41.364428 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 529/1076] 10.222.0.82 () {56 vars in 1148 bytes} [Tue Aug 2 10:03:41 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 216 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 02 10:03:41.366019 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b4a8dfc-c56f-413b-947f-5c5f7faf69e2 None admin] Authenticating user token {{(pid=80378) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:03:41.452180 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG dbcounter [-] [80377] Writing DB stats keystone:SELECT=100 {{(pid=80377) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 10:03:41.512442 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG dbcounter [-] [80378] Writing DB stats keystone:SELECT=100 {{(pid=80378) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 10:03:41.540391 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b4a8dfc-c56f-413b-947f-5c5f7faf69e2 None admin] Validating token access rules against request {{(pid=80378) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:03:41.541085 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90d05892-c15a-4595-8b81-5d3bf9f440f0 None admin] Validating token access rules against request {{(pid=80377) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:03:41.591664 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ff927f4-9fa3-4c25-b85a-d7eb90941a79 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '891005fa5ff74041b75747d16cc66798', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80378) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:03:41.592263 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ff927f4-9fa3-4c25-b85a-d7eb90941a79 None admin] REQUEST_METHOD: `PUT` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:03:41.592544 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ff927f4-9fa3-4c25-b85a-d7eb90941a79 None admin] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:03:41.592826 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ff927f4-9fa3-4c25-b85a-d7eb90941a79 None admin] PATH_INFO: `/v3/projects/f1b38ebb804c41c69772c998aae72c41/users/88af518196674fe593ad88d30111fb7d/roles/f57a652471c8414597c5fdde3d6f4636` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:03:41.595109 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ff927f4-9fa3-4c25-b85a-d7eb90941a79 None admin] RBAC: Authorizing `identity:create_grant(project_id=f1b38ebb804c41c69772c998aae72c41, user_id=88af518196674fe593ad88d30111fb7d, role_id=f57a652471c8414597c5fdde3d6f4636)` {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:03:41.595431 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b48ac832-3e6e-43ad-a246-c66ad5a1de4a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '891005fa5ff74041b75747d16cc66798', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80377) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:03:41.596678 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b48ac832-3e6e-43ad-a246-c66ad5a1de4a None admin] REQUEST_METHOD: `GET` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:03:41.596949 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b48ac832-3e6e-43ad-a246-c66ad5a1de4a None admin] SCRIPT_NAME: `/identity` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:03:41.597134 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ff927f4-9fa3-4c25-b85a-d7eb90941a79 None admin] RBAC: Authorization granted {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:03:41.597240 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b48ac832-3e6e-43ad-a246-c66ad5a1de4a None admin] PATH_INFO: `/v3/projects/edf242cc36af4db9b5d76d897711bb6c` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:03:41.598439 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b48ac832-3e6e-43ad-a246-c66ad5a1de4a None admin] RBAC: Authorizing `identity:get_project(project_id=edf242cc36af4db9b5d76d897711bb6c)` {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:03:41.599866 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b48ac832-3e6e-43ad-a246-c66ad5a1de4a None admin] RBAC: Authorization granted {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:03:41.601734 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: [pid: 80377|app: 0|req: 548/1077] 10.222.0.82 () {56 vars in 1186 bytes} [Tue Aug 2 10:03:41 2022] GET /identity/v3/projects/edf242cc36af4db9b5d76d897711bb6c => generated 311 bytes in 263 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 02 10:03:41.609213 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b48ac832-3e6e-43ad-a246-c66ad5a1de4a None admin] Authenticating user token {{(pid=80377) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:03:41.625814 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 530/1078] 10.222.0.82 () {58 vars in 1439 bytes} [Tue Aug 2 10:03:41 2022] PUT /identity/v3/projects/f1b38ebb804c41c69772c998aae72c41/users/88af518196674fe593ad88d30111fb7d/roles/f57a652471c8414597c5fdde3d6f4636 => generated 0 bytes in 252 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 02 10:03:41.769171 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b48ac832-3e6e-43ad-a246-c66ad5a1de4a None admin] Validating token access rules against request {{(pid=80377) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:03:41.810719 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG dbcounter [-] [80377] Writing DB stats keystone:SELECT=100 {{(pid=80377) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 10:03:41.824895 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57e1e2f9-7584-48d8-9ac6-58e564ef666f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '891005fa5ff74041b75747d16cc66798', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80377) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:03:41.824895 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57e1e2f9-7584-48d8-9ac6-58e564ef666f None admin] REQUEST_METHOD: `PUT` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:03:41.825207 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57e1e2f9-7584-48d8-9ac6-58e564ef666f None admin] SCRIPT_NAME: `/identity` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:03:41.825375 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57e1e2f9-7584-48d8-9ac6-58e564ef666f None admin] PATH_INFO: `/v3/projects/edf242cc36af4db9b5d76d897711bb6c/users/b0299b32194741659e74f6d38f78aca7/roles/237966442f05411191105bf336bc14f3` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:03:41.827758 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57e1e2f9-7584-48d8-9ac6-58e564ef666f None admin] RBAC: Authorizing `identity:create_grant(project_id=edf242cc36af4db9b5d76d897711bb6c, user_id=b0299b32194741659e74f6d38f78aca7, role_id=237966442f05411191105bf336bc14f3)` {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:03:41.829244 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57e1e2f9-7584-48d8-9ac6-58e564ef666f None admin] RBAC: Authorization granted {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:03:41.850303 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: [pid: 80377|app: 0|req: 549/1079] 10.222.0.82 () {58 vars in 1439 bytes} [Tue Aug 2 10:03:41 2022] PUT /identity/v3/projects/edf242cc36af4db9b5d76d897711bb6c/users/b0299b32194741659e74f6d38f78aca7/roles/237966442f05411191105bf336bc14f3 => generated 0 bytes in 240 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 02 10:03:42.469573 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53127aa7-99a2-49a7-bb35-c343525893d2 None None] REQUEST_METHOD: `GET` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:03:42.470278 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53127aa7-99a2-49a7-bb35-c343525893d2 None None] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:03:42.470517 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53127aa7-99a2-49a7-bb35-c343525893d2 None None] PATH_INFO: `/` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:03:42.472831 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 531/1080] 10.222.0.82 () {54 vars in 926 bytes} [Tue Aug 2 10:03:42 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 02 10:03:42.479216 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-044e1ca0-f061-4477-b2a7-140a7951bee1 None None] REQUEST_METHOD: `POST` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:03:42.479551 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-044e1ca0-f061-4477-b2a7-140a7951bee1 None None] SCRIPT_NAME: `/identity` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:03:42.479783 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-044e1ca0-f061-4477-b2a7-140a7951bee1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:03:42.507838 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.auth.core [None req-044e1ca0-f061-4477-b2a7-140a7951bee1 None None] MFA Rules not processed for user `891005fa5ff74041b75747d16cc66798`. Rule list: `[]` (Enabled: `True`). {{(pid=80377) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 02 10:03:42.509079 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.fernet_utils [None req-044e1ca0-f061-4477-b2a7-140a7951bee1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80377) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 10:03:42.615445 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: [pid: 80377|app: 0|req: 550/1081] 10.222.0.82 () {58 vars in 1023 bytes} [Tue Aug 2 10:03:42 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 137 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 02 10:03:42.643440 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53127aa7-99a2-49a7-bb35-c343525893d2 None None] Authenticating user token {{(pid=80378) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:03:42.816734 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53127aa7-99a2-49a7-bb35-c343525893d2 None None] Validating token access rules against request {{(pid=80378) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:03:42.880611 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c92c7a49-9b40-46c9-9808-73e24b1a2d9a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '891005fa5ff74041b75747d16cc66798', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80378) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:03:42.881560 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c92c7a49-9b40-46c9-9808-73e24b1a2d9a None admin] REQUEST_METHOD: `GET` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:03:42.882124 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c92c7a49-9b40-46c9-9808-73e24b1a2d9a None admin] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:03:42.882603 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c92c7a49-9b40-46c9-9808-73e24b1a2d9a None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:03:42.903305 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG dbcounter [-] [80378] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80378) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 10:03:42.979704 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c92c7a49-9b40-46c9-9808-73e24b1a2d9a None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:03:42.981235 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c92c7a49-9b40-46c9-9808-73e24b1a2d9a None admin] RBAC: Authorization granted {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:03:43.122926 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 532/1082] 10.222.0.82 () {58 vars in 1282 bytes} [Tue Aug 2 10:03:42 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 481 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Aug 02 10:03:43.132222 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-044e1ca0-f061-4477-b2a7-140a7951bee1 None None] Authenticating user token {{(pid=80377) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:03:43.257664 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-044e1ca0-f061-4477-b2a7-140a7951bee1 None None] Validating token access rules against request {{(pid=80377) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:03:43.279466 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG dbcounter [-] [80377] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80377) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 10:03:43.302428 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d232386b-88ba-4bee-951c-98db3db86445 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '891005fa5ff74041b75747d16cc66798', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80377) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:03:43.303097 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d232386b-88ba-4bee-951c-98db3db86445 None admin] REQUEST_METHOD: `GET` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:03:43.303381 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d232386b-88ba-4bee-951c-98db3db86445 None admin] SCRIPT_NAME: `/identity` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:03:43.303657 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d232386b-88ba-4bee-951c-98db3db86445 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:03:43.315438 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d232386b-88ba-4bee-951c-98db3db86445 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:03:43.317023 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d232386b-88ba-4bee-951c-98db3db86445 None admin] RBAC: Authorization granted {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:03:43.334669 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application [None req-d232386b-88ba-4bee-951c-98db3db86445 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 02 10:03:43.334669 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 02 10:03:43.334669 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 02 10:03:43.334669 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 02 10:03:43.334669 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 02 10:03:43.334669 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 02 10:03:43.334669 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 02 10:03:43.334669 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 02 10:03:43.334669 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 02 10:03:43.334669 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 02 10:03:43.334669 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 02 10:03:43.334669 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 02 10:03:43.334669 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 02 10:03:43.334669 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 02 10:03:43.334669 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 02 10:03:43.336471 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 02 10:03:43.336471 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 02 10:03:43.336471 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 02 10:03:43.336471 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 02 10:03:43.336471 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 02 10:03:43.336471 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 02 10:03:43.336471 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application return self.get_or_create( Aug 02 10:03:43.336471 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 02 10:03:43.336471 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application with Lock( Aug 02 10:03:43.336471 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 02 10:03:43.336471 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application return self._enter() Aug 02 10:03:43.336471 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 02 10:03:43.336471 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 02 10:03:43.336471 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 02 10:03:43.336471 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application return self.creator() Aug 02 10:03:43.336471 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 02 10:03:43.336471 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application created_value = creator( Aug 02 10:03:43.347057 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 02 10:03:43.347057 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 02 10:03:43.347057 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 02 10:03:43.347057 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application Aug 02 10:03:43.347057 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: [pid: 80377|app: 0|req: 551/1083] 10.222.0.82 () {56 vars in 1108 bytes} [Tue Aug 2 10:03:43 2022] GET /identity/v3/domains/Default => generated 87 bytes in 205 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 02 10:03:43.350708 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c92c7a49-9b40-46c9-9808-73e24b1a2d9a None admin] Authenticating user token {{(pid=80378) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:03:43.431065 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG dbcounter [-] [80378] Writing DB stats keystone:SELECT=100 {{(pid=80378) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 10:03:43.495935 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c92c7a49-9b40-46c9-9808-73e24b1a2d9a None admin] Validating token access rules against request {{(pid=80378) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:03:43.537032 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-172dd553-4001-4b88-b9b5-9720de07d1fb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '891005fa5ff74041b75747d16cc66798', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80378) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:03:43.537659 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-172dd553-4001-4b88-b9b5-9720de07d1fb None admin] REQUEST_METHOD: `GET` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:03:43.537976 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-172dd553-4001-4b88-b9b5-9720de07d1fb None admin] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:03:43.538250 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-172dd553-4001-4b88-b9b5-9720de07d1fb None admin] PATH_INFO: `/v3/domains` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:03:43.539030 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-172dd553-4001-4b88-b9b5-9720de07d1fb None admin] RBAC: Adding query filter params (name=Default) {{(pid=80378) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 02 10:03:43.539341 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-172dd553-4001-4b88-b9b5-9720de07d1fb None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:03:43.540763 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-172dd553-4001-4b88-b9b5-9720de07d1fb None admin] RBAC: Authorization granted {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:03:43.554697 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 533/1084] 10.222.0.82 () {56 vars in 1109 bytes} [Tue Aug 2 10:03:43 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 205 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 02 10:03:43.854240 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2af2ee7-b27a-4124-ac4b-accce3b05395 None None] REQUEST_METHOD: `GET` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:03:43.855266 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2af2ee7-b27a-4124-ac4b-accce3b05395 None None] SCRIPT_NAME: `/identity` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:03:43.856003 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2af2ee7-b27a-4124-ac4b-accce3b05395 None None] PATH_INFO: `/` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:03:43.859129 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: [pid: 80377|app: 0|req: 552/1085] 10.222.0.82 () {54 vars in 926 bytes} [Tue Aug 2 10:03:43 2022] GET /identity => generated 270 bytes in 7 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 02 10:03:43.867505 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09549de1-0055-4311-887c-21507927db04 None None] REQUEST_METHOD: `POST` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:03:43.867792 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09549de1-0055-4311-887c-21507927db04 None None] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:03:43.868307 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09549de1-0055-4311-887c-21507927db04 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:03:43.891941 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.auth.core [None req-09549de1-0055-4311-887c-21507927db04 None None] MFA Rules not processed for user `891005fa5ff74041b75747d16cc66798`. Rule list: `[]` (Enabled: `True`). {{(pid=80378) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 02 10:03:43.893763 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.fernet_utils [None req-09549de1-0055-4311-887c-21507927db04 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80378) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 10:03:43.971381 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 534/1086] 10.222.0.82 () {58 vars in 1023 bytes} [Tue Aug 2 10:03:43 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 104 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 02 10:03:44.003653 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2af2ee7-b27a-4124-ac4b-accce3b05395 None None] Authenticating user token {{(pid=80377) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:03:44.123143 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73010359-bb2a-43b4-a59f-dbd5a64c9464 None None] REQUEST_METHOD: `GET` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:03:44.123328 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73010359-bb2a-43b4-a59f-dbd5a64c9464 None None] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:03:44.123610 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73010359-bb2a-43b4-a59f-dbd5a64c9464 None None] PATH_INFO: `/` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:03:44.125046 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 535/1087] 10.222.0.82 () {54 vars in 926 bytes} [Tue Aug 2 10:03:44 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 02 10:03:44.130771 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3ae44d1-41cb-42ce-9ac9-47f760bf3b60 None None] REQUEST_METHOD: `POST` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:03:44.131059 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3ae44d1-41cb-42ce-9ac9-47f760bf3b60 None None] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:03:44.131332 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3ae44d1-41cb-42ce-9ac9-47f760bf3b60 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:03:44.151723 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.auth.core [None req-d3ae44d1-41cb-42ce-9ac9-47f760bf3b60 None None] MFA Rules not processed for user `891005fa5ff74041b75747d16cc66798`. Rule list: `[]` (Enabled: `True`). {{(pid=80378) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 02 10:03:44.152953 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.fernet_utils [None req-d3ae44d1-41cb-42ce-9ac9-47f760bf3b60 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80378) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 10:03:44.164998 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2af2ee7-b27a-4124-ac4b-accce3b05395 None None] Validating token access rules against request {{(pid=80377) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:03:44.185486 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG dbcounter [-] [80378] Writing DB stats keystone:SELECT=100 {{(pid=80378) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 10:03:44.210364 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-613aa9d7-c985-4a68-85c3-f9183f145b9f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '891005fa5ff74041b75747d16cc66798', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80377) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:03:44.211036 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-613aa9d7-c985-4a68-85c3-f9183f145b9f None admin] REQUEST_METHOD: `GET` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:03:44.211367 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-613aa9d7-c985-4a68-85c3-f9183f145b9f None admin] SCRIPT_NAME: `/identity` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:03:44.211620 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-613aa9d7-c985-4a68-85c3-f9183f145b9f None admin] PATH_INFO: `/v3/roles/ResellerAdmin` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:03:44.224654 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-613aa9d7-c985-4a68-85c3-f9183f145b9f None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 02 10:03:44.229854 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application [None req-613aa9d7-c985-4a68-85c3-f9183f145b9f None admin] Could not find role: ResellerAdmin.: keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Aug 02 10:03:44.229854 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 02 10:03:44.229854 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 02 10:03:44.229854 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 02 10:03:44.229854 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 02 10:03:44.229854 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 02 10:03:44.229854 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 02 10:03:44.229854 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 02 10:03:44.229854 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 02 10:03:44.229854 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 02 10:03:44.229854 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 02 10:03:44.229854 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 02 10:03:44.229854 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 02 10:03:44.229854 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 02 10:03:44.229854 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 02 10:03:44.229854 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 02 10:03:44.231332 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 02 10:03:44.231332 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 02 10:03:44.231332 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 02 10:03:44.231332 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 02 10:03:44.231332 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 02 10:03:44.231332 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 02 10:03:44.231332 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 02 10:03:44.231332 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 02 10:03:44.231332 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 02 10:03:44.231332 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application return self.get_or_create( Aug 02 10:03:44.231332 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 02 10:03:44.231332 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application with Lock( Aug 02 10:03:44.231332 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 02 10:03:44.231332 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application return self._enter() Aug 02 10:03:44.231332 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 02 10:03:44.231332 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 02 10:03:44.232625 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 02 10:03:44.232625 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application return self.creator() Aug 02 10:03:44.232625 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 02 10:03:44.232625 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application created_value = creator( Aug 02 10:03:44.232625 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 02 10:03:44.232625 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 02 10:03:44.232625 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 02 10:03:44.232625 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 02 10:03:44.232625 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 02 10:03:44.232625 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 02 10:03:44.232625 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Aug 02 10:03:44.232625 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application Aug 02 10:03:44.234467 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: [pid: 80377|app: 0|req: 553/1088] 10.222.0.82 () {56 vars in 1120 bytes} [Tue Aug 2 10:03:44 2022] GET /identity/v3/roles/ResellerAdmin => generated 91 bytes in 231 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 02 10:03:44.239251 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-613aa9d7-c985-4a68-85c3-f9183f145b9f None admin] Authenticating user token {{(pid=80377) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:03:44.247881 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 536/1089] 10.222.0.82 () {58 vars in 1023 bytes} [Tue Aug 2 10:03:44 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 118 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Aug 02 10:03:44.275967 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG dbcounter [-] [80377] Writing DB stats keystone:SELECT=100 {{(pid=80377) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 10:03:44.277629 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3ae44d1-41cb-42ce-9ac9-47f760bf3b60 None None] Authenticating user token {{(pid=80378) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:03:44.376843 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-613aa9d7-c985-4a68-85c3-f9183f145b9f None admin] Validating token access rules against request {{(pid=80377) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:03:44.427918 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c2369e9-ba52-455c-8252-2bc41011c3ad None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '891005fa5ff74041b75747d16cc66798', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80377) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:03:44.428555 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c2369e9-ba52-455c-8252-2bc41011c3ad None admin] REQUEST_METHOD: `GET` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:03:44.428840 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c2369e9-ba52-455c-8252-2bc41011c3ad None admin] SCRIPT_NAME: `/identity` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:03:44.429116 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c2369e9-ba52-455c-8252-2bc41011c3ad None admin] PATH_INFO: `/v3/roles` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:03:44.429884 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c2369e9-ba52-455c-8252-2bc41011c3ad None admin] RBAC: Adding query filter params (name=ResellerAdmin) {{(pid=80377) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 02 10:03:44.430190 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c2369e9-ba52-455c-8252-2bc41011c3ad None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:03:44.431602 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c2369e9-ba52-455c-8252-2bc41011c3ad None admin] RBAC: Authorization granted {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:03:44.434596 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3ae44d1-41cb-42ce-9ac9-47f760bf3b60 None None] Validating token access rules against request {{(pid=80378) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:03:44.442398 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: [pid: 80377|app: 0|req: 554/1090] 10.222.0.82 () {56 vars in 1115 bytes} [Tue Aug 2 10:03:44 2022] GET /identity/v3/roles?name=ResellerAdmin => generated 338 bytes in 205 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 02 10:03:44.452583 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c2369e9-ba52-455c-8252-2bc41011c3ad None admin] Authenticating user token {{(pid=80377) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:03:44.481320 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3f6565a-e8c9-49e5-8e7d-bc33093da013 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '891005fa5ff74041b75747d16cc66798', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80378) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:03:44.482010 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3f6565a-e8c9-49e5-8e7d-bc33093da013 None admin] REQUEST_METHOD: `GET` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:03:44.482299 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3f6565a-e8c9-49e5-8e7d-bc33093da013 None admin] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:03:44.482577 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3f6565a-e8c9-49e5-8e7d-bc33093da013 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:03:44.500316 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3f6565a-e8c9-49e5-8e7d-bc33093da013 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 02 10:03:44.506260 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application [None req-c3f6565a-e8c9-49e5-8e7d-bc33093da013 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 02 10:03:44.506260 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 02 10:03:44.506260 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 02 10:03:44.506260 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 02 10:03:44.506260 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 02 10:03:44.506260 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 02 10:03:44.506260 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 02 10:03:44.506260 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 02 10:03:44.506260 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 02 10:03:44.506260 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 02 10:03:44.506260 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 02 10:03:44.506260 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 02 10:03:44.506260 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 02 10:03:44.506260 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 02 10:03:44.506260 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 02 10:03:44.506260 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 02 10:03:44.507280 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 02 10:03:44.507280 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 02 10:03:44.507280 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 02 10:03:44.507280 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 02 10:03:44.507280 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 02 10:03:44.507280 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 02 10:03:44.507280 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 02 10:03:44.507280 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 02 10:03:44.507280 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 02 10:03:44.507280 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application return self.get_or_create( Aug 02 10:03:44.507280 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 02 10:03:44.507280 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application with Lock( Aug 02 10:03:44.507280 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 02 10:03:44.507280 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application return self._enter() Aug 02 10:03:44.507280 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 02 10:03:44.507280 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 02 10:03:44.508154 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 02 10:03:44.508154 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application return self.creator() Aug 02 10:03:44.508154 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 02 10:03:44.508154 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application created_value = creator( Aug 02 10:03:44.508154 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 02 10:03:44.508154 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 02 10:03:44.508154 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 02 10:03:44.508154 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 02 10:03:44.508154 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 02 10:03:44.508154 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 02 10:03:44.508154 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 02 10:03:44.508154 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application Aug 02 10:03:44.508154 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 537/1091] 10.222.0.82 () {56 vars in 1096 bytes} [Tue Aug 2 10:03:44 2022] GET /identity/v3/roles/admin => generated 83 bytes in 230 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 02 10:03:44.515668 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3f6565a-e8c9-49e5-8e7d-bc33093da013 None admin] Authenticating user token {{(pid=80378) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:03:44.547872 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG dbcounter [-] [80378] Writing DB stats keystone:SELECT=100 {{(pid=80378) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 10:03:44.591076 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG dbcounter [-] [80377] Writing DB stats keystone:SELECT=100 {{(pid=80377) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 10:03:44.604028 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c2369e9-ba52-455c-8252-2bc41011c3ad None admin] Validating token access rules against request {{(pid=80377) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:03:44.651768 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17dabcb3-5a82-41ff-91a4-dadc51d3d65b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '891005fa5ff74041b75747d16cc66798', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80377) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:03:44.652834 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17dabcb3-5a82-41ff-91a4-dadc51d3d65b None admin] REQUEST_METHOD: `GET` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:03:44.653329 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17dabcb3-5a82-41ff-91a4-dadc51d3d65b None admin] SCRIPT_NAME: `/identity` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:03:44.653822 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17dabcb3-5a82-41ff-91a4-dadc51d3d65b None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:03:44.674776 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17dabcb3-5a82-41ff-91a4-dadc51d3d65b None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:03:44.676074 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3f6565a-e8c9-49e5-8e7d-bc33093da013 None admin] Validating token access rules against request {{(pid=80378) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:03:44.677462 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17dabcb3-5a82-41ff-91a4-dadc51d3d65b None admin] RBAC: Authorization granted {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:03:44.694470 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application [None req-17dabcb3-5a82-41ff-91a4-dadc51d3d65b None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 02 10:03:44.694470 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 02 10:03:44.694470 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 02 10:03:44.694470 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 02 10:03:44.694470 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 02 10:03:44.694470 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 02 10:03:44.694470 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 02 10:03:44.694470 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 02 10:03:44.694470 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 02 10:03:44.694470 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 02 10:03:44.694470 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 02 10:03:44.694470 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 02 10:03:44.694470 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 02 10:03:44.694470 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 02 10:03:44.694470 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 02 10:03:44.695817 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 02 10:03:44.695817 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 02 10:03:44.695817 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 02 10:03:44.695817 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 02 10:03:44.695817 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 02 10:03:44.695817 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 02 10:03:44.695817 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application return self.get_or_create( Aug 02 10:03:44.695817 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 02 10:03:44.695817 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application with Lock( Aug 02 10:03:44.695817 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 02 10:03:44.695817 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application return self._enter() Aug 02 10:03:44.695817 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 02 10:03:44.695817 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 02 10:03:44.695817 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 02 10:03:44.695817 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application return self.creator() Aug 02 10:03:44.695817 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 02 10:03:44.695817 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application created_value = creator( Aug 02 10:03:44.696885 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 02 10:03:44.696885 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 02 10:03:44.696885 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 02 10:03:44.696885 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application Aug 02 10:03:44.696885 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: [pid: 80377|app: 0|req: 555/1092] 10.222.0.82 () {56 vars in 1108 bytes} [Tue Aug 2 10:03:44 2022] GET /identity/v3/domains/Default => generated 87 bytes in 244 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 02 10:03:44.701990 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17dabcb3-5a82-41ff-91a4-dadc51d3d65b None admin] Authenticating user token {{(pid=80377) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:03:44.732049 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ce3ceda-50a1-482b-9528-573fb87763a2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '891005fa5ff74041b75747d16cc66798', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80378) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:03:44.732821 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ce3ceda-50a1-482b-9528-573fb87763a2 None admin] REQUEST_METHOD: `GET` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:03:44.733208 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ce3ceda-50a1-482b-9528-573fb87763a2 None admin] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:03:44.733604 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ce3ceda-50a1-482b-9528-573fb87763a2 None admin] PATH_INFO: `/v3/roles` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:03:44.734818 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ce3ceda-50a1-482b-9528-573fb87763a2 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80378) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 02 10:03:44.735166 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ce3ceda-50a1-482b-9528-573fb87763a2 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:03:44.737391 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ce3ceda-50a1-482b-9528-573fb87763a2 None admin] RBAC: Authorization granted {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:03:44.746652 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 538/1093] 10.222.0.82 () {56 vars in 1099 bytes} [Tue Aug 2 10:03:44 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 231 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 02 10:03:44.752752 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ce3ceda-50a1-482b-9528-573fb87763a2 None admin] Authenticating user token {{(pid=80378) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:03:44.841002 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17dabcb3-5a82-41ff-91a4-dadc51d3d65b None admin] Validating token access rules against request {{(pid=80377) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:03:44.881410 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG dbcounter [-] [80378] Writing DB stats keystone:SELECT=100 {{(pid=80378) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 10:03:44.895504 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aca40882-63d7-4816-b4c6-14d7c7e4a8ec None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '891005fa5ff74041b75747d16cc66798', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80377) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:03:44.896291 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aca40882-63d7-4816-b4c6-14d7c7e4a8ec None admin] REQUEST_METHOD: `GET` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:03:44.896672 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aca40882-63d7-4816-b4c6-14d7c7e4a8ec None admin] SCRIPT_NAME: `/identity` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:03:44.897044 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aca40882-63d7-4816-b4c6-14d7c7e4a8ec None admin] PATH_INFO: `/v3/domains` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:03:44.897956 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aca40882-63d7-4816-b4c6-14d7c7e4a8ec None admin] RBAC: Adding query filter params (name=Default) {{(pid=80377) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 02 10:03:44.898391 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aca40882-63d7-4816-b4c6-14d7c7e4a8ec None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:03:44.900438 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aca40882-63d7-4816-b4c6-14d7c7e4a8ec None admin] RBAC: Authorization granted {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:03:44.906546 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ce3ceda-50a1-482b-9528-573fb87763a2 None admin] Validating token access rules against request {{(pid=80378) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:03:44.917589 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: [pid: 80377|app: 0|req: 556/1094] 10.222.0.82 () {56 vars in 1109 bytes} [Tue Aug 2 10:03:44 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 216 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 02 10:03:44.925579 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aca40882-63d7-4816-b4c6-14d7c7e4a8ec None admin] Authenticating user token {{(pid=80377) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:03:44.931828 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG dbcounter [-] [80377] Writing DB stats keystone:SELECT=100 {{(pid=80377) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 10:03:44.963127 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0f4dc79-0f70-43ec-8631-48f4eaa0081c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '891005fa5ff74041b75747d16cc66798', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80378) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:03:44.963945 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0f4dc79-0f70-43ec-8631-48f4eaa0081c None admin] REQUEST_METHOD: `GET` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:03:44.964311 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0f4dc79-0f70-43ec-8631-48f4eaa0081c None admin] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:03:44.964639 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0f4dc79-0f70-43ec-8631-48f4eaa0081c None admin] PATH_INFO: `/v3/users/b0299b32194741659e74f6d38f78aca7` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:03:44.966024 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0f4dc79-0f70-43ec-8631-48f4eaa0081c None admin] RBAC: Authorizing `identity:get_user(user_id=b0299b32194741659e74f6d38f78aca7)` {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:03:44.967778 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0f4dc79-0f70-43ec-8631-48f4eaa0081c None admin] RBAC: Authorization granted {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:03:44.969655 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 539/1095] 10.222.0.82 () {56 vars in 1177 bytes} [Tue Aug 2 10:03:44 2022] GET /identity/v3/users/b0299b32194741659e74f6d38f78aca7 => generated 285 bytes in 217 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 02 10:03:44.976524 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0f4dc79-0f70-43ec-8631-48f4eaa0081c None admin] Authenticating user token {{(pid=80378) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:03:45.060574 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aca40882-63d7-4816-b4c6-14d7c7e4a8ec None admin] Validating token access rules against request {{(pid=80377) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:03:45.110248 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67ae8533-214c-4d9e-a110-724b16ba0e83 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '891005fa5ff74041b75747d16cc66798', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80377) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:03:45.110904 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67ae8533-214c-4d9e-a110-724b16ba0e83 None admin] REQUEST_METHOD: `GET` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:03:45.111184 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67ae8533-214c-4d9e-a110-724b16ba0e83 None admin] SCRIPT_NAME: `/identity` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:03:45.111612 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67ae8533-214c-4d9e-a110-724b16ba0e83 None admin] PATH_INFO: `/v3/users/nova` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:03:45.126303 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67ae8533-214c-4d9e-a110-724b16ba0e83 None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:03:45.128994 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67ae8533-214c-4d9e-a110-724b16ba0e83 None admin] RBAC: Authorization granted {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:03:45.146373 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application [None req-67ae8533-214c-4d9e-a110-724b16ba0e83 None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Aug 02 10:03:45.146373 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 02 10:03:45.146373 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 02 10:03:45.146373 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 02 10:03:45.146373 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 02 10:03:45.146373 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 02 10:03:45.146373 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 02 10:03:45.146373 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 02 10:03:45.146373 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 02 10:03:45.146373 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 02 10:03:45.146373 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 02 10:03:45.146373 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 02 10:03:45.146373 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 02 10:03:45.146373 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 02 10:03:45.146373 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 02 10:03:45.146373 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 02 10:03:45.147983 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 02 10:03:45.147983 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 02 10:03:45.147983 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 02 10:03:45.147983 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 02 10:03:45.147983 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 02 10:03:45.147983 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 02 10:03:45.147983 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 02 10:03:45.147983 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 02 10:03:45.147983 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 02 10:03:45.147983 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application return self.get_or_create( Aug 02 10:03:45.147983 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 02 10:03:45.147983 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application with Lock( Aug 02 10:03:45.147983 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 02 10:03:45.147983 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application return self._enter() Aug 02 10:03:45.147983 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 02 10:03:45.147983 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 02 10:03:45.147983 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 02 10:03:45.147983 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application return self.creator() Aug 02 10:03:45.149268 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 02 10:03:45.149268 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application created_value = creator( Aug 02 10:03:45.149268 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 02 10:03:45.149268 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 02 10:03:45.149268 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 02 10:03:45.149268 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 02 10:03:45.149268 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 02 10:03:45.149268 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 02 10:03:45.149268 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Aug 02 10:03:45.149268 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application Aug 02 10:03:45.149907 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0f4dc79-0f70-43ec-8631-48f4eaa0081c None admin] Validating token access rules against request {{(pid=80378) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:03:45.150040 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: [pid: 80377|app: 0|req: 557/1096] 10.222.0.82 () {56 vars in 1093 bytes} [Tue Aug 2 10:03:44 2022] GET /identity/v3/users/nova => generated 82 bytes in 224 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 02 10:03:45.161880 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67ae8533-214c-4d9e-a110-724b16ba0e83 None admin] Authenticating user token {{(pid=80377) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:03:45.202638 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-875efd08-aa6b-4070-b696-e646db462efd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '891005fa5ff74041b75747d16cc66798', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80378) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:03:45.203276 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-875efd08-aa6b-4070-b696-e646db462efd None admin] REQUEST_METHOD: `GET` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:03:45.203579 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-875efd08-aa6b-4070-b696-e646db462efd None admin] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:03:45.203824 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-875efd08-aa6b-4070-b696-e646db462efd None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:03:45.232057 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG dbcounter [-] [80378] Writing DB stats keystone:SELECT=100 {{(pid=80378) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 10:03:45.242512 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG dbcounter [-] [80377] Writing DB stats keystone:SELECT=100 {{(pid=80377) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 10:03:45.287725 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-875efd08-aa6b-4070-b696-e646db462efd None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:03:45.289365 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-875efd08-aa6b-4070-b696-e646db462efd None admin] RBAC: Authorization granted {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:03:45.305869 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67ae8533-214c-4d9e-a110-724b16ba0e83 None admin] Validating token access rules against request {{(pid=80377) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:03:45.343578 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c2d6ac7-e05c-4610-92a0-4f12121f09b5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '891005fa5ff74041b75747d16cc66798', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80377) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:03:45.344216 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c2d6ac7-e05c-4610-92a0-4f12121f09b5 None admin] REQUEST_METHOD: `GET` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:03:45.344502 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c2d6ac7-e05c-4610-92a0-4f12121f09b5 None admin] SCRIPT_NAME: `/identity` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:03:45.344837 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c2d6ac7-e05c-4610-92a0-4f12121f09b5 None admin] PATH_INFO: `/v3/users` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:03:45.345762 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c2d6ac7-e05c-4610-92a0-4f12121f09b5 None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=80377) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 02 10:03:45.346168 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c2d6ac7-e05c-4610-92a0-4f12121f09b5 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:03:45.348128 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c2d6ac7-e05c-4610-92a0-4f12121f09b5 None admin] RBAC: Authorization granted {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:03:45.370156 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: [pid: 80377|app: 0|req: 558/1097] 10.222.0.82 () {56 vars in 1133 bytes} [Tue Aug 2 10:03:45 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 209 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 02 10:03:45.378474 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c2d6ac7-e05c-4610-92a0-4f12121f09b5 None admin] Authenticating user token {{(pid=80377) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:03:45.423792 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 540/1098] 10.222.0.82 () {58 vars in 1282 bytes} [Tue Aug 2 10:03:44 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 448 msecs (HTTP/1.1 200) 6 headers in 359 bytes (2 switches on core 0) Aug 02 10:03:45.429670 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-875efd08-aa6b-4070-b696-e646db462efd None admin] Authenticating user token {{(pid=80378) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:03:45.511255 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG dbcounter [-] [80378] Writing DB stats keystone:SELECT=100 {{(pid=80378) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 10:03:45.517752 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c2d6ac7-e05c-4610-92a0-4f12121f09b5 None admin] Validating token access rules against request {{(pid=80377) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:03:45.546954 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG dbcounter [-] [80377] Writing DB stats keystone:SELECT=100 {{(pid=80377) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 10:03:45.568184 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a6a948f-efee-499e-90bf-d0ddca0f6375 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '891005fa5ff74041b75747d16cc66798', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80377) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:03:45.569188 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a6a948f-efee-499e-90bf-d0ddca0f6375 None admin] REQUEST_METHOD: `GET` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:03:45.569661 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a6a948f-efee-499e-90bf-d0ddca0f6375 None admin] SCRIPT_NAME: `/identity` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:03:45.570194 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a6a948f-efee-499e-90bf-d0ddca0f6375 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:03:45.586809 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-875efd08-aa6b-4070-b696-e646db462efd None admin] Validating token access rules against request {{(pid=80378) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:03:45.632798 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-211a61d8-1217-4ed4-a05a-20ff742f1868 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '891005fa5ff74041b75747d16cc66798', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80378) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:03:45.633466 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-211a61d8-1217-4ed4-a05a-20ff742f1868 None admin] REQUEST_METHOD: `GET` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:03:45.633754 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-211a61d8-1217-4ed4-a05a-20ff742f1868 None admin] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:03:45.634058 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-211a61d8-1217-4ed4-a05a-20ff742f1868 None admin] PATH_INFO: `/v3/projects/edf242cc36af4db9b5d76d897711bb6c` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:03:45.635308 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-211a61d8-1217-4ed4-a05a-20ff742f1868 None admin] RBAC: Authorizing `identity:get_project(project_id=edf242cc36af4db9b5d76d897711bb6c)` {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:03:45.636978 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-211a61d8-1217-4ed4-a05a-20ff742f1868 None admin] RBAC: Authorization granted {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:03:45.638752 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 541/1099] 10.222.0.82 () {56 vars in 1186 bytes} [Tue Aug 2 10:03:45 2022] GET /identity/v3/projects/edf242cc36af4db9b5d76d897711bb6c => generated 311 bytes in 210 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 02 10:03:45.647939 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-211a61d8-1217-4ed4-a05a-20ff742f1868 None admin] Authenticating user token {{(pid=80378) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:03:45.662648 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a6a948f-efee-499e-90bf-d0ddca0f6375 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:03:45.664357 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a6a948f-efee-499e-90bf-d0ddca0f6375 None admin] RBAC: Authorization granted {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:03:45.777146 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-211a61d8-1217-4ed4-a05a-20ff742f1868 None admin] Validating token access rules against request {{(pid=80378) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:03:45.789715 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: [pid: 80377|app: 0|req: 559/1100] 10.222.0.82 () {58 vars in 1282 bytes} [Tue Aug 2 10:03:45 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 412 msecs (HTTP/1.1 200) 6 headers in 359 bytes (2 switches on core 0) Aug 02 10:03:45.796062 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a6a948f-efee-499e-90bf-d0ddca0f6375 None admin] Authenticating user token {{(pid=80377) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:03:45.806133 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG dbcounter [-] [80378] Writing DB stats keystone:SELECT=100 {{(pid=80378) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 10:03:45.821514 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG dbcounter [-] [80377] Writing DB stats keystone:SELECT=100 {{(pid=80377) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 10:03:45.821730 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5177e5aa-6f4b-4e62-803e-8968e893e444 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '891005fa5ff74041b75747d16cc66798', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80378) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:03:45.822314 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5177e5aa-6f4b-4e62-803e-8968e893e444 None admin] REQUEST_METHOD: `GET` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:03:45.822727 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5177e5aa-6f4b-4e62-803e-8968e893e444 None admin] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:03:45.823117 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5177e5aa-6f4b-4e62-803e-8968e893e444 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:03:45.824079 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5177e5aa-6f4b-4e62-803e-8968e893e444 None admin] RBAC: Adding query filter params (role.id=237966442f05411191105bf336bc14f3, scope.project.id=edf242cc36af4db9b5d76d897711bb6c, user.id=b0299b32194741659e74f6d38f78aca7) {{(pid=80378) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 02 10:03:45.824408 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5177e5aa-6f4b-4e62-803e-8968e893e444 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:03:45.829628 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5177e5aa-6f4b-4e62-803e-8968e893e444 None admin] RBAC: Authorization granted {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:03:45.838370 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 542/1101] 10.222.0.82 () {56 vars in 1374 bytes} [Tue Aug 2 10:03:45 2022] GET /identity/v3/role_assignments?user.id=b0299b32194741659e74f6d38f78aca7&scope.project.id=edf242cc36af4db9b5d76d897711bb6c&role.id=237966442f05411191105bf336bc14f3 => generated 608 bytes in 191 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 02 10:03:45.965825 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a6a948f-efee-499e-90bf-d0ddca0f6375 None admin] Validating token access rules against request {{(pid=80377) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:03:46.021181 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afe37200-a9a2-42ec-b6a5-7f21381eb842 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '891005fa5ff74041b75747d16cc66798', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80377) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:03:46.021852 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afe37200-a9a2-42ec-b6a5-7f21381eb842 None admin] REQUEST_METHOD: `GET` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:03:46.022129 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afe37200-a9a2-42ec-b6a5-7f21381eb842 None admin] SCRIPT_NAME: `/identity` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:03:46.022406 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afe37200-a9a2-42ec-b6a5-7f21381eb842 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:03:46.037965 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-afe37200-a9a2-42ec-b6a5-7f21381eb842 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:03:46.039545 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-afe37200-a9a2-42ec-b6a5-7f21381eb842 None admin] RBAC: Authorization granted {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:03:46.063417 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application [None req-afe37200-a9a2-42ec-b6a5-7f21381eb842 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 02 10:03:46.063417 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 02 10:03:46.063417 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 02 10:03:46.063417 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 02 10:03:46.063417 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 02 10:03:46.063417 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 02 10:03:46.063417 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 02 10:03:46.063417 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 02 10:03:46.063417 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 02 10:03:46.063417 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 02 10:03:46.063417 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 02 10:03:46.063417 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 02 10:03:46.063417 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 02 10:03:46.063417 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 02 10:03:46.063417 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 02 10:03:46.064637 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 02 10:03:46.064637 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 02 10:03:46.064637 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 02 10:03:46.064637 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 02 10:03:46.064637 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 02 10:03:46.064637 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 02 10:03:46.064637 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application return self.get_or_create( Aug 02 10:03:46.064637 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 02 10:03:46.064637 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application with Lock( Aug 02 10:03:46.064637 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 02 10:03:46.064637 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application return self._enter() Aug 02 10:03:46.064637 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 02 10:03:46.064637 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 02 10:03:46.064637 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 02 10:03:46.064637 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application return self.creator() Aug 02 10:03:46.064637 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 02 10:03:46.064637 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application created_value = creator( Aug 02 10:03:46.066656 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 02 10:03:46.066656 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 02 10:03:46.066656 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 02 10:03:46.066656 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application Aug 02 10:03:46.066656 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: [pid: 80377|app: 0|req: 560/1102] 10.222.0.82 () {56 vars in 1108 bytes} [Tue Aug 2 10:03:45 2022] GET /identity/v3/domains/Default => generated 87 bytes in 270 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 02 10:03:46.071769 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5177e5aa-6f4b-4e62-803e-8968e893e444 None admin] Authenticating user token {{(pid=80378) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:03:46.227192 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5177e5aa-6f4b-4e62-803e-8968e893e444 None admin] Validating token access rules against request {{(pid=80378) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:03:46.277431 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-432c119f-6272-460f-82a6-70f91f8d0198 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '891005fa5ff74041b75747d16cc66798', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80378) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:03:46.278118 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-432c119f-6272-460f-82a6-70f91f8d0198 None admin] REQUEST_METHOD: `GET` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:03:46.278403 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-432c119f-6272-460f-82a6-70f91f8d0198 None admin] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:03:46.278671 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-432c119f-6272-460f-82a6-70f91f8d0198 None admin] PATH_INFO: `/v3/domains` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:03:46.279385 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-432c119f-6272-460f-82a6-70f91f8d0198 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80378) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 02 10:03:46.279689 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-432c119f-6272-460f-82a6-70f91f8d0198 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:03:46.281535 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-432c119f-6272-460f-82a6-70f91f8d0198 None admin] RBAC: Authorization granted {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:03:46.301947 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 543/1103] 10.222.0.82 () {56 vars in 1109 bytes} [Tue Aug 2 10:03:46 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 231 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 02 10:03:46.312222 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afe37200-a9a2-42ec-b6a5-7f21381eb842 None admin] Authenticating user token {{(pid=80377) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:03:46.417860 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG dbcounter [-] [80377] Writing DB stats keystone:SELECT=100 {{(pid=80377) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 10:03:46.462525 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afe37200-a9a2-42ec-b6a5-7f21381eb842 None admin] Validating token access rules against request {{(pid=80377) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:03:46.510296 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe4bd0e3-b2b7-45cf-af06-fd5f7a87e66a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '891005fa5ff74041b75747d16cc66798', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80377) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:03:46.511075 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe4bd0e3-b2b7-45cf-af06-fd5f7a87e66a None admin] REQUEST_METHOD: `GET` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:03:46.511516 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe4bd0e3-b2b7-45cf-af06-fd5f7a87e66a None admin] SCRIPT_NAME: `/identity` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:03:46.511710 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe4bd0e3-b2b7-45cf-af06-fd5f7a87e66a None admin] PATH_INFO: `/v3/projects/service` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:03:46.521848 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe4bd0e3-b2b7-45cf-af06-fd5f7a87e66a None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:03:46.523481 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe4bd0e3-b2b7-45cf-af06-fd5f7a87e66a None admin] RBAC: Authorization granted {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:03:46.537778 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application [None req-fe4bd0e3-b2b7-45cf-af06-fd5f7a87e66a None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 02 10:03:46.537778 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 02 10:03:46.537778 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 02 10:03:46.537778 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 02 10:03:46.537778 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 02 10:03:46.537778 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 02 10:03:46.537778 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 02 10:03:46.537778 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 02 10:03:46.537778 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 02 10:03:46.537778 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 02 10:03:46.537778 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 02 10:03:46.537778 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 02 10:03:46.537778 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 02 10:03:46.537778 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 02 10:03:46.537778 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 02 10:03:46.539355 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 02 10:03:46.539355 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 02 10:03:46.539355 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 02 10:03:46.539355 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 02 10:03:46.539355 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 02 10:03:46.539355 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 02 10:03:46.539355 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application return self.get_or_create( Aug 02 10:03:46.539355 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 02 10:03:46.539355 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application with Lock( Aug 02 10:03:46.539355 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 02 10:03:46.539355 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application return self._enter() Aug 02 10:03:46.539355 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 02 10:03:46.539355 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 02 10:03:46.539355 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 02 10:03:46.539355 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application return self.creator() Aug 02 10:03:46.539355 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 02 10:03:46.539355 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application created_value = creator( Aug 02 10:03:46.540424 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 02 10:03:46.540424 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 02 10:03:46.540424 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 02 10:03:46.540424 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 02 10:03:46.540424 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 02 10:03:46.540424 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 02 10:03:46.540424 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 02 10:03:46.540424 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application Aug 02 10:03:46.540424 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: [pid: 80377|app: 0|req: 561/1104] 10.222.0.82 () {56 vars in 1111 bytes} [Tue Aug 2 10:03:46 2022] GET /identity/v3/projects/service => generated 88 bytes in 228 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 02 10:03:46.545501 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-432c119f-6272-460f-82a6-70f91f8d0198 None admin] Authenticating user token {{(pid=80378) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:03:46.615864 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG dbcounter [-] [80378] Writing DB stats keystone:SELECT=100 {{(pid=80378) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 10:03:46.693357 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-432c119f-6272-460f-82a6-70f91f8d0198 None admin] Validating token access rules against request {{(pid=80378) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:03:46.743293 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a58400a8-9e0e-4fca-9887-227e8aa7040f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '891005fa5ff74041b75747d16cc66798', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80378) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:03:46.743945 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a58400a8-9e0e-4fca-9887-227e8aa7040f None admin] REQUEST_METHOD: `GET` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:03:46.744227 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a58400a8-9e0e-4fca-9887-227e8aa7040f None admin] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:03:46.744503 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a58400a8-9e0e-4fca-9887-227e8aa7040f None admin] PATH_INFO: `/v3/projects` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:03:46.745273 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a58400a8-9e0e-4fca-9887-227e8aa7040f None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80378) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 02 10:03:46.745581 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a58400a8-9e0e-4fca-9887-227e8aa7040f None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:03:46.747084 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a58400a8-9e0e-4fca-9887-227e8aa7040f None admin] RBAC: Authorization granted {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:03:46.760998 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 544/1105] 10.222.0.82 () {56 vars in 1148 bytes} [Tue Aug 2 10:03:46 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 216 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 02 10:03:46.767258 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe4bd0e3-b2b7-45cf-af06-fd5f7a87e66a None admin] Authenticating user token {{(pid=80377) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:03:46.926684 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe4bd0e3-b2b7-45cf-af06-fd5f7a87e66a None admin] Validating token access rules against request {{(pid=80377) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:03:46.970837 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG dbcounter [-] [80377] Writing DB stats keystone:SELECT=100 {{(pid=80377) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 10:03:46.980687 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80f49cc1-9bfb-4251-8643-0c0cf54b1534 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '891005fa5ff74041b75747d16cc66798', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80377) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:03:46.981386 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80f49cc1-9bfb-4251-8643-0c0cf54b1534 None admin] REQUEST_METHOD: `GET` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:03:46.981669 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80f49cc1-9bfb-4251-8643-0c0cf54b1534 None admin] SCRIPT_NAME: `/identity` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:03:46.981981 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80f49cc1-9bfb-4251-8643-0c0cf54b1534 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:03:46.982797 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80f49cc1-9bfb-4251-8643-0c0cf54b1534 None admin] RBAC: Adding query filter params (role.id=f57a652471c8414597c5fdde3d6f4636, scope.project.id=f1b38ebb804c41c69772c998aae72c41, user.id=88af518196674fe593ad88d30111fb7d) {{(pid=80377) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 02 10:03:46.983108 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80f49cc1-9bfb-4251-8643-0c0cf54b1534 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:03:46.984563 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80f49cc1-9bfb-4251-8643-0c0cf54b1534 None admin] RBAC: Authorization granted {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:03:46.994698 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: [pid: 80377|app: 0|req: 562/1106] 10.222.0.82 () {56 vars in 1374 bytes} [Tue Aug 2 10:03:46 2022] GET /identity/v3/role_assignments?user.id=88af518196674fe593ad88d30111fb7d&scope.project.id=f1b38ebb804c41c69772c998aae72c41&role.id=f57a652471c8414597c5fdde3d6f4636 => generated 608 bytes in 228 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 02 10:03:48.107405 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45af98ef-994f-4bea-9967-e42aead77f57 None None] REQUEST_METHOD: `GET` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:03:48.107912 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45af98ef-994f-4bea-9967-e42aead77f57 None None] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:03:48.107912 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45af98ef-994f-4bea-9967-e42aead77f57 None None] PATH_INFO: `/` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:03:48.109306 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 545/1107] 10.222.0.82 () {54 vars in 926 bytes} [Tue Aug 2 10:03:48 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 02 10:03:48.118963 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82b38fd6-32ef-4776-8efc-6655ce50e1be None None] REQUEST_METHOD: `POST` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:03:48.119912 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82b38fd6-32ef-4776-8efc-6655ce50e1be None None] SCRIPT_NAME: `/identity` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:03:48.119912 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82b38fd6-32ef-4776-8efc-6655ce50e1be None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:03:48.146580 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.auth.core [None req-82b38fd6-32ef-4776-8efc-6655ce50e1be None None] MFA Rules not processed for user `891005fa5ff74041b75747d16cc66798`. Rule list: `[]` (Enabled: `True`). {{(pid=80377) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 02 10:03:48.147968 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.fernet_utils [None req-82b38fd6-32ef-4776-8efc-6655ce50e1be None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80377) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 10:03:48.232893 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: [pid: 80377|app: 0|req: 563/1108] 10.222.0.82 () {58 vars in 1023 bytes} [Tue Aug 2 10:03:48 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 115 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 02 10:03:48.258786 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45af98ef-994f-4bea-9967-e42aead77f57 None None] Authenticating user token {{(pid=80378) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:03:48.386961 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45af98ef-994f-4bea-9967-e42aead77f57 None None] Validating token access rules against request {{(pid=80378) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:03:48.401999 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG dbcounter [-] [80378] Writing DB stats keystone:SELECT=100 {{(pid=80378) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 10:03:48.433313 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b613e937-32ce-4b95-b3d3-66a6c44aa2ed None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '891005fa5ff74041b75747d16cc66798', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80378) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:03:48.434000 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b613e937-32ce-4b95-b3d3-66a6c44aa2ed None admin] REQUEST_METHOD: `GET` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:03:48.434286 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b613e937-32ce-4b95-b3d3-66a6c44aa2ed None admin] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:03:48.434561 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b613e937-32ce-4b95-b3d3-66a6c44aa2ed None admin] PATH_INFO: `/v3/domains/default` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:03:48.435798 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b613e937-32ce-4b95-b3d3-66a6c44aa2ed None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:03:48.437277 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b613e937-32ce-4b95-b3d3-66a6c44aa2ed None admin] RBAC: Authorization granted {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:03:48.439007 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 546/1109] 10.222.0.82 () {56 vars in 1108 bytes} [Tue Aug 2 10:03:48 2022] GET /identity/v3/domains/default => generated 199 bytes in 181 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 02 10:03:48.445539 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82b38fd6-32ef-4776-8efc-6655ce50e1be None None] Authenticating user token {{(pid=80377) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:03:48.563849 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82b38fd6-32ef-4776-8efc-6655ce50e1be None None] Validating token access rules against request {{(pid=80377) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:03:48.593774 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG dbcounter [-] [80377] Writing DB stats keystone:SELECT=100 {{(pid=80377) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 10:03:48.604028 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1196d8f9-99bd-45f4-9032-24009d3405a6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '891005fa5ff74041b75747d16cc66798', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80377) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:03:48.604660 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1196d8f9-99bd-45f4-9032-24009d3405a6 None admin] REQUEST_METHOD: `POST` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:03:48.604942 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1196d8f9-99bd-45f4-9032-24009d3405a6 None admin] SCRIPT_NAME: `/identity` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:03:48.605220 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1196d8f9-99bd-45f4-9032-24009d3405a6 None admin] PATH_INFO: `/v3/users` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:03:48.605958 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1196d8f9-99bd-45f4-9032-24009d3405a6 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:03:48.607406 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1196d8f9-99bd-45f4-9032-24009d3405a6 None admin] RBAC: Authorization granted {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:03:48.631144 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: [pid: 80377|app: 0|req: 564/1110] 10.222.0.82 () {60 vars in 1132 bytes} [Tue Aug 2 10:03:48 2022] POST /identity/v3/users => generated 286 bytes in 186 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 02 10:03:50.778236 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15a9130a-db78-4270-bf7b-2d4da225745a None None] REQUEST_METHOD: `GET` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:03:50.778807 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15a9130a-db78-4270-bf7b-2d4da225745a None None] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:03:50.778807 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15a9130a-db78-4270-bf7b-2d4da225745a None None] PATH_INFO: `/` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:03:50.780132 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 547/1111] 10.222.0.82 () {54 vars in 926 bytes} [Tue Aug 2 10:03:50 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 02 10:03:50.785694 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-caa8e98f-8e96-4f22-affb-c8d4a73f7810 None None] REQUEST_METHOD: `POST` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:03:50.786608 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-caa8e98f-8e96-4f22-affb-c8d4a73f7810 None None] SCRIPT_NAME: `/identity` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:03:50.786608 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-caa8e98f-8e96-4f22-affb-c8d4a73f7810 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:03:50.805332 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.auth.core [None req-caa8e98f-8e96-4f22-affb-c8d4a73f7810 None None] MFA Rules not processed for user `891005fa5ff74041b75747d16cc66798`. Rule list: `[]` (Enabled: `True`). {{(pid=80377) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 02 10:03:50.806578 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.fernet_utils [None req-caa8e98f-8e96-4f22-affb-c8d4a73f7810 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80377) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 10:03:50.881382 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: [pid: 80377|app: 0|req: 565/1112] 10.222.0.82 () {58 vars in 1023 bytes} [Tue Aug 2 10:03:50 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 02 10:03:50.905435 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15a9130a-db78-4270-bf7b-2d4da225745a None None] Authenticating user token {{(pid=80378) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:03:51.028610 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15a9130a-db78-4270-bf7b-2d4da225745a None None] Validating token access rules against request {{(pid=80378) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:03:51.068073 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01cb93b2-12f5-4958-b812-110ebab4a10b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '891005fa5ff74041b75747d16cc66798', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80378) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:03:51.068720 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01cb93b2-12f5-4958-b812-110ebab4a10b None admin] REQUEST_METHOD: `GET` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:03:51.068998 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01cb93b2-12f5-4958-b812-110ebab4a10b None admin] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:03:51.069272 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01cb93b2-12f5-4958-b812-110ebab4a10b None admin] PATH_INFO: `/v3/roles/649993d933d44c50b268df7fdf336fbf` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:03:51.070735 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01cb93b2-12f5-4958-b812-110ebab4a10b None admin] RBAC: Authorizing `identity:get_role(role_id=649993d933d44c50b268df7fdf336fbf)` {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:03:51.072113 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01cb93b2-12f5-4958-b812-110ebab4a10b None admin] RBAC: Authorization granted {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:03:51.073484 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 548/1113] 10.222.0.82 () {56 vars in 1177 bytes} [Tue Aug 2 10:03:50 2022] GET /identity/v3/roles/649993d933d44c50b268df7fdf336fbf => generated 223 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 02 10:03:51.079564 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-caa8e98f-8e96-4f22-affb-c8d4a73f7810 None None] Authenticating user token {{(pid=80377) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:03:51.192093 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-caa8e98f-8e96-4f22-affb-c8d4a73f7810 None None] Validating token access rules against request {{(pid=80377) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:03:51.207239 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG dbcounter [-] [80377] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80377) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 10:03:51.233245 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e684b9b-4ddf-4bcd-b321-34c788551492 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '891005fa5ff74041b75747d16cc66798', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80377) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:03:51.233941 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e684b9b-4ddf-4bcd-b321-34c788551492 None admin] REQUEST_METHOD: `GET` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:03:51.234225 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e684b9b-4ddf-4bcd-b321-34c788551492 None admin] SCRIPT_NAME: `/identity` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:03:51.234504 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e684b9b-4ddf-4bcd-b321-34c788551492 None admin] PATH_INFO: `/v3/users/713e09bc2e29425f9683fdb152a21a51` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:03:51.256724 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e684b9b-4ddf-4bcd-b321-34c788551492 None admin] RBAC: Authorizing `identity:get_user(user_id=713e09bc2e29425f9683fdb152a21a51)` {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:03:51.260511 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e684b9b-4ddf-4bcd-b321-34c788551492 None admin] RBAC: Authorization granted {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:03:51.265257 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: [pid: 80377|app: 0|req: 566/1114] 10.222.0.82 () {56 vars in 1177 bytes} [Tue Aug 2 10:03:51 2022] GET /identity/v3/users/713e09bc2e29425f9683fdb152a21a51 => generated 286 bytes in 186 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 02 10:03:51.271562 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01cb93b2-12f5-4958-b812-110ebab4a10b None admin] Authenticating user token {{(pid=80378) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:03:51.320847 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG dbcounter [-] [80378] Writing DB stats keystone:SELECT=100 {{(pid=80378) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 10:03:51.387818 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01cb93b2-12f5-4958-b812-110ebab4a10b None admin] Validating token access rules against request {{(pid=80378) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:03:51.429648 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4fcf407-72ca-4020-9772-c9da679edcd2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '891005fa5ff74041b75747d16cc66798', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80378) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:03:51.430321 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4fcf407-72ca-4020-9772-c9da679edcd2 None admin] REQUEST_METHOD: `GET` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:03:51.430669 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4fcf407-72ca-4020-9772-c9da679edcd2 None admin] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:03:51.430948 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4fcf407-72ca-4020-9772-c9da679edcd2 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:03:51.516700 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4fcf407-72ca-4020-9772-c9da679edcd2 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:03:51.518795 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4fcf407-72ca-4020-9772-c9da679edcd2 None admin] RBAC: Authorization granted {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:03:51.571021 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG dbcounter [-] [80378] Writing DB stats keystone:SELECT=100 {{(pid=80378) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 10:03:51.633448 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 549/1115] 10.222.0.82 () {58 vars in 1282 bytes} [Tue Aug 2 10:03:51 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 362 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Aug 02 10:03:51.639600 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e684b9b-4ddf-4bcd-b321-34c788551492 None admin] Authenticating user token {{(pid=80377) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:03:51.757055 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e684b9b-4ddf-4bcd-b321-34c788551492 None admin] Validating token access rules against request {{(pid=80377) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:03:51.797382 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d285d6ca-ae1e-448a-91cb-a98353d979ef None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '891005fa5ff74041b75747d16cc66798', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80377) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:03:51.798118 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d285d6ca-ae1e-448a-91cb-a98353d979ef None admin] REQUEST_METHOD: `GET` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:03:51.798351 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d285d6ca-ae1e-448a-91cb-a98353d979ef None admin] SCRIPT_NAME: `/identity` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:03:51.798623 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d285d6ca-ae1e-448a-91cb-a98353d979ef None admin] PATH_INFO: `/v3/projects/edf242cc36af4db9b5d76d897711bb6c` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:03:51.799826 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d285d6ca-ae1e-448a-91cb-a98353d979ef None admin] RBAC: Authorizing `identity:get_project(project_id=edf242cc36af4db9b5d76d897711bb6c)` {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:03:51.801269 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d285d6ca-ae1e-448a-91cb-a98353d979ef None admin] RBAC: Authorization granted {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:03:51.805032 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: [pid: 80377|app: 0|req: 567/1116] 10.222.0.82 () {56 vars in 1186 bytes} [Tue Aug 2 10:03:51 2022] GET /identity/v3/projects/edf242cc36af4db9b5d76d897711bb6c => generated 311 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 02 10:03:51.813359 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4fcf407-72ca-4020-9772-c9da679edcd2 None admin] Authenticating user token {{(pid=80378) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:03:51.946947 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4fcf407-72ca-4020-9772-c9da679edcd2 None admin] Validating token access rules against request {{(pid=80378) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:03:52.000043 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-533e1824-b811-42dc-9c53-a6a805113ce0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '891005fa5ff74041b75747d16cc66798', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80378) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:03:52.000668 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-533e1824-b811-42dc-9c53-a6a805113ce0 None admin] REQUEST_METHOD: `GET` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:03:52.000949 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-533e1824-b811-42dc-9c53-a6a805113ce0 None admin] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:03:52.001221 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-533e1824-b811-42dc-9c53-a6a805113ce0 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:03:52.002043 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-533e1824-b811-42dc-9c53-a6a805113ce0 None admin] RBAC: Adding query filter params (role.id=649993d933d44c50b268df7fdf336fbf, scope.project.id=edf242cc36af4db9b5d76d897711bb6c, user.id=713e09bc2e29425f9683fdb152a21a51) {{(pid=80378) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 02 10:03:52.002352 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-533e1824-b811-42dc-9c53-a6a805113ce0 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:03:52.003782 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-533e1824-b811-42dc-9c53-a6a805113ce0 None admin] RBAC: Authorization granted {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:03:52.009861 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 550/1117] 10.222.0.82 () {56 vars in 1374 bytes} [Tue Aug 2 10:03:51 2022] GET /identity/v3/role_assignments?user.id=713e09bc2e29425f9683fdb152a21a51&scope.project.id=edf242cc36af4db9b5d76d897711bb6c&role.id=649993d933d44c50b268df7fdf336fbf => generated 259 bytes in 198 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 02 10:03:54.112939 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0358a73-31c0-4c24-b991-9a157ea4e3d8 None None] REQUEST_METHOD: `GET` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:03:54.113578 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0358a73-31c0-4c24-b991-9a157ea4e3d8 None None] SCRIPT_NAME: `/identity` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:03:54.113809 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0358a73-31c0-4c24-b991-9a157ea4e3d8 None None] PATH_INFO: `/` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:03:54.116414 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: [pid: 80377|app: 0|req: 568/1118] 10.222.0.82 () {54 vars in 926 bytes} [Tue Aug 2 10:03:54 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 02 10:03:54.123460 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5836628-4557-4736-9446-18317b8c62dc None None] REQUEST_METHOD: `POST` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:03:54.124279 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5836628-4557-4736-9446-18317b8c62dc None None] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:03:54.124279 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5836628-4557-4736-9446-18317b8c62dc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:03:54.138941 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG dbcounter [-] [80378] Writing DB stats keystone:SELECT=100 {{(pid=80378) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 10:03:54.145682 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.auth.core [None req-d5836628-4557-4736-9446-18317b8c62dc None None] MFA Rules not processed for user `891005fa5ff74041b75747d16cc66798`. Rule list: `[]` (Enabled: `True`). {{(pid=80378) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 02 10:03:54.147008 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.fernet_utils [None req-d5836628-4557-4736-9446-18317b8c62dc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80378) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 10:03:54.242421 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 551/1119] 10.222.0.82 () {58 vars in 1023 bytes} [Tue Aug 2 10:03:54 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 120 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Aug 02 10:03:54.268208 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0358a73-31c0-4c24-b991-9a157ea4e3d8 None None] Authenticating user token {{(pid=80377) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:03:54.328116 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG dbcounter [-] [80377] Writing DB stats keystone:SELECT=100 {{(pid=80377) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 10:03:54.430812 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0358a73-31c0-4c24-b991-9a157ea4e3d8 None None] Validating token access rules against request {{(pid=80377) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:03:54.474688 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f91f11bb-fbab-40a1-a2ee-35028f809576 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '891005fa5ff74041b75747d16cc66798', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80377) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:03:54.475387 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f91f11bb-fbab-40a1-a2ee-35028f809576 None admin] REQUEST_METHOD: `GET` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:03:54.475706 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f91f11bb-fbab-40a1-a2ee-35028f809576 None admin] SCRIPT_NAME: `/identity` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:03:54.476017 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f91f11bb-fbab-40a1-a2ee-35028f809576 None admin] PATH_INFO: `/v3/roles/649993d933d44c50b268df7fdf336fbf` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:03:54.477632 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f91f11bb-fbab-40a1-a2ee-35028f809576 None admin] RBAC: Authorizing `identity:get_role(role_id=649993d933d44c50b268df7fdf336fbf)` {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:03:54.479106 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f91f11bb-fbab-40a1-a2ee-35028f809576 None admin] RBAC: Authorization granted {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:03:54.480599 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: [pid: 80377|app: 0|req: 569/1120] 10.222.0.82 () {56 vars in 1177 bytes} [Tue Aug 2 10:03:54 2022] GET /identity/v3/roles/649993d933d44c50b268df7fdf336fbf => generated 223 bytes in 213 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 02 10:03:54.488526 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5836628-4557-4736-9446-18317b8c62dc None None] Authenticating user token {{(pid=80378) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:03:54.642950 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5836628-4557-4736-9446-18317b8c62dc None None] Validating token access rules against request {{(pid=80378) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:03:54.688114 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-366d1a65-3b7d-41f5-8674-288f2bf62343 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '891005fa5ff74041b75747d16cc66798', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80378) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:03:54.688684 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-366d1a65-3b7d-41f5-8674-288f2bf62343 None admin] REQUEST_METHOD: `GET` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:03:54.688968 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-366d1a65-3b7d-41f5-8674-288f2bf62343 None admin] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:03:54.689249 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-366d1a65-3b7d-41f5-8674-288f2bf62343 None admin] PATH_INFO: `/v3/users/713e09bc2e29425f9683fdb152a21a51` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:03:54.690521 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-366d1a65-3b7d-41f5-8674-288f2bf62343 None admin] RBAC: Authorizing `identity:get_user(user_id=713e09bc2e29425f9683fdb152a21a51)` {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:03:54.691970 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-366d1a65-3b7d-41f5-8674-288f2bf62343 None admin] RBAC: Authorization granted {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:03:54.693615 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 552/1121] 10.222.0.82 () {56 vars in 1177 bytes} [Tue Aug 2 10:03:54 2022] GET /identity/v3/users/713e09bc2e29425f9683fdb152a21a51 => generated 286 bytes in 207 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 02 10:03:54.699505 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f91f11bb-fbab-40a1-a2ee-35028f809576 None admin] Authenticating user token {{(pid=80377) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:03:54.835843 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f91f11bb-fbab-40a1-a2ee-35028f809576 None admin] Validating token access rules against request {{(pid=80377) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:03:54.844327 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG dbcounter [-] [80377] Writing DB stats keystone:SELECT=100 {{(pid=80377) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 10:03:54.882454 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-351369b2-1537-4ea5-b6b0-c95c1ec5ebc7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '891005fa5ff74041b75747d16cc66798', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80377) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:03:54.883235 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-351369b2-1537-4ea5-b6b0-c95c1ec5ebc7 None admin] REQUEST_METHOD: `GET` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:03:54.883623 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-351369b2-1537-4ea5-b6b0-c95c1ec5ebc7 None admin] SCRIPT_NAME: `/identity` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:03:54.884003 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-351369b2-1537-4ea5-b6b0-c95c1ec5ebc7 None admin] PATH_INFO: `/v3/projects/edf242cc36af4db9b5d76d897711bb6c` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:03:54.885324 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-351369b2-1537-4ea5-b6b0-c95c1ec5ebc7 None admin] RBAC: Authorizing `identity:get_project(project_id=edf242cc36af4db9b5d76d897711bb6c)` {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:03:54.886930 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-351369b2-1537-4ea5-b6b0-c95c1ec5ebc7 None admin] RBAC: Authorization granted {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:03:54.888626 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: [pid: 80377|app: 0|req: 570/1122] 10.222.0.82 () {56 vars in 1186 bytes} [Tue Aug 2 10:03:54 2022] GET /identity/v3/projects/edf242cc36af4db9b5d76d897711bb6c => generated 311 bytes in 190 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 02 10:03:54.896237 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-366d1a65-3b7d-41f5-8674-288f2bf62343 None admin] Authenticating user token {{(pid=80378) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:03:54.902237 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG dbcounter [-] [80378] Writing DB stats keystone:SELECT=100 {{(pid=80378) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 10:03:55.012269 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-366d1a65-3b7d-41f5-8674-288f2bf62343 None admin] Validating token access rules against request {{(pid=80378) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:03:55.053867 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1e3cd77-5e57-4c6c-84d9-debe39a031b3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '891005fa5ff74041b75747d16cc66798', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80378) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:03:55.054556 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1e3cd77-5e57-4c6c-84d9-debe39a031b3 None admin] REQUEST_METHOD: `PUT` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:03:55.054839 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1e3cd77-5e57-4c6c-84d9-debe39a031b3 None admin] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:03:55.055115 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1e3cd77-5e57-4c6c-84d9-debe39a031b3 None admin] PATH_INFO: `/v3/projects/edf242cc36af4db9b5d76d897711bb6c/users/713e09bc2e29425f9683fdb152a21a51/roles/649993d933d44c50b268df7fdf336fbf` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:03:55.057435 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1e3cd77-5e57-4c6c-84d9-debe39a031b3 None admin] RBAC: Authorizing `identity:create_grant(project_id=edf242cc36af4db9b5d76d897711bb6c, user_id=713e09bc2e29425f9683fdb152a21a51, role_id=649993d933d44c50b268df7fdf336fbf)` {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:03:55.059850 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1e3cd77-5e57-4c6c-84d9-debe39a031b3 None admin] RBAC: Authorization granted {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:03:55.076222 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 553/1123] 10.222.0.82 () {58 vars in 1439 bytes} [Tue Aug 2 10:03:54 2022] PUT /identity/v3/projects/edf242cc36af4db9b5d76d897711bb6c/users/713e09bc2e29425f9683fdb152a21a51/roles/649993d933d44c50b268df7fdf336fbf => generated 0 bytes in 181 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 02 10:03:57.167931 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a09260b3-8575-4d7b-89e3-28d79e7bb639 None None] REQUEST_METHOD: `GET` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:03:57.168446 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a09260b3-8575-4d7b-89e3-28d79e7bb639 None None] SCRIPT_NAME: `/identity` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:03:57.168446 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a09260b3-8575-4d7b-89e3-28d79e7bb639 None None] PATH_INFO: `/` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:03:57.169740 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: [pid: 80377|app: 0|req: 571/1124] 10.222.0.82 () {54 vars in 926 bytes} [Tue Aug 2 10:03:57 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 02 10:03:57.175782 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22128f63-e91c-467b-81ee-0f30a81dff55 None None] REQUEST_METHOD: `POST` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:03:57.176324 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22128f63-e91c-467b-81ee-0f30a81dff55 None None] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:03:57.176324 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22128f63-e91c-467b-81ee-0f30a81dff55 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:03:57.195669 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.auth.core [None req-22128f63-e91c-467b-81ee-0f30a81dff55 None None] MFA Rules not processed for user `891005fa5ff74041b75747d16cc66798`. Rule list: `[]` (Enabled: `True`). {{(pid=80378) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 02 10:03:57.196896 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.fernet_utils [None req-22128f63-e91c-467b-81ee-0f30a81dff55 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80378) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 10:03:57.269314 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG dbcounter [-] [80378] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80378) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 10:03:57.280414 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 554/1125] 10.222.0.82 () {58 vars in 1023 bytes} [Tue Aug 2 10:03:57 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 106 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 02 10:03:57.305919 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a09260b3-8575-4d7b-89e3-28d79e7bb639 None None] Authenticating user token {{(pid=80377) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:03:57.426429 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a09260b3-8575-4d7b-89e3-28d79e7bb639 None None] Validating token access rules against request {{(pid=80377) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:03:57.467912 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0a95679-511d-42d9-9353-c556ee979d82 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '891005fa5ff74041b75747d16cc66798', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80377) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:03:57.468574 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0a95679-511d-42d9-9353-c556ee979d82 None admin] REQUEST_METHOD: `GET` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:03:57.468858 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0a95679-511d-42d9-9353-c556ee979d82 None admin] SCRIPT_NAME: `/identity` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:03:57.469136 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0a95679-511d-42d9-9353-c556ee979d82 None admin] PATH_INFO: `/v3/roles/649993d933d44c50b268df7fdf336fbf` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:03:57.470659 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0a95679-511d-42d9-9353-c556ee979d82 None admin] RBAC: Authorizing `identity:get_role(role_id=649993d933d44c50b268df7fdf336fbf)` {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:03:57.472084 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0a95679-511d-42d9-9353-c556ee979d82 None admin] RBAC: Authorization granted {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:03:57.473579 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: [pid: 80377|app: 0|req: 572/1126] 10.222.0.82 () {56 vars in 1177 bytes} [Tue Aug 2 10:03:57 2022] GET /identity/v3/roles/649993d933d44c50b268df7fdf336fbf => generated 223 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 02 10:03:57.479575 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22128f63-e91c-467b-81ee-0f30a81dff55 None None] Authenticating user token {{(pid=80378) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:03:57.605991 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22128f63-e91c-467b-81ee-0f30a81dff55 None None] Validating token access rules against request {{(pid=80378) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:03:57.653063 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1869c101-3ff4-40b7-b0f6-895ccc7ea3d6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '891005fa5ff74041b75747d16cc66798', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80378) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:03:57.653713 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1869c101-3ff4-40b7-b0f6-895ccc7ea3d6 None admin] REQUEST_METHOD: `GET` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:03:57.654028 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1869c101-3ff4-40b7-b0f6-895ccc7ea3d6 None admin] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:03:57.654302 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1869c101-3ff4-40b7-b0f6-895ccc7ea3d6 None admin] PATH_INFO: `/v3/users/713e09bc2e29425f9683fdb152a21a51` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:03:57.655541 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1869c101-3ff4-40b7-b0f6-895ccc7ea3d6 None admin] RBAC: Authorizing `identity:get_user(user_id=713e09bc2e29425f9683fdb152a21a51)` {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:03:57.656988 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1869c101-3ff4-40b7-b0f6-895ccc7ea3d6 None admin] RBAC: Authorization granted {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:03:57.659289 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 555/1127] 10.222.0.82 () {56 vars in 1177 bytes} [Tue Aug 2 10:03:57 2022] GET /identity/v3/users/713e09bc2e29425f9683fdb152a21a51 => generated 286 bytes in 180 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 02 10:03:57.666784 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0a95679-511d-42d9-9353-c556ee979d82 None admin] Authenticating user token {{(pid=80377) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:03:57.729654 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG dbcounter [-] [80377] Writing DB stats keystone:SELECT=100 {{(pid=80377) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 10:03:57.810374 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0a95679-511d-42d9-9353-c556ee979d82 None admin] Validating token access rules against request {{(pid=80377) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:03:57.854085 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0fac897-e965-4272-bc7c-de1dd3c74f25 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '891005fa5ff74041b75747d16cc66798', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80377) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:03:57.854778 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0fac897-e965-4272-bc7c-de1dd3c74f25 None admin] REQUEST_METHOD: `GET` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:03:57.855099 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0fac897-e965-4272-bc7c-de1dd3c74f25 None admin] SCRIPT_NAME: `/identity` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:03:57.855416 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0fac897-e965-4272-bc7c-de1dd3c74f25 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:03:57.941404 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0fac897-e965-4272-bc7c-de1dd3c74f25 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:03:57.942915 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0fac897-e965-4272-bc7c-de1dd3c74f25 None admin] RBAC: Authorization granted {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:03:57.993904 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG dbcounter [-] [80377] Writing DB stats keystone:SELECT=100 {{(pid=80377) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 10:03:58.068265 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: [pid: 80377|app: 0|req: 573/1128] 10.222.0.82 () {58 vars in 1282 bytes} [Tue Aug 2 10:03:57 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 402 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Aug 02 10:03:58.074923 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1869c101-3ff4-40b7-b0f6-895ccc7ea3d6 None admin] Authenticating user token {{(pid=80378) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:03:58.149764 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG dbcounter [-] [80378] Writing DB stats keystone:SELECT=100 {{(pid=80378) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 10:03:58.199119 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1869c101-3ff4-40b7-b0f6-895ccc7ea3d6 None admin] Validating token access rules against request {{(pid=80378) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:03:58.242451 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ded02da-f67e-42a8-9c35-ec54036e355a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '891005fa5ff74041b75747d16cc66798', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80378) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:03:58.243114 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ded02da-f67e-42a8-9c35-ec54036e355a None admin] REQUEST_METHOD: `GET` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:03:58.243398 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ded02da-f67e-42a8-9c35-ec54036e355a None admin] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:03:58.243673 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ded02da-f67e-42a8-9c35-ec54036e355a None admin] PATH_INFO: `/v3/projects/edf242cc36af4db9b5d76d897711bb6c` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:03:58.244964 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ded02da-f67e-42a8-9c35-ec54036e355a None admin] RBAC: Authorizing `identity:get_project(project_id=edf242cc36af4db9b5d76d897711bb6c)` {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:03:58.246410 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ded02da-f67e-42a8-9c35-ec54036e355a None admin] RBAC: Authorization granted {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:03:58.248227 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 556/1129] 10.222.0.82 () {56 vars in 1186 bytes} [Tue Aug 2 10:03:58 2022] GET /identity/v3/projects/edf242cc36af4db9b5d76d897711bb6c => generated 311 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 02 10:03:58.255324 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0fac897-e965-4272-bc7c-de1dd3c74f25 None admin] Authenticating user token {{(pid=80377) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:03:58.386313 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0fac897-e965-4272-bc7c-de1dd3c74f25 None admin] Validating token access rules against request {{(pid=80377) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:03:58.434020 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-683dbd01-74ec-4838-808d-e235ca78fef3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '891005fa5ff74041b75747d16cc66798', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80377) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:03:58.435071 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-683dbd01-74ec-4838-808d-e235ca78fef3 None admin] REQUEST_METHOD: `GET` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:03:58.435565 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-683dbd01-74ec-4838-808d-e235ca78fef3 None admin] SCRIPT_NAME: `/identity` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:03:58.436050 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-683dbd01-74ec-4838-808d-e235ca78fef3 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:03:58.437489 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-683dbd01-74ec-4838-808d-e235ca78fef3 None admin] RBAC: Adding query filter params (role.id=649993d933d44c50b268df7fdf336fbf, scope.project.id=edf242cc36af4db9b5d76d897711bb6c, user.id=713e09bc2e29425f9683fdb152a21a51) {{(pid=80377) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 02 10:03:58.438067 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-683dbd01-74ec-4838-808d-e235ca78fef3 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:03:58.440095 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-683dbd01-74ec-4838-808d-e235ca78fef3 None admin] RBAC: Authorization granted {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:03:58.450995 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: [pid: 80377|app: 0|req: 574/1130] 10.222.0.82 () {56 vars in 1374 bytes} [Tue Aug 2 10:03:58 2022] GET /identity/v3/role_assignments?user.id=713e09bc2e29425f9683fdb152a21a51&scope.project.id=edf242cc36af4db9b5d76d897711bb6c&role.id=649993d933d44c50b268df7fdf336fbf => generated 608 bytes in 196 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 02 10:04:00.600400 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5cf91fa-25a3-4c38-b248-51a59ce04f34 None None] REQUEST_METHOD: `GET` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:04:00.600947 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5cf91fa-25a3-4c38-b248-51a59ce04f34 None None] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:04:00.600947 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5cf91fa-25a3-4c38-b248-51a59ce04f34 None None] PATH_INFO: `/` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:04:00.602388 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 557/1131] 10.222.0.82 () {54 vars in 926 bytes} [Tue Aug 2 10:04:00 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 02 10:04:00.607750 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4515b57-5adb-4cdf-ada0-b6e43375b987 None None] REQUEST_METHOD: `POST` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:04:00.608196 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4515b57-5adb-4cdf-ada0-b6e43375b987 None None] SCRIPT_NAME: `/identity` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:04:00.608307 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4515b57-5adb-4cdf-ada0-b6e43375b987 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:04:00.622045 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG dbcounter [-] [80377] Writing DB stats keystone:SELECT=100 {{(pid=80377) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 10:04:00.627352 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.auth.core [None req-c4515b57-5adb-4cdf-ada0-b6e43375b987 None None] MFA Rules not processed for user `891005fa5ff74041b75747d16cc66798`. Rule list: `[]` (Enabled: `True`). {{(pid=80377) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 02 10:04:00.628559 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.fernet_utils [None req-c4515b57-5adb-4cdf-ada0-b6e43375b987 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80377) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 10:04:00.697568 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: [pid: 80377|app: 0|req: 575/1132] 10.222.0.82 () {58 vars in 1023 bytes} [Tue Aug 2 10:04:00 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 91 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 02 10:04:00.720447 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5cf91fa-25a3-4c38-b248-51a59ce04f34 None None] Authenticating user token {{(pid=80378) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:04:00.873758 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5cf91fa-25a3-4c38-b248-51a59ce04f34 None None] Validating token access rules against request {{(pid=80378) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:04:00.910287 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG dbcounter [-] [80378] Writing DB stats keystone:SELECT=100 {{(pid=80378) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 10:04:00.917956 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-839355ed-c66b-4e24-b04f-9644469480fa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '891005fa5ff74041b75747d16cc66798', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80378) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:04:00.918653 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-839355ed-c66b-4e24-b04f-9644469480fa None admin] REQUEST_METHOD: `GET` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:04:00.918935 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-839355ed-c66b-4e24-b04f-9644469480fa None admin] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:04:00.919215 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-839355ed-c66b-4e24-b04f-9644469480fa None admin] PATH_INFO: `/v3/domains/default` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:04:00.920558 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-839355ed-c66b-4e24-b04f-9644469480fa None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:04:00.922077 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-839355ed-c66b-4e24-b04f-9644469480fa None admin] RBAC: Authorization granted {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:04:00.923882 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 558/1133] 10.222.0.82 () {56 vars in 1108 bytes} [Tue Aug 2 10:04:00 2022] GET /identity/v3/domains/default => generated 199 bytes in 204 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 02 10:04:00.930883 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4515b57-5adb-4cdf-ada0-b6e43375b987 None None] Authenticating user token {{(pid=80377) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:04:01.052378 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4515b57-5adb-4cdf-ada0-b6e43375b987 None None] Validating token access rules against request {{(pid=80377) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:04:01.097946 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6272d829-a1e6-422d-bf64-678a51ed9f2e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '891005fa5ff74041b75747d16cc66798', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80377) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:04:01.098600 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6272d829-a1e6-422d-bf64-678a51ed9f2e None admin] REQUEST_METHOD: `POST` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:04:01.098885 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6272d829-a1e6-422d-bf64-678a51ed9f2e None admin] SCRIPT_NAME: `/identity` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:04:01.099167 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6272d829-a1e6-422d-bf64-678a51ed9f2e None admin] PATH_INFO: `/v3/projects` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:04:01.099879 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6272d829-a1e6-422d-bf64-678a51ed9f2e None admin] RBAC: Authorizing `identity:create_project()` {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:04:01.101524 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6272d829-a1e6-422d-bf64-678a51ed9f2e None admin] RBAC: Authorization granted {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:04:01.107502 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG dbcounter [-] [80377] Writing DB stats keystone:SELECT=100 {{(pid=80377) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 10:04:01.133539 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: [pid: 80377|app: 0|req: 576/1134] 10.222.0.82 () {60 vars in 1141 bytes} [Tue Aug 2 10:04:00 2022] POST /identity/v3/projects => generated 311 bytes in 203 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 02 10:04:03.339552 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f30609e-6ecb-441b-b196-e17aad011a38 None None] REQUEST_METHOD: `GET` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:04:03.340124 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f30609e-6ecb-441b-b196-e17aad011a38 None None] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:04:03.340124 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f30609e-6ecb-441b-b196-e17aad011a38 None None] PATH_INFO: `/` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:04:03.341462 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 559/1135] 10.222.0.82 () {54 vars in 926 bytes} [Tue Aug 2 10:04:03 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 02 10:04:03.348035 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d272ec6d-3c54-4d68-8258-691401151478 None None] REQUEST_METHOD: `POST` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:04:03.348546 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d272ec6d-3c54-4d68-8258-691401151478 None None] SCRIPT_NAME: `/identity` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:04:03.348698 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d272ec6d-3c54-4d68-8258-691401151478 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:04:03.371648 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.auth.core [None req-d272ec6d-3c54-4d68-8258-691401151478 None None] MFA Rules not processed for user `891005fa5ff74041b75747d16cc66798`. Rule list: `[]` (Enabled: `True`). {{(pid=80377) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 02 10:04:03.372856 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.fernet_utils [None req-d272ec6d-3c54-4d68-8258-691401151478 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80377) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 10:04:03.454047 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: [pid: 80377|app: 0|req: 577/1136] 10.222.0.82 () {58 vars in 1023 bytes} [Tue Aug 2 10:04:03 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 108 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 02 10:04:03.479020 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f30609e-6ecb-441b-b196-e17aad011a38 None None] Authenticating user token {{(pid=80378) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:04:03.590759 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f30609e-6ecb-441b-b196-e17aad011a38 None None] Validating token access rules against request {{(pid=80378) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:04:03.633382 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58d62a8e-f97a-4cd1-93d0-4afba167790c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '891005fa5ff74041b75747d16cc66798', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80378) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:04:03.634061 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58d62a8e-f97a-4cd1-93d0-4afba167790c None admin] REQUEST_METHOD: `GET` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:04:03.634346 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58d62a8e-f97a-4cd1-93d0-4afba167790c None admin] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:04:03.634613 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58d62a8e-f97a-4cd1-93d0-4afba167790c None admin] PATH_INFO: `/v3/domains/default` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:04:03.635823 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58d62a8e-f97a-4cd1-93d0-4afba167790c None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:04:03.637303 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58d62a8e-f97a-4cd1-93d0-4afba167790c None admin] RBAC: Authorization granted {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:04:03.639012 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 560/1137] 10.222.0.82 () {56 vars in 1108 bytes} [Tue Aug 2 10:04:03 2022] GET /identity/v3/domains/default => generated 199 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 02 10:04:03.645232 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d272ec6d-3c54-4d68-8258-691401151478 None None] Authenticating user token {{(pid=80377) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:04:03.775293 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d272ec6d-3c54-4d68-8258-691401151478 None None] Validating token access rules against request {{(pid=80377) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:04:03.798699 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG dbcounter [-] [80377] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80377) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 10:04:03.820989 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa722536-8dd7-4b4a-93a3-d7137adb8300 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '891005fa5ff74041b75747d16cc66798', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80377) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:04:03.821623 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa722536-8dd7-4b4a-93a3-d7137adb8300 None admin] REQUEST_METHOD: `POST` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:04:03.821934 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa722536-8dd7-4b4a-93a3-d7137adb8300 None admin] SCRIPT_NAME: `/identity` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:04:03.822206 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa722536-8dd7-4b4a-93a3-d7137adb8300 None admin] PATH_INFO: `/v3/users` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:04:03.822911 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa722536-8dd7-4b4a-93a3-d7137adb8300 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:04:03.824379 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa722536-8dd7-4b4a-93a3-d7137adb8300 None admin] RBAC: Authorization granted {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:04:03.850890 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: [pid: 80377|app: 0|req: 578/1138] 10.222.0.82 () {60 vars in 1132 bytes} [Tue Aug 2 10:04:03 2022] POST /identity/v3/users => generated 286 bytes in 206 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 02 10:04:06.006929 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37a50d98-8482-429a-bd80-af8d87468494 None None] REQUEST_METHOD: `GET` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:04:06.007501 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37a50d98-8482-429a-bd80-af8d87468494 None None] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:04:06.007501 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37a50d98-8482-429a-bd80-af8d87468494 None None] PATH_INFO: `/` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:04:06.008841 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 561/1139] 10.222.0.82 () {54 vars in 926 bytes} [Tue Aug 2 10:04:06 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 02 10:04:06.015875 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb5322d7-6ff4-4454-83d3-f17263c4b091 None None] REQUEST_METHOD: `POST` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:04:06.016390 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb5322d7-6ff4-4454-83d3-f17263c4b091 None None] SCRIPT_NAME: `/identity` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:04:06.016496 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb5322d7-6ff4-4454-83d3-f17263c4b091 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:04:06.037221 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.auth.core [None req-cb5322d7-6ff4-4454-83d3-f17263c4b091 None None] MFA Rules not processed for user `891005fa5ff74041b75747d16cc66798`. Rule list: `[]` (Enabled: `True`). {{(pid=80377) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 02 10:04:06.038490 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.fernet_utils [None req-cb5322d7-6ff4-4454-83d3-f17263c4b091 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80377) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 10:04:06.116784 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: [pid: 80377|app: 0|req: 579/1140] 10.222.0.82 () {58 vars in 1023 bytes} [Tue Aug 2 10:04:06 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 02 10:04:06.140288 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37a50d98-8482-429a-bd80-af8d87468494 None None] Authenticating user token {{(pid=80378) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:04:06.221237 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG dbcounter [-] [80378] Writing DB stats keystone:SELECT=100 {{(pid=80378) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 10:04:06.262489 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37a50d98-8482-429a-bd80-af8d87468494 None None] Validating token access rules against request {{(pid=80378) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:04:06.299183 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba43d35a-3b32-4dda-a973-5a4dbc497618 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '891005fa5ff74041b75747d16cc66798', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80378) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:04:06.300351 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba43d35a-3b32-4dda-a973-5a4dbc497618 None admin] REQUEST_METHOD: `GET` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:04:06.300707 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba43d35a-3b32-4dda-a973-5a4dbc497618 None admin] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:04:06.300707 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba43d35a-3b32-4dda-a973-5a4dbc497618 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:04:06.311997 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba43d35a-3b32-4dda-a973-5a4dbc497618 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 02 10:04:06.317992 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application [None req-ba43d35a-3b32-4dda-a973-5a4dbc497618 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 02 10:04:06.317992 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 02 10:04:06.317992 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 02 10:04:06.317992 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 02 10:04:06.317992 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 02 10:04:06.317992 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 02 10:04:06.317992 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 02 10:04:06.317992 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 02 10:04:06.317992 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 02 10:04:06.317992 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 02 10:04:06.317992 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 02 10:04:06.317992 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 02 10:04:06.317992 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 02 10:04:06.317992 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 02 10:04:06.317992 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 02 10:04:06.317992 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 02 10:04:06.319654 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 02 10:04:06.319654 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 02 10:04:06.319654 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 02 10:04:06.319654 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 02 10:04:06.319654 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 02 10:04:06.319654 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 02 10:04:06.319654 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 02 10:04:06.319654 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 02 10:04:06.319654 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 02 10:04:06.319654 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application return self.get_or_create( Aug 02 10:04:06.319654 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 02 10:04:06.319654 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application with Lock( Aug 02 10:04:06.319654 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 02 10:04:06.319654 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application return self._enter() Aug 02 10:04:06.319654 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 02 10:04:06.319654 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 02 10:04:06.321714 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 02 10:04:06.321714 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application return self.creator() Aug 02 10:04:06.321714 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 02 10:04:06.321714 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application created_value = creator( Aug 02 10:04:06.321714 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 02 10:04:06.321714 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 02 10:04:06.321714 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 02 10:04:06.321714 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 02 10:04:06.321714 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 02 10:04:06.321714 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 02 10:04:06.321714 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 02 10:04:06.321714 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application Aug 02 10:04:06.321714 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 562/1141] 10.222.0.82 () {56 vars in 1096 bytes} [Tue Aug 2 10:04:06 2022] GET /identity/v3/roles/admin => generated 83 bytes in 180 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 02 10:04:06.327122 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb5322d7-6ff4-4454-83d3-f17263c4b091 None None] Authenticating user token {{(pid=80377) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:04:06.435386 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb5322d7-6ff4-4454-83d3-f17263c4b091 None None] Validating token access rules against request {{(pid=80377) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:04:06.437988 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG dbcounter [-] [80377] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80377) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 10:04:06.475723 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-806c611e-85e8-4072-b733-37f48e9f8ca3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '891005fa5ff74041b75747d16cc66798', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80377) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:04:06.476346 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-806c611e-85e8-4072-b733-37f48e9f8ca3 None admin] REQUEST_METHOD: `GET` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:04:06.476629 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-806c611e-85e8-4072-b733-37f48e9f8ca3 None admin] SCRIPT_NAME: `/identity` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:04:06.476902 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-806c611e-85e8-4072-b733-37f48e9f8ca3 None admin] PATH_INFO: `/v3/roles` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:04:06.477639 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-806c611e-85e8-4072-b733-37f48e9f8ca3 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80377) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 02 10:04:06.477974 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-806c611e-85e8-4072-b733-37f48e9f8ca3 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:04:06.479358 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-806c611e-85e8-4072-b733-37f48e9f8ca3 None admin] RBAC: Authorization granted {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:04:06.488780 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: [pid: 80377|app: 0|req: 580/1142] 10.222.0.82 () {56 vars in 1099 bytes} [Tue Aug 2 10:04:06 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 02 10:04:06.494930 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba43d35a-3b32-4dda-a973-5a4dbc497618 None admin] Authenticating user token {{(pid=80378) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:04:06.609068 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba43d35a-3b32-4dda-a973-5a4dbc497618 None admin] Validating token access rules against request {{(pid=80378) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:04:06.638216 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG dbcounter [-] [80378] Writing DB stats keystone:SELECT=100 {{(pid=80378) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 10:04:06.648160 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ff5c459-7ef3-4e3d-b70d-dba53759d395 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '891005fa5ff74041b75747d16cc66798', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80378) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:04:06.648822 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ff5c459-7ef3-4e3d-b70d-dba53759d395 None admin] REQUEST_METHOD: `GET` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:04:06.649105 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ff5c459-7ef3-4e3d-b70d-dba53759d395 None admin] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:04:06.649378 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ff5c459-7ef3-4e3d-b70d-dba53759d395 None admin] PATH_INFO: `/v3/users/070ec283a0664665bd63b48f3435b7b6` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:04:06.665581 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ff5c459-7ef3-4e3d-b70d-dba53759d395 None admin] RBAC: Authorizing `identity:get_user(user_id=070ec283a0664665bd63b48f3435b7b6)` {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:04:06.667094 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ff5c459-7ef3-4e3d-b70d-dba53759d395 None admin] RBAC: Authorization granted {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:04:06.668839 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 563/1143] 10.222.0.82 () {56 vars in 1177 bytes} [Tue Aug 2 10:04:06 2022] GET /identity/v3/users/070ec283a0664665bd63b48f3435b7b6 => generated 286 bytes in 175 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 02 10:04:06.675736 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-806c611e-85e8-4072-b733-37f48e9f8ca3 None admin] Authenticating user token {{(pid=80377) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:04:06.790052 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-806c611e-85e8-4072-b733-37f48e9f8ca3 None admin] Validating token access rules against request {{(pid=80377) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:04:06.826645 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8aa5a568-7b62-4199-bb1b-3b5fcedd190a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '891005fa5ff74041b75747d16cc66798', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80377) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:04:06.827276 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8aa5a568-7b62-4199-bb1b-3b5fcedd190a None admin] REQUEST_METHOD: `GET` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:04:06.827560 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8aa5a568-7b62-4199-bb1b-3b5fcedd190a None admin] SCRIPT_NAME: `/identity` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:04:06.827838 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8aa5a568-7b62-4199-bb1b-3b5fcedd190a None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:04:06.870635 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG dbcounter [-] [80377] Writing DB stats keystone:SELECT=100 {{(pid=80377) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 10:04:06.919015 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8aa5a568-7b62-4199-bb1b-3b5fcedd190a None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:04:06.920676 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8aa5a568-7b62-4199-bb1b-3b5fcedd190a None admin] RBAC: Authorization granted {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:04:07.034579 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: [pid: 80377|app: 0|req: 581/1144] 10.222.0.82 () {58 vars in 1282 bytes} [Tue Aug 2 10:04:06 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 360 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Aug 02 10:04:07.040861 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ff5c459-7ef3-4e3d-b70d-dba53759d395 None admin] Authenticating user token {{(pid=80378) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:04:07.153279 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ff5c459-7ef3-4e3d-b70d-dba53759d395 None admin] Validating token access rules against request {{(pid=80378) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:04:07.190177 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd6c005b-ff87-4b1c-ab98-1c7f4a4fc84e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '891005fa5ff74041b75747d16cc66798', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80378) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:04:07.190831 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd6c005b-ff87-4b1c-ab98-1c7f4a4fc84e None admin] REQUEST_METHOD: `GET` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:04:07.191119 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd6c005b-ff87-4b1c-ab98-1c7f4a4fc84e None admin] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:04:07.191396 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd6c005b-ff87-4b1c-ab98-1c7f4a4fc84e None admin] PATH_INFO: `/v3/projects/038d3b64e40d4eccbee14e23810d37a6` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:04:07.192574 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd6c005b-ff87-4b1c-ab98-1c7f4a4fc84e None admin] RBAC: Authorizing `identity:get_project(project_id=038d3b64e40d4eccbee14e23810d37a6)` {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:04:07.194056 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd6c005b-ff87-4b1c-ab98-1c7f4a4fc84e None admin] RBAC: Authorization granted {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:04:07.195748 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 564/1145] 10.222.0.82 () {56 vars in 1186 bytes} [Tue Aug 2 10:04:07 2022] GET /identity/v3/projects/038d3b64e40d4eccbee14e23810d37a6 => generated 311 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 02 10:04:07.202152 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8aa5a568-7b62-4199-bb1b-3b5fcedd190a None admin] Authenticating user token {{(pid=80377) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:04:07.279596 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG dbcounter [-] [80377] Writing DB stats keystone:SELECT=100 {{(pid=80377) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 10:04:07.320361 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8aa5a568-7b62-4199-bb1b-3b5fcedd190a None admin] Validating token access rules against request {{(pid=80377) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:04:07.366470 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4274e2b5-f814-44e7-baf0-71bcd811e4c6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '891005fa5ff74041b75747d16cc66798', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80377) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:04:07.367096 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4274e2b5-f814-44e7-baf0-71bcd811e4c6 None admin] REQUEST_METHOD: `GET` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:04:07.367379 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4274e2b5-f814-44e7-baf0-71bcd811e4c6 None admin] SCRIPT_NAME: `/identity` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:04:07.367653 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4274e2b5-f814-44e7-baf0-71bcd811e4c6 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:04:07.368445 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4274e2b5-f814-44e7-baf0-71bcd811e4c6 None admin] RBAC: Adding query filter params (role.id=237966442f05411191105bf336bc14f3, scope.project.id=038d3b64e40d4eccbee14e23810d37a6, user.id=070ec283a0664665bd63b48f3435b7b6) {{(pid=80377) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 02 10:04:07.368757 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4274e2b5-f814-44e7-baf0-71bcd811e4c6 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:04:07.370224 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4274e2b5-f814-44e7-baf0-71bcd811e4c6 None admin] RBAC: Authorization granted {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:04:07.376246 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: [pid: 80377|app: 0|req: 582/1146] 10.222.0.82 () {56 vars in 1374 bytes} [Tue Aug 2 10:04:07 2022] GET /identity/v3/role_assignments?user.id=070ec283a0664665bd63b48f3435b7b6&scope.project.id=038d3b64e40d4eccbee14e23810d37a6&role.id=237966442f05411191105bf336bc14f3 => generated 259 bytes in 175 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 02 10:04:09.484224 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-807ca238-def7-48e0-8dae-ea8abfa97325 None None] REQUEST_METHOD: `GET` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:04:09.484763 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-807ca238-def7-48e0-8dae-ea8abfa97325 None None] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:04:09.484763 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-807ca238-def7-48e0-8dae-ea8abfa97325 None None] PATH_INFO: `/` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:04:09.486165 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 565/1147] 10.222.0.82 () {54 vars in 926 bytes} [Tue Aug 2 10:04:09 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 02 10:04:09.491990 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f92e8c57-575c-4885-bd24-e81060778add None None] REQUEST_METHOD: `POST` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:04:09.492478 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f92e8c57-575c-4885-bd24-e81060778add None None] SCRIPT_NAME: `/identity` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:04:09.492609 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f92e8c57-575c-4885-bd24-e81060778add None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:04:09.519327 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.auth.core [None req-f92e8c57-575c-4885-bd24-e81060778add None None] MFA Rules not processed for user `891005fa5ff74041b75747d16cc66798`. Rule list: `[]` (Enabled: `True`). {{(pid=80377) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 02 10:04:09.522232 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.fernet_utils [None req-f92e8c57-575c-4885-bd24-e81060778add None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80377) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 10:04:09.590687 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: [pid: 80377|app: 0|req: 583/1148] 10.222.0.82 () {58 vars in 1023 bytes} [Tue Aug 2 10:04:09 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 02 10:04:09.613033 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-807ca238-def7-48e0-8dae-ea8abfa97325 None None] Authenticating user token {{(pid=80378) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:04:09.670027 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG dbcounter [-] [80378] Writing DB stats keystone:SELECT=100 {{(pid=80378) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 10:04:09.734758 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-807ca238-def7-48e0-8dae-ea8abfa97325 None None] Validating token access rules against request {{(pid=80378) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:04:09.778512 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5821e1dc-6999-495d-98b0-a7f6ba191579 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '891005fa5ff74041b75747d16cc66798', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80378) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:04:09.779166 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5821e1dc-6999-495d-98b0-a7f6ba191579 None admin] REQUEST_METHOD: `GET` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:04:09.779448 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5821e1dc-6999-495d-98b0-a7f6ba191579 None admin] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:04:09.779724 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5821e1dc-6999-495d-98b0-a7f6ba191579 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:04:09.791483 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5821e1dc-6999-495d-98b0-a7f6ba191579 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 02 10:04:09.796447 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application [None req-5821e1dc-6999-495d-98b0-a7f6ba191579 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 02 10:04:09.796447 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 02 10:04:09.796447 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 02 10:04:09.796447 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 02 10:04:09.796447 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 02 10:04:09.796447 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 02 10:04:09.796447 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 02 10:04:09.796447 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 02 10:04:09.796447 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 02 10:04:09.796447 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 02 10:04:09.796447 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 02 10:04:09.796447 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 02 10:04:09.796447 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 02 10:04:09.796447 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 02 10:04:09.796447 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 02 10:04:09.796447 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 02 10:04:09.797614 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 02 10:04:09.797614 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 02 10:04:09.797614 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 02 10:04:09.797614 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 02 10:04:09.797614 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 02 10:04:09.797614 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 02 10:04:09.797614 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 02 10:04:09.797614 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 02 10:04:09.797614 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 02 10:04:09.797614 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application return self.get_or_create( Aug 02 10:04:09.797614 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 02 10:04:09.797614 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application with Lock( Aug 02 10:04:09.797614 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 02 10:04:09.797614 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application return self._enter() Aug 02 10:04:09.797614 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 02 10:04:09.797614 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 02 10:04:09.798900 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 02 10:04:09.798900 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application return self.creator() Aug 02 10:04:09.798900 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 02 10:04:09.798900 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application created_value = creator( Aug 02 10:04:09.798900 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 02 10:04:09.798900 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 02 10:04:09.798900 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 02 10:04:09.798900 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 02 10:04:09.798900 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 02 10:04:09.798900 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 02 10:04:09.798900 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 02 10:04:09.798900 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application Aug 02 10:04:09.798900 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 566/1149] 10.222.0.82 () {56 vars in 1096 bytes} [Tue Aug 2 10:04:09 2022] GET /identity/v3/roles/admin => generated 83 bytes in 185 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 02 10:04:09.803785 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f92e8c57-575c-4885-bd24-e81060778add None None] Authenticating user token {{(pid=80377) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:04:09.872761 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG dbcounter [-] [80377] Writing DB stats keystone:SELECT=100 {{(pid=80377) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 10:04:09.921311 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f92e8c57-575c-4885-bd24-e81060778add None None] Validating token access rules against request {{(pid=80377) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:04:09.967818 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc3004cd-6402-43e6-802d-2d1a29d0e061 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '891005fa5ff74041b75747d16cc66798', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80377) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:04:09.968711 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc3004cd-6402-43e6-802d-2d1a29d0e061 None admin] REQUEST_METHOD: `GET` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:04:09.969035 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc3004cd-6402-43e6-802d-2d1a29d0e061 None admin] SCRIPT_NAME: `/identity` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:04:09.969314 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc3004cd-6402-43e6-802d-2d1a29d0e061 None admin] PATH_INFO: `/v3/roles` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:04:09.970091 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc3004cd-6402-43e6-802d-2d1a29d0e061 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80377) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 02 10:04:09.970399 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc3004cd-6402-43e6-802d-2d1a29d0e061 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:04:09.972514 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc3004cd-6402-43e6-802d-2d1a29d0e061 None admin] RBAC: Authorization granted {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:04:09.981323 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: [pid: 80377|app: 0|req: 584/1150] 10.222.0.82 () {56 vars in 1099 bytes} [Tue Aug 2 10:04:09 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 178 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 02 10:04:09.987512 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5821e1dc-6999-495d-98b0-a7f6ba191579 None admin] Authenticating user token {{(pid=80378) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:04:10.101992 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5821e1dc-6999-495d-98b0-a7f6ba191579 None admin] Validating token access rules against request {{(pid=80378) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:04:10.117542 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG dbcounter [-] [80378] Writing DB stats keystone:SELECT=100 {{(pid=80378) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 10:04:10.140524 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cea34d4e-8426-4f89-a22e-4ef4d8393f10 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '891005fa5ff74041b75747d16cc66798', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80378) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:04:10.141180 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cea34d4e-8426-4f89-a22e-4ef4d8393f10 None admin] REQUEST_METHOD: `GET` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:04:10.141459 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cea34d4e-8426-4f89-a22e-4ef4d8393f10 None admin] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:04:10.141733 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cea34d4e-8426-4f89-a22e-4ef4d8393f10 None admin] PATH_INFO: `/v3/users/070ec283a0664665bd63b48f3435b7b6` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:04:10.143130 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cea34d4e-8426-4f89-a22e-4ef4d8393f10 None admin] RBAC: Authorizing `identity:get_user(user_id=070ec283a0664665bd63b48f3435b7b6)` {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:04:10.145146 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cea34d4e-8426-4f89-a22e-4ef4d8393f10 None admin] RBAC: Authorization granted {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:04:10.146846 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 567/1151] 10.222.0.82 () {56 vars in 1177 bytes} [Tue Aug 2 10:04:09 2022] GET /identity/v3/users/070ec283a0664665bd63b48f3435b7b6 => generated 286 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 02 10:04:10.152773 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc3004cd-6402-43e6-802d-2d1a29d0e061 None admin] Authenticating user token {{(pid=80377) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:04:10.263106 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc3004cd-6402-43e6-802d-2d1a29d0e061 None admin] Validating token access rules against request {{(pid=80377) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:04:10.287565 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG dbcounter [-] [80377] Writing DB stats keystone:SELECT=100 {{(pid=80377) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 10:04:10.299432 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c89e7122-9439-421c-91fc-855bffe149e9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '891005fa5ff74041b75747d16cc66798', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80377) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:04:10.300639 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c89e7122-9439-421c-91fc-855bffe149e9 None admin] REQUEST_METHOD: `GET` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:04:10.300917 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c89e7122-9439-421c-91fc-855bffe149e9 None admin] SCRIPT_NAME: `/identity` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:04:10.301187 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c89e7122-9439-421c-91fc-855bffe149e9 None admin] PATH_INFO: `/v3/projects/038d3b64e40d4eccbee14e23810d37a6` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:04:10.302727 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c89e7122-9439-421c-91fc-855bffe149e9 None admin] RBAC: Authorizing `identity:get_project(project_id=038d3b64e40d4eccbee14e23810d37a6)` {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:04:10.304392 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c89e7122-9439-421c-91fc-855bffe149e9 None admin] RBAC: Authorization granted {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:04:10.306301 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: [pid: 80377|app: 0|req: 585/1152] 10.222.0.82 () {56 vars in 1186 bytes} [Tue Aug 2 10:04:10 2022] GET /identity/v3/projects/038d3b64e40d4eccbee14e23810d37a6 => generated 311 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 02 10:04:10.313517 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cea34d4e-8426-4f89-a22e-4ef4d8393f10 None admin] Authenticating user token {{(pid=80378) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:04:10.419159 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cea34d4e-8426-4f89-a22e-4ef4d8393f10 None admin] Validating token access rules against request {{(pid=80378) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:04:10.459312 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65368b50-2d70-472e-a336-fcbad432f5ff None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '891005fa5ff74041b75747d16cc66798', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80378) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:04:10.459987 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65368b50-2d70-472e-a336-fcbad432f5ff None admin] REQUEST_METHOD: `PUT` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:04:10.460269 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65368b50-2d70-472e-a336-fcbad432f5ff None admin] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:04:10.460544 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65368b50-2d70-472e-a336-fcbad432f5ff None admin] PATH_INFO: `/v3/projects/038d3b64e40d4eccbee14e23810d37a6/users/070ec283a0664665bd63b48f3435b7b6/roles/237966442f05411191105bf336bc14f3` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:04:10.462523 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65368b50-2d70-472e-a336-fcbad432f5ff None admin] RBAC: Authorizing `identity:create_grant(project_id=038d3b64e40d4eccbee14e23810d37a6, user_id=070ec283a0664665bd63b48f3435b7b6, role_id=237966442f05411191105bf336bc14f3)` {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:04:10.463970 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65368b50-2d70-472e-a336-fcbad432f5ff None admin] RBAC: Authorization granted {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:04:10.476934 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 568/1153] 10.222.0.82 () {58 vars in 1439 bytes} [Tue Aug 2 10:04:10 2022] PUT /identity/v3/projects/038d3b64e40d4eccbee14e23810d37a6/users/070ec283a0664665bd63b48f3435b7b6/roles/237966442f05411191105bf336bc14f3 => generated 0 bytes in 164 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 02 10:04:12.544517 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1321a4a0-c3f7-443f-84fd-979416940fa2 None None] REQUEST_METHOD: `GET` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:04:12.545119 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1321a4a0-c3f7-443f-84fd-979416940fa2 None None] SCRIPT_NAME: `/identity` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:04:12.545119 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1321a4a0-c3f7-443f-84fd-979416940fa2 None None] PATH_INFO: `/` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:04:12.546448 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: [pid: 80377|app: 0|req: 586/1154] 10.222.0.82 () {54 vars in 926 bytes} [Tue Aug 2 10:04:12 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 02 10:04:12.553231 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7be0520c-1697-43e3-bb41-85b49b2e4bb2 None None] REQUEST_METHOD: `POST` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:04:12.553752 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7be0520c-1697-43e3-bb41-85b49b2e4bb2 None None] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:04:12.554292 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7be0520c-1697-43e3-bb41-85b49b2e4bb2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:04:12.575041 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.auth.core [None req-7be0520c-1697-43e3-bb41-85b49b2e4bb2 None None] MFA Rules not processed for user `891005fa5ff74041b75747d16cc66798`. Rule list: `[]` (Enabled: `True`). {{(pid=80378) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 02 10:04:12.576345 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.fernet_utils [None req-7be0520c-1697-43e3-bb41-85b49b2e4bb2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80378) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 10:04:12.610679 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG dbcounter [-] [80378] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80378) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 10:04:12.644914 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 569/1155] 10.222.0.82 () {58 vars in 1023 bytes} [Tue Aug 2 10:04:12 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 02 10:04:12.668149 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1321a4a0-c3f7-443f-84fd-979416940fa2 None None] Authenticating user token {{(pid=80377) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:04:12.783792 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1321a4a0-c3f7-443f-84fd-979416940fa2 None None] Validating token access rules against request {{(pid=80377) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:04:12.819201 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88bb1288-75e4-4c2f-bc95-f4ccb4506815 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '891005fa5ff74041b75747d16cc66798', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80377) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:04:12.819761 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88bb1288-75e4-4c2f-bc95-f4ccb4506815 None admin] REQUEST_METHOD: `GET` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:04:12.820041 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88bb1288-75e4-4c2f-bc95-f4ccb4506815 None admin] SCRIPT_NAME: `/identity` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:04:12.820315 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88bb1288-75e4-4c2f-bc95-f4ccb4506815 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:04:12.831627 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88bb1288-75e4-4c2f-bc95-f4ccb4506815 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 02 10:04:12.836730 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application [None req-88bb1288-75e4-4c2f-bc95-f4ccb4506815 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 02 10:04:12.836730 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 02 10:04:12.836730 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 02 10:04:12.836730 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 02 10:04:12.836730 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 02 10:04:12.836730 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 02 10:04:12.836730 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 02 10:04:12.836730 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 02 10:04:12.836730 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 02 10:04:12.836730 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 02 10:04:12.836730 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 02 10:04:12.836730 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 02 10:04:12.836730 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 02 10:04:12.836730 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 02 10:04:12.836730 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 02 10:04:12.836730 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 02 10:04:12.837861 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 02 10:04:12.837861 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 02 10:04:12.837861 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 02 10:04:12.837861 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 02 10:04:12.837861 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 02 10:04:12.837861 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 02 10:04:12.837861 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 02 10:04:12.837861 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 02 10:04:12.837861 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 02 10:04:12.837861 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application return self.get_or_create( Aug 02 10:04:12.837861 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 02 10:04:12.837861 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application with Lock( Aug 02 10:04:12.837861 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 02 10:04:12.837861 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application return self._enter() Aug 02 10:04:12.837861 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 02 10:04:12.837861 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 02 10:04:12.839115 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 02 10:04:12.839115 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application return self.creator() Aug 02 10:04:12.839115 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 02 10:04:12.839115 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application created_value = creator( Aug 02 10:04:12.839115 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 02 10:04:12.839115 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 02 10:04:12.839115 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 02 10:04:12.839115 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 02 10:04:12.839115 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 02 10:04:12.839115 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 02 10:04:12.839115 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 02 10:04:12.839115 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application Aug 02 10:04:12.839115 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: [pid: 80377|app: 0|req: 587/1156] 10.222.0.82 () {56 vars in 1096 bytes} [Tue Aug 2 10:04:12 2022] GET /identity/v3/roles/admin => generated 83 bytes in 170 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 02 10:04:12.843949 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7be0520c-1697-43e3-bb41-85b49b2e4bb2 None None] Authenticating user token {{(pid=80378) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:04:12.945579 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7be0520c-1697-43e3-bb41-85b49b2e4bb2 None None] Validating token access rules against request {{(pid=80378) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:04:12.980822 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11bc26ca-bc83-4f5a-ae62-d195c933c7f4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '891005fa5ff74041b75747d16cc66798', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80378) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:04:12.981449 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11bc26ca-bc83-4f5a-ae62-d195c933c7f4 None admin] REQUEST_METHOD: `GET` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:04:12.981733 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11bc26ca-bc83-4f5a-ae62-d195c933c7f4 None admin] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:04:12.982037 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11bc26ca-bc83-4f5a-ae62-d195c933c7f4 None admin] PATH_INFO: `/v3/roles` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:04:12.982762 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11bc26ca-bc83-4f5a-ae62-d195c933c7f4 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80378) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 02 10:04:12.983073 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11bc26ca-bc83-4f5a-ae62-d195c933c7f4 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:04:12.984463 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11bc26ca-bc83-4f5a-ae62-d195c933c7f4 None admin] RBAC: Authorization granted {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:04:12.993812 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 570/1157] 10.222.0.82 () {56 vars in 1099 bytes} [Tue Aug 2 10:04:12 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 02 10:04:12.999828 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88bb1288-75e4-4c2f-bc95-f4ccb4506815 None admin] Authenticating user token {{(pid=80377) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:04:13.056910 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG dbcounter [-] [80377] Writing DB stats keystone:SELECT=100 {{(pid=80377) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 10:04:13.126108 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88bb1288-75e4-4c2f-bc95-f4ccb4506815 None admin] Validating token access rules against request {{(pid=80377) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:04:13.168483 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8d4b8a2-bce9-49f9-8462-bee1e1d0fcc6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '891005fa5ff74041b75747d16cc66798', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80377) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:04:13.169125 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8d4b8a2-bce9-49f9-8462-bee1e1d0fcc6 None admin] REQUEST_METHOD: `GET` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:04:13.169407 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8d4b8a2-bce9-49f9-8462-bee1e1d0fcc6 None admin] SCRIPT_NAME: `/identity` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:04:13.169685 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8d4b8a2-bce9-49f9-8462-bee1e1d0fcc6 None admin] PATH_INFO: `/v3/users/070ec283a0664665bd63b48f3435b7b6` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:04:13.170871 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8d4b8a2-bce9-49f9-8462-bee1e1d0fcc6 None admin] RBAC: Authorizing `identity:get_user(user_id=070ec283a0664665bd63b48f3435b7b6)` {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:04:13.172312 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8d4b8a2-bce9-49f9-8462-bee1e1d0fcc6 None admin] RBAC: Authorization granted {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:04:13.174101 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: [pid: 80377|app: 0|req: 588/1158] 10.222.0.82 () {56 vars in 1177 bytes} [Tue Aug 2 10:04:12 2022] GET /identity/v3/users/070ec283a0664665bd63b48f3435b7b6 => generated 286 bytes in 175 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 02 10:04:13.179956 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11bc26ca-bc83-4f5a-ae62-d195c933c7f4 None admin] Authenticating user token {{(pid=80378) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:04:13.218720 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG dbcounter [-] [80378] Writing DB stats keystone:SELECT=100 {{(pid=80378) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 10:04:13.304709 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11bc26ca-bc83-4f5a-ae62-d195c933c7f4 None admin] Validating token access rules against request {{(pid=80378) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:04:13.340907 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e325e799-ad35-4714-86e8-b7a92a695a7a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '891005fa5ff74041b75747d16cc66798', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80378) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:04:13.340907 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e325e799-ad35-4714-86e8-b7a92a695a7a None admin] REQUEST_METHOD: `GET` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:04:13.340907 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e325e799-ad35-4714-86e8-b7a92a695a7a None admin] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:04:13.341337 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e325e799-ad35-4714-86e8-b7a92a695a7a None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:04:13.428516 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e325e799-ad35-4714-86e8-b7a92a695a7a None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:04:13.429921 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e325e799-ad35-4714-86e8-b7a92a695a7a None admin] RBAC: Authorization granted {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:04:13.476503 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG dbcounter [-] [80378] Writing DB stats keystone:SELECT=100 {{(pid=80378) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 10:04:13.558845 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 571/1159] 10.222.0.82 () {58 vars in 1282 bytes} [Tue Aug 2 10:04:13 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 379 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Aug 02 10:04:13.568454 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8d4b8a2-bce9-49f9-8462-bee1e1d0fcc6 None admin] Authenticating user token {{(pid=80377) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:04:13.678301 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8d4b8a2-bce9-49f9-8462-bee1e1d0fcc6 None admin] Validating token access rules against request {{(pid=80377) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:04:13.698076 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG dbcounter [-] [80377] Writing DB stats keystone:SELECT=100 {{(pid=80377) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 10:04:13.717201 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65b279d4-8cbe-425a-94e5-c6c390a35742 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '891005fa5ff74041b75747d16cc66798', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80377) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:04:13.717877 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65b279d4-8cbe-425a-94e5-c6c390a35742 None admin] REQUEST_METHOD: `GET` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:04:13.718158 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65b279d4-8cbe-425a-94e5-c6c390a35742 None admin] SCRIPT_NAME: `/identity` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:04:13.718426 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65b279d4-8cbe-425a-94e5-c6c390a35742 None admin] PATH_INFO: `/v3/projects/038d3b64e40d4eccbee14e23810d37a6` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:04:13.719584 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65b279d4-8cbe-425a-94e5-c6c390a35742 None admin] RBAC: Authorizing `identity:get_project(project_id=038d3b64e40d4eccbee14e23810d37a6)` {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:04:13.721014 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65b279d4-8cbe-425a-94e5-c6c390a35742 None admin] RBAC: Authorization granted {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:04:13.722753 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: [pid: 80377|app: 0|req: 589/1160] 10.222.0.82 () {56 vars in 1186 bytes} [Tue Aug 2 10:04:13 2022] GET /identity/v3/projects/038d3b64e40d4eccbee14e23810d37a6 => generated 311 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 02 10:04:13.728956 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e325e799-ad35-4714-86e8-b7a92a695a7a None admin] Authenticating user token {{(pid=80378) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:04:13.896162 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e325e799-ad35-4714-86e8-b7a92a695a7a None admin] Validating token access rules against request {{(pid=80378) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:04:13.945091 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG dbcounter [-] [80378] Writing DB stats keystone:SELECT=100 {{(pid=80378) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 10:04:13.953513 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8b44ec5-7e50-4147-94fa-cf5ee312b2f2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '891005fa5ff74041b75747d16cc66798', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80378) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:04:13.954500 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8b44ec5-7e50-4147-94fa-cf5ee312b2f2 None admin] REQUEST_METHOD: `GET` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:04:13.954982 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8b44ec5-7e50-4147-94fa-cf5ee312b2f2 None admin] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:04:13.955452 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8b44ec5-7e50-4147-94fa-cf5ee312b2f2 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:04:13.957489 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8b44ec5-7e50-4147-94fa-cf5ee312b2f2 None admin] RBAC: Adding query filter params (role.id=237966442f05411191105bf336bc14f3, scope.project.id=038d3b64e40d4eccbee14e23810d37a6, user.id=070ec283a0664665bd63b48f3435b7b6) {{(pid=80378) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 02 10:04:13.958056 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8b44ec5-7e50-4147-94fa-cf5ee312b2f2 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:04:13.960677 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8b44ec5-7e50-4147-94fa-cf5ee312b2f2 None admin] RBAC: Authorization granted {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:04:13.971509 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 572/1161] 10.222.0.82 () {56 vars in 1374 bytes} [Tue Aug 2 10:04:13 2022] GET /identity/v3/role_assignments?user.id=070ec283a0664665bd63b48f3435b7b6&scope.project.id=038d3b64e40d4eccbee14e23810d37a6&role.id=237966442f05411191105bf336bc14f3 => generated 608 bytes in 243 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 02 10:04:16.042794 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5faf6cc-2b72-433c-ae9b-0e8ab332d683 None None] REQUEST_METHOD: `GET` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:04:16.043323 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5faf6cc-2b72-433c-ae9b-0e8ab332d683 None None] SCRIPT_NAME: `/identity` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:04:16.043323 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5faf6cc-2b72-433c-ae9b-0e8ab332d683 None None] PATH_INFO: `/` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:04:16.044687 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: [pid: 80377|app: 0|req: 590/1162] 10.222.0.82 () {54 vars in 926 bytes} [Tue Aug 2 10:04:16 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 02 10:04:16.050674 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95391f72-bda8-4d63-bc03-f6c4a903d685 None None] REQUEST_METHOD: `POST` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:04:16.051097 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95391f72-bda8-4d63-bc03-f6c4a903d685 None None] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:04:16.051228 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95391f72-bda8-4d63-bc03-f6c4a903d685 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:04:16.076171 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.auth.core [None req-95391f72-bda8-4d63-bc03-f6c4a903d685 None None] MFA Rules not processed for user `891005fa5ff74041b75747d16cc66798`. Rule list: `[]` (Enabled: `True`). {{(pid=80378) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 02 10:04:16.077357 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.fernet_utils [None req-95391f72-bda8-4d63-bc03-f6c4a903d685 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80378) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 10:04:16.152473 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 573/1163] 10.222.0.82 () {58 vars in 1023 bytes} [Tue Aug 2 10:04:16 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 02 10:04:16.175168 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5faf6cc-2b72-433c-ae9b-0e8ab332d683 None None] Authenticating user token {{(pid=80377) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:04:16.278816 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5faf6cc-2b72-433c-ae9b-0e8ab332d683 None None] Validating token access rules against request {{(pid=80377) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:04:16.311742 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a639c59-9907-4157-a611-52aa46f2929d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '891005fa5ff74041b75747d16cc66798', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80377) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:04:16.312360 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a639c59-9907-4157-a611-52aa46f2929d None admin] REQUEST_METHOD: `GET` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:04:16.312639 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a639c59-9907-4157-a611-52aa46f2929d None admin] SCRIPT_NAME: `/identity` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:04:16.312912 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a639c59-9907-4157-a611-52aa46f2929d None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:04:16.370077 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG dbcounter [-] [80377] Writing DB stats keystone:SELECT=100 {{(pid=80377) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 10:04:16.384144 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a639c59-9907-4157-a611-52aa46f2929d None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:04:16.385606 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a639c59-9907-4157-a611-52aa46f2929d None admin] RBAC: Authorization granted {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:04:16.485966 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: [pid: 80377|app: 0|req: 591/1164] 10.222.0.82 () {58 vars in 1282 bytes} [Tue Aug 2 10:04:16 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 311 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Aug 02 10:04:16.493119 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95391f72-bda8-4d63-bc03-f6c4a903d685 None None] Authenticating user token {{(pid=80378) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:04:16.594977 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95391f72-bda8-4d63-bc03-f6c4a903d685 None None] Validating token access rules against request {{(pid=80378) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:04:16.625348 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG dbcounter [-] [80378] Writing DB stats keystone:SELECT=100 {{(pid=80378) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 10:04:16.636010 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5400831-3ca4-4535-a029-226ce31234fc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '891005fa5ff74041b75747d16cc66798', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80378) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:04:16.636909 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5400831-3ca4-4535-a029-226ce31234fc None admin] REQUEST_METHOD: `GET` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:04:16.637287 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5400831-3ca4-4535-a029-226ce31234fc None admin] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:04:16.637567 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5400831-3ca4-4535-a029-226ce31234fc None admin] PATH_INFO: `/v3/domains/swift_test` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:04:16.645639 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5400831-3ca4-4535-a029-226ce31234fc None admin] RBAC: Authorizing `identity:get_domain(domain_id=swift_test)` {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:04:16.647242 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5400831-3ca4-4535-a029-226ce31234fc None admin] RBAC: Authorization granted {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:04:16.660431 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application [None req-a5400831-3ca4-4535-a029-226ce31234fc None admin] Could not find domain: swift_test.: keystone.exception.DomainNotFound: Could not find domain: swift_test. Aug 02 10:04:16.660431 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 02 10:04:16.660431 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 712, in get_domain Aug 02 10:04:16.660431 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application project = self.driver.get_project(domain_id) Aug 02 10:04:16.660431 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 02 10:04:16.660431 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 02 10:04:16.660431 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 02 10:04:16.660431 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 02 10:04:16.660431 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: swift_test. Aug 02 10:04:16.660431 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application Aug 02 10:04:16.660431 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 02 10:04:16.660431 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application Aug 02 10:04:16.660431 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 02 10:04:16.660431 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 02 10:04:16.660431 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 02 10:04:16.660431 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 02 10:04:16.660431 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 02 10:04:16.661813 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 02 10:04:16.661813 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 02 10:04:16.661813 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 02 10:04:16.661813 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 02 10:04:16.661813 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 02 10:04:16.661813 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 02 10:04:16.661813 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 02 10:04:16.661813 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 02 10:04:16.661813 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 02 10:04:16.661813 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 02 10:04:16.661813 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 02 10:04:16.661813 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 02 10:04:16.661813 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 02 10:04:16.661813 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 02 10:04:16.661813 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 02 10:04:16.661813 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application return self.get_or_create( Aug 02 10:04:16.663019 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 02 10:04:16.663019 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application with Lock( Aug 02 10:04:16.663019 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 02 10:04:16.663019 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application return self._enter() Aug 02 10:04:16.663019 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 02 10:04:16.663019 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 02 10:04:16.663019 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 02 10:04:16.663019 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application return self.creator() Aug 02 10:04:16.663019 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 02 10:04:16.663019 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application created_value = creator( Aug 02 10:04:16.663019 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 718, in get_domain Aug 02 10:04:16.663019 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 02 10:04:16.663019 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: swift_test. Aug 02 10:04:16.663019 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application Aug 02 10:04:16.663019 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 574/1165] 10.222.0.82 () {56 vars in 1117 bytes} [Tue Aug 2 10:04:16 2022] GET /identity/v3/domains/swift_test => generated 90 bytes in 170 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 02 10:04:16.670250 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a639c59-9907-4157-a611-52aa46f2929d None admin] Authenticating user token {{(pid=80377) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:04:16.771447 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG dbcounter [-] [80377] Writing DB stats keystone:SELECT=100 {{(pid=80377) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 10:04:16.790097 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a639c59-9907-4157-a611-52aa46f2929d None admin] Validating token access rules against request {{(pid=80377) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:04:16.826322 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d98c6ffb-b679-4ea5-9ca6-e77f5b217b26 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '891005fa5ff74041b75747d16cc66798', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80377) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:04:16.826947 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d98c6ffb-b679-4ea5-9ca6-e77f5b217b26 None admin] REQUEST_METHOD: `GET` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:04:16.827226 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d98c6ffb-b679-4ea5-9ca6-e77f5b217b26 None admin] SCRIPT_NAME: `/identity` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:04:16.827501 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d98c6ffb-b679-4ea5-9ca6-e77f5b217b26 None admin] PATH_INFO: `/v3/domains` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:04:16.828190 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d98c6ffb-b679-4ea5-9ca6-e77f5b217b26 None admin] RBAC: Adding query filter params (name=swift_test) {{(pid=80377) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 02 10:04:16.828491 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d98c6ffb-b679-4ea5-9ca6-e77f5b217b26 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:04:16.830123 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d98c6ffb-b679-4ea5-9ca6-e77f5b217b26 None admin] RBAC: Authorization granted {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:04:16.837656 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: [pid: 80377|app: 0|req: 592/1166] 10.222.0.82 () {56 vars in 1115 bytes} [Tue Aug 2 10:04:16 2022] GET /identity/v3/domains?name=swift_test => generated 125 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 02 10:04:19.008725 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-861079f1-1b24-4145-831e-ee58f4858ae1 None None] REQUEST_METHOD: `GET` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:04:19.009285 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-861079f1-1b24-4145-831e-ee58f4858ae1 None None] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:04:19.009613 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-861079f1-1b24-4145-831e-ee58f4858ae1 None None] PATH_INFO: `/` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:04:19.012082 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 575/1167] 10.222.0.82 () {54 vars in 926 bytes} [Tue Aug 2 10:04:19 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 02 10:04:19.018916 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12abd29c-c01c-4959-a7e1-ecfb78e6d8a5 None None] REQUEST_METHOD: `POST` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:04:19.019448 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12abd29c-c01c-4959-a7e1-ecfb78e6d8a5 None None] SCRIPT_NAME: `/identity` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:04:19.019542 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12abd29c-c01c-4959-a7e1-ecfb78e6d8a5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:04:19.043040 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.auth.core [None req-12abd29c-c01c-4959-a7e1-ecfb78e6d8a5 None None] MFA Rules not processed for user `891005fa5ff74041b75747d16cc66798`. Rule list: `[]` (Enabled: `True`). {{(pid=80377) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 02 10:04:19.044961 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.fernet_utils [None req-12abd29c-c01c-4959-a7e1-ecfb78e6d8a5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80377) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 10:04:19.142396 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: [pid: 80377|app: 0|req: 593/1168] 10.222.0.82 () {58 vars in 1023 bytes} [Tue Aug 2 10:04:19 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 125 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 02 10:04:19.166359 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-861079f1-1b24-4145-831e-ee58f4858ae1 None None] Authenticating user token {{(pid=80378) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:04:19.327288 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-861079f1-1b24-4145-831e-ee58f4858ae1 None None] Validating token access rules against request {{(pid=80378) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:04:19.387873 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c29c7bc-81f3-4fd0-910f-0808826471ff None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '891005fa5ff74041b75747d16cc66798', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80378) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:04:19.388504 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c29c7bc-81f3-4fd0-910f-0808826471ff None admin] REQUEST_METHOD: `POST` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:04:19.388789 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c29c7bc-81f3-4fd0-910f-0808826471ff None admin] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:04:19.389066 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c29c7bc-81f3-4fd0-910f-0808826471ff None admin] PATH_INFO: `/v3/domains` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:04:19.389843 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c29c7bc-81f3-4fd0-910f-0808826471ff None admin] RBAC: Authorizing `identity:create_domain()` {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:04:19.391363 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c29c7bc-81f3-4fd0-910f-0808826471ff None admin] RBAC: Authorization granted {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:04:19.415324 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 576/1169] 10.222.0.82 () {60 vars in 1138 bytes} [Tue Aug 2 10:04:19 2022] POST /identity/v3/domains => generated 267 bytes in 250 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 02 10:04:21.556989 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f16fb9c-b5ce-49ef-a6db-7da35ec62238 None None] REQUEST_METHOD: `GET` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:04:21.557567 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f16fb9c-b5ce-49ef-a6db-7da35ec62238 None None] SCRIPT_NAME: `/identity` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:04:21.557664 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f16fb9c-b5ce-49ef-a6db-7da35ec62238 None None] PATH_INFO: `/` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:04:21.559032 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: [pid: 80377|app: 0|req: 594/1170] 10.222.0.82 () {54 vars in 926 bytes} [Tue Aug 2 10:04:21 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 02 10:04:21.564763 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e451731-387a-4a1b-ba47-2a5e9c2db655 None None] REQUEST_METHOD: `POST` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:04:21.565395 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e451731-387a-4a1b-ba47-2a5e9c2db655 None None] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:04:21.565395 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e451731-387a-4a1b-ba47-2a5e9c2db655 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:04:21.584376 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.auth.core [None req-9e451731-387a-4a1b-ba47-2a5e9c2db655 None None] MFA Rules not processed for user `891005fa5ff74041b75747d16cc66798`. Rule list: `[]` (Enabled: `True`). {{(pid=80378) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 02 10:04:21.586122 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.fernet_utils [None req-9e451731-387a-4a1b-ba47-2a5e9c2db655 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80378) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 10:04:21.665908 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG dbcounter [-] [80378] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80378) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 10:04:21.704819 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 577/1171] 10.222.0.82 () {58 vars in 1023 bytes} [Tue Aug 2 10:04:21 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 141 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 02 10:04:21.729820 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f16fb9c-b5ce-49ef-a6db-7da35ec62238 None None] Authenticating user token {{(pid=80377) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:04:21.824874 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG dbcounter [-] [80377] Writing DB stats keystone:SELECT=100 {{(pid=80377) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 10:04:21.849099 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f16fb9c-b5ce-49ef-a6db-7da35ec62238 None None] Validating token access rules against request {{(pid=80377) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:04:21.893487 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e08b21b5-7d1c-47d8-b9cf-0955f63a92b6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '891005fa5ff74041b75747d16cc66798', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80377) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:04:21.894672 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e08b21b5-7d1c-47d8-b9cf-0955f63a92b6 None admin] REQUEST_METHOD: `GET` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:04:21.894953 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e08b21b5-7d1c-47d8-b9cf-0955f63a92b6 None admin] SCRIPT_NAME: `/identity` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:04:21.895227 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e08b21b5-7d1c-47d8-b9cf-0955f63a92b6 None admin] PATH_INFO: `/v3/domains/3d7d054785bb4b8b8f9a9a764be3a6e0` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:04:21.906569 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e08b21b5-7d1c-47d8-b9cf-0955f63a92b6 None admin] RBAC: Authorizing `identity:get_domain(domain_id=3d7d054785bb4b8b8f9a9a764be3a6e0)` {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:04:21.908087 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e08b21b5-7d1c-47d8-b9cf-0955f63a92b6 None admin] RBAC: Authorization granted {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:04:21.910373 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: [pid: 80377|app: 0|req: 595/1172] 10.222.0.82 () {56 vars in 1183 bytes} [Tue Aug 2 10:04:21 2022] GET /identity/v3/domains/3d7d054785bb4b8b8f9a9a764be3a6e0 => generated 267 bytes in 181 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 02 10:04:21.919490 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e451731-387a-4a1b-ba47-2a5e9c2db655 None None] Authenticating user token {{(pid=80378) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:04:22.063314 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e451731-387a-4a1b-ba47-2a5e9c2db655 None None] Validating token access rules against request {{(pid=80378) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:04:22.108271 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e57ad0f6-27cf-482a-b725-4b170b86cc89 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '891005fa5ff74041b75747d16cc66798', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80378) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:04:22.108825 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e57ad0f6-27cf-482a-b725-4b170b86cc89 None admin] REQUEST_METHOD: `POST` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:04:22.109619 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e57ad0f6-27cf-482a-b725-4b170b86cc89 None admin] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:04:22.109619 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e57ad0f6-27cf-482a-b725-4b170b86cc89 None admin] PATH_INFO: `/v3/projects` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:04:22.110128 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e57ad0f6-27cf-482a-b725-4b170b86cc89 None admin] RBAC: Authorizing `identity:create_project()` {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:04:22.111619 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e57ad0f6-27cf-482a-b725-4b170b86cc89 None admin] RBAC: Authorization granted {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:04:22.145415 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 578/1173] 10.222.0.82 () {60 vars in 1141 bytes} [Tue Aug 2 10:04:21 2022] POST /identity/v3/projects => generated 361 bytes in 227 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 02 10:04:24.307725 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34923626-04bc-4d7f-82b9-4ba383f2e498 None None] REQUEST_METHOD: `GET` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:04:24.308666 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34923626-04bc-4d7f-82b9-4ba383f2e498 None None] SCRIPT_NAME: `/identity` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:04:24.308666 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34923626-04bc-4d7f-82b9-4ba383f2e498 None None] PATH_INFO: `/` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:04:24.310266 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: [pid: 80377|app: 0|req: 596/1174] 10.222.0.82 () {54 vars in 926 bytes} [Tue Aug 2 10:04:24 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 02 10:04:24.316274 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4edcbe3-99de-47c5-aee2-3a08327a69a4 None None] REQUEST_METHOD: `POST` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:04:24.316831 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4edcbe3-99de-47c5-aee2-3a08327a69a4 None None] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:04:24.316831 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4edcbe3-99de-47c5-aee2-3a08327a69a4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:04:24.336685 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.auth.core [None req-f4edcbe3-99de-47c5-aee2-3a08327a69a4 None None] MFA Rules not processed for user `891005fa5ff74041b75747d16cc66798`. Rule list: `[]` (Enabled: `True`). {{(pid=80378) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 02 10:04:24.337973 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.fernet_utils [None req-f4edcbe3-99de-47c5-aee2-3a08327a69a4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80378) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 10:04:24.362293 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG dbcounter [-] [80378] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80378) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 10:04:24.418985 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 579/1175] 10.222.0.82 () {58 vars in 1023 bytes} [Tue Aug 2 10:04:24 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 104 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 02 10:04:24.444563 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34923626-04bc-4d7f-82b9-4ba383f2e498 None None] Authenticating user token {{(pid=80377) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:04:24.563611 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34923626-04bc-4d7f-82b9-4ba383f2e498 None None] Validating token access rules against request {{(pid=80377) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:04:24.603549 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69622d9b-3bc5-4692-90f6-215ea5405f47 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '891005fa5ff74041b75747d16cc66798', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80377) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:04:24.604215 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69622d9b-3bc5-4692-90f6-215ea5405f47 None admin] REQUEST_METHOD: `GET` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:04:24.604500 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69622d9b-3bc5-4692-90f6-215ea5405f47 None admin] SCRIPT_NAME: `/identity` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:04:24.604774 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69622d9b-3bc5-4692-90f6-215ea5405f47 None admin] PATH_INFO: `/v3/domains/3d7d054785bb4b8b8f9a9a764be3a6e0` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:04:24.607725 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69622d9b-3bc5-4692-90f6-215ea5405f47 None admin] RBAC: Authorizing `identity:get_domain(domain_id=3d7d054785bb4b8b8f9a9a764be3a6e0)` {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:04:24.609185 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69622d9b-3bc5-4692-90f6-215ea5405f47 None admin] RBAC: Authorization granted {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:04:24.610849 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: [pid: 80377|app: 0|req: 597/1176] 10.222.0.82 () {56 vars in 1183 bytes} [Tue Aug 2 10:04:24 2022] GET /identity/v3/domains/3d7d054785bb4b8b8f9a9a764be3a6e0 => generated 267 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 02 10:04:24.616896 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4edcbe3-99de-47c5-aee2-3a08327a69a4 None None] Authenticating user token {{(pid=80378) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:04:24.756495 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4edcbe3-99de-47c5-aee2-3a08327a69a4 None None] Validating token access rules against request {{(pid=80378) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:04:24.795709 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13de0e00-b4c9-4c8a-8b68-819520ca23e9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '891005fa5ff74041b75747d16cc66798', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80378) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:04:24.796339 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13de0e00-b4c9-4c8a-8b68-819520ca23e9 None admin] REQUEST_METHOD: `POST` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:04:24.796618 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13de0e00-b4c9-4c8a-8b68-819520ca23e9 None admin] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:04:24.796891 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13de0e00-b4c9-4c8a-8b68-819520ca23e9 None admin] PATH_INFO: `/v3/users` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:04:24.797600 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13de0e00-b4c9-4c8a-8b68-819520ca23e9 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:04:24.799070 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13de0e00-b4c9-4c8a-8b68-819520ca23e9 None admin] RBAC: Authorization granted {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:04:24.825886 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 580/1177] 10.222.0.82 () {60 vars in 1132 bytes} [Tue Aug 2 10:04:24 2022] POST /identity/v3/users => generated 311 bytes in 209 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 02 10:04:26.936596 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-127f0ecf-2982-485a-a780-a4da506fae84 None None] REQUEST_METHOD: `GET` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:04:26.937158 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-127f0ecf-2982-485a-a780-a4da506fae84 None None] SCRIPT_NAME: `/identity` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:04:26.937158 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-127f0ecf-2982-485a-a780-a4da506fae84 None None] PATH_INFO: `/` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:04:26.938555 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: [pid: 80377|app: 0|req: 598/1178] 10.222.0.82 () {54 vars in 926 bytes} [Tue Aug 2 10:04:26 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 02 10:04:26.945009 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d30b2141-655b-4ee3-8f74-4e23631009a3 None None] REQUEST_METHOD: `POST` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:04:26.945493 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d30b2141-655b-4ee3-8f74-4e23631009a3 None None] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:04:26.945951 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d30b2141-655b-4ee3-8f74-4e23631009a3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:04:26.965519 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.auth.core [None req-d30b2141-655b-4ee3-8f74-4e23631009a3 None None] MFA Rules not processed for user `891005fa5ff74041b75747d16cc66798`. Rule list: `[]` (Enabled: `True`). {{(pid=80378) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 02 10:04:26.966776 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.fernet_utils [None req-d30b2141-655b-4ee3-8f74-4e23631009a3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80378) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 10:04:26.977436 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG dbcounter [-] [80378] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80378) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 10:04:27.045043 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 581/1179] 10.222.0.82 () {58 vars in 1023 bytes} [Tue Aug 2 10:04:26 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 02 10:04:27.068356 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-127f0ecf-2982-485a-a780-a4da506fae84 None None] Authenticating user token {{(pid=80377) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:04:27.080295 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG dbcounter [-] [80377] Writing DB stats keystone:SELECT=100 {{(pid=80377) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 10:04:27.175140 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-127f0ecf-2982-485a-a780-a4da506fae84 None None] Validating token access rules against request {{(pid=80377) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:04:27.209257 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ebadf12-e6c8-4c71-b5c3-f0cddd469909 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '891005fa5ff74041b75747d16cc66798', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80377) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:04:27.209974 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ebadf12-e6c8-4c71-b5c3-f0cddd469909 None admin] REQUEST_METHOD: `GET` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:04:27.210284 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ebadf12-e6c8-4c71-b5c3-f0cddd469909 None admin] SCRIPT_NAME: `/identity` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:04:27.210558 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ebadf12-e6c8-4c71-b5c3-f0cddd469909 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:04:27.221481 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ebadf12-e6c8-4c71-b5c3-f0cddd469909 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 02 10:04:27.227117 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application [None req-3ebadf12-e6c8-4c71-b5c3-f0cddd469909 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 02 10:04:27.227117 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 02 10:04:27.227117 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 02 10:04:27.227117 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 02 10:04:27.227117 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 02 10:04:27.227117 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 02 10:04:27.227117 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 02 10:04:27.227117 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 02 10:04:27.227117 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 02 10:04:27.227117 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 02 10:04:27.227117 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 02 10:04:27.227117 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 02 10:04:27.227117 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 02 10:04:27.227117 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 02 10:04:27.227117 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 02 10:04:27.227117 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 02 10:04:27.228391 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 02 10:04:27.228391 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 02 10:04:27.228391 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 02 10:04:27.228391 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 02 10:04:27.228391 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 02 10:04:27.228391 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 02 10:04:27.228391 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 02 10:04:27.228391 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 02 10:04:27.228391 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 02 10:04:27.228391 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application return self.get_or_create( Aug 02 10:04:27.228391 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 02 10:04:27.228391 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application with Lock( Aug 02 10:04:27.228391 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 02 10:04:27.228391 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application return self._enter() Aug 02 10:04:27.228391 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 02 10:04:27.228391 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 02 10:04:27.229722 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 02 10:04:27.229722 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application return self.creator() Aug 02 10:04:27.229722 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 02 10:04:27.229722 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application created_value = creator( Aug 02 10:04:27.229722 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 02 10:04:27.229722 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 02 10:04:27.229722 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 02 10:04:27.229722 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 02 10:04:27.229722 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 02 10:04:27.229722 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 02 10:04:27.229722 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 02 10:04:27.229722 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application Aug 02 10:04:27.229722 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: [pid: 80377|app: 0|req: 599/1180] 10.222.0.82 () {56 vars in 1096 bytes} [Tue Aug 2 10:04:27 2022] GET /identity/v3/roles/admin => generated 83 bytes in 160 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 02 10:04:27.234644 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d30b2141-655b-4ee3-8f74-4e23631009a3 None None] Authenticating user token {{(pid=80378) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:04:27.363807 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d30b2141-655b-4ee3-8f74-4e23631009a3 None None] Validating token access rules against request {{(pid=80378) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:04:27.399730 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8697cec-7b2e-4bed-8e46-d58372331da3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '891005fa5ff74041b75747d16cc66798', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80378) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:04:27.400340 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8697cec-7b2e-4bed-8e46-d58372331da3 None admin] REQUEST_METHOD: `GET` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:04:27.400621 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8697cec-7b2e-4bed-8e46-d58372331da3 None admin] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:04:27.400892 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8697cec-7b2e-4bed-8e46-d58372331da3 None admin] PATH_INFO: `/v3/roles` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:04:27.401610 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8697cec-7b2e-4bed-8e46-d58372331da3 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80378) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 02 10:04:27.401948 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8697cec-7b2e-4bed-8e46-d58372331da3 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:04:27.403328 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8697cec-7b2e-4bed-8e46-d58372331da3 None admin] RBAC: Authorization granted {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:04:27.412915 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 582/1181] 10.222.0.82 () {56 vars in 1099 bytes} [Tue Aug 2 10:04:27 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 179 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 02 10:04:27.419007 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ebadf12-e6c8-4c71-b5c3-f0cddd469909 None admin] Authenticating user token {{(pid=80377) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:04:27.506127 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG dbcounter [-] [80377] Writing DB stats keystone:SELECT=100 {{(pid=80377) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 10:04:27.549645 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ebadf12-e6c8-4c71-b5c3-f0cddd469909 None admin] Validating token access rules against request {{(pid=80377) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:04:27.587235 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e49023b-123c-4ffe-b6d6-b32d25ea9955 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '891005fa5ff74041b75747d16cc66798', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80377) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:04:27.587819 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e49023b-123c-4ffe-b6d6-b32d25ea9955 None admin] REQUEST_METHOD: `GET` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:04:27.588097 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e49023b-123c-4ffe-b6d6-b32d25ea9955 None admin] SCRIPT_NAME: `/identity` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:04:27.588371 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e49023b-123c-4ffe-b6d6-b32d25ea9955 None admin] PATH_INFO: `/v3/users/b5d506bd78e44c038492ac8d6a337d0d` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:04:27.601769 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e49023b-123c-4ffe-b6d6-b32d25ea9955 None admin] RBAC: Authorizing `identity:get_user(user_id=b5d506bd78e44c038492ac8d6a337d0d)` {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:04:27.603222 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e49023b-123c-4ffe-b6d6-b32d25ea9955 None admin] RBAC: Authorization granted {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:04:27.604850 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: [pid: 80377|app: 0|req: 600/1182] 10.222.0.82 () {56 vars in 1177 bytes} [Tue Aug 2 10:04:27 2022] GET /identity/v3/users/b5d506bd78e44c038492ac8d6a337d0d => generated 311 bytes in 186 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 02 10:04:27.611204 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8697cec-7b2e-4bed-8e46-d58372331da3 None admin] Authenticating user token {{(pid=80378) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:04:27.621145 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG dbcounter [-] [80378] Writing DB stats keystone:SELECT=100 {{(pid=80378) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 10:04:27.724192 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8697cec-7b2e-4bed-8e46-d58372331da3 None admin] Validating token access rules against request {{(pid=80378) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:04:27.759058 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bc3f4ca-ecda-4742-934e-035c58758ea1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '891005fa5ff74041b75747d16cc66798', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80378) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:04:27.759594 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bc3f4ca-ecda-4742-934e-035c58758ea1 None admin] REQUEST_METHOD: `GET` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:04:27.759880 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bc3f4ca-ecda-4742-934e-035c58758ea1 None admin] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:04:27.760158 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bc3f4ca-ecda-4742-934e-035c58758ea1 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:04:27.852502 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7bc3f4ca-ecda-4742-934e-035c58758ea1 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:04:27.853573 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7bc3f4ca-ecda-4742-934e-035c58758ea1 None admin] RBAC: Authorization granted {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:04:27.863204 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG dbcounter [-] [80378] Writing DB stats keystone:SELECT=100 {{(pid=80378) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 10:04:27.956854 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 583/1183] 10.222.0.82 () {58 vars in 1282 bytes} [Tue Aug 2 10:04:27 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 346 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Aug 02 10:04:27.964089 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e49023b-123c-4ffe-b6d6-b32d25ea9955 None admin] Authenticating user token {{(pid=80377) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:04:28.075330 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e49023b-123c-4ffe-b6d6-b32d25ea9955 None admin] Validating token access rules against request {{(pid=80377) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:04:28.111409 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG dbcounter [-] [80377] Writing DB stats keystone:SELECT=100 {{(pid=80377) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 10:04:28.127468 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3f3a537-633e-4028-8443-4fe01fabdc96 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '891005fa5ff74041b75747d16cc66798', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80377) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:04:28.128665 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3f3a537-633e-4028-8443-4fe01fabdc96 None admin] REQUEST_METHOD: `GET` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:04:28.128946 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3f3a537-633e-4028-8443-4fe01fabdc96 None admin] SCRIPT_NAME: `/identity` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:04:28.129220 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3f3a537-633e-4028-8443-4fe01fabdc96 None admin] PATH_INFO: `/v3/projects/c7080bd1bd284215bc7f023dbe7f17bf` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:04:28.130453 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3f3a537-633e-4028-8443-4fe01fabdc96 None admin] RBAC: Authorizing `identity:get_project(project_id=c7080bd1bd284215bc7f023dbe7f17bf)` {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:04:28.131897 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3f3a537-633e-4028-8443-4fe01fabdc96 None admin] RBAC: Authorization granted {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:04:28.133596 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: [pid: 80377|app: 0|req: 601/1184] 10.222.0.82 () {56 vars in 1186 bytes} [Tue Aug 2 10:04:27 2022] GET /identity/v3/projects/c7080bd1bd284215bc7f023dbe7f17bf => generated 361 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 02 10:04:28.140040 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bc3f4ca-ecda-4742-934e-035c58758ea1 None admin] Authenticating user token {{(pid=80378) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:04:28.255962 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bc3f4ca-ecda-4742-934e-035c58758ea1 None admin] Validating token access rules against request {{(pid=80378) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:04:28.263895 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG dbcounter [-] [80378] Writing DB stats keystone:SELECT=100 {{(pid=80378) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 10:04:28.294893 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0978553-660a-4a07-ab4f-c644a36b0d5e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '891005fa5ff74041b75747d16cc66798', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80378) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:04:28.295517 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0978553-660a-4a07-ab4f-c644a36b0d5e None admin] REQUEST_METHOD: `GET` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:04:28.295793 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0978553-660a-4a07-ab4f-c644a36b0d5e None admin] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:04:28.296063 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0978553-660a-4a07-ab4f-c644a36b0d5e None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:04:28.296858 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0978553-660a-4a07-ab4f-c644a36b0d5e None admin] RBAC: Adding query filter params (role.id=237966442f05411191105bf336bc14f3, scope.project.id=c7080bd1bd284215bc7f023dbe7f17bf, user.id=b5d506bd78e44c038492ac8d6a337d0d) {{(pid=80378) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 02 10:04:28.297163 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0978553-660a-4a07-ab4f-c644a36b0d5e None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:04:28.298635 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0978553-660a-4a07-ab4f-c644a36b0d5e None admin] RBAC: Authorization granted {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:04:28.307196 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 584/1185] 10.222.0.82 () {56 vars in 1374 bytes} [Tue Aug 2 10:04:28 2022] GET /identity/v3/role_assignments?user.id=b5d506bd78e44c038492ac8d6a337d0d&scope.project.id=c7080bd1bd284215bc7f023dbe7f17bf&role.id=237966442f05411191105bf336bc14f3 => generated 259 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 02 10:04:30.372102 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b60c4751-5f5d-4322-b57c-62c684974dbf None None] REQUEST_METHOD: `GET` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:04:30.372699 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b60c4751-5f5d-4322-b57c-62c684974dbf None None] SCRIPT_NAME: `/identity` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:04:30.373064 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b60c4751-5f5d-4322-b57c-62c684974dbf None None] PATH_INFO: `/` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:04:30.375251 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: [pid: 80377|app: 0|req: 602/1186] 10.222.0.82 () {54 vars in 926 bytes} [Tue Aug 2 10:04:30 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 02 10:04:30.382055 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe891bec-daf2-4a20-8889-47524bc1d529 None None] REQUEST_METHOD: `POST` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:04:30.382602 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe891bec-daf2-4a20-8889-47524bc1d529 None None] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:04:30.382602 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe891bec-daf2-4a20-8889-47524bc1d529 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:04:30.405192 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.auth.core [None req-fe891bec-daf2-4a20-8889-47524bc1d529 None None] MFA Rules not processed for user `891005fa5ff74041b75747d16cc66798`. Rule list: `[]` (Enabled: `True`). {{(pid=80378) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 02 10:04:30.406483 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.fernet_utils [None req-fe891bec-daf2-4a20-8889-47524bc1d529 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80378) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 10:04:30.487845 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 585/1187] 10.222.0.82 () {58 vars in 1023 bytes} [Tue Aug 2 10:04:30 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 107 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 02 10:04:30.512711 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b60c4751-5f5d-4322-b57c-62c684974dbf None None] Authenticating user token {{(pid=80377) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:04:30.632964 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b60c4751-5f5d-4322-b57c-62c684974dbf None None] Validating token access rules against request {{(pid=80377) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:04:30.670134 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c53d3494-e403-4ab5-adfd-891471b5feac None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '891005fa5ff74041b75747d16cc66798', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80377) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:04:30.670795 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c53d3494-e403-4ab5-adfd-891471b5feac None admin] REQUEST_METHOD: `GET` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:04:30.671080 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c53d3494-e403-4ab5-adfd-891471b5feac None admin] SCRIPT_NAME: `/identity` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:04:30.671353 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c53d3494-e403-4ab5-adfd-891471b5feac None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:04:30.683025 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c53d3494-e403-4ab5-adfd-891471b5feac None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 02 10:04:30.688027 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application [None req-c53d3494-e403-4ab5-adfd-891471b5feac None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 02 10:04:30.688027 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 02 10:04:30.688027 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 02 10:04:30.688027 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 02 10:04:30.688027 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 02 10:04:30.688027 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 02 10:04:30.688027 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 02 10:04:30.688027 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 02 10:04:30.688027 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 02 10:04:30.688027 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 02 10:04:30.688027 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 02 10:04:30.688027 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 02 10:04:30.688027 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 02 10:04:30.688027 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 02 10:04:30.688027 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 02 10:04:30.688027 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 02 10:04:30.689221 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 02 10:04:30.689221 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 02 10:04:30.689221 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 02 10:04:30.689221 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 02 10:04:30.689221 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 02 10:04:30.689221 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 02 10:04:30.689221 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 02 10:04:30.689221 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 02 10:04:30.689221 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 02 10:04:30.689221 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application return self.get_or_create( Aug 02 10:04:30.689221 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 02 10:04:30.689221 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application with Lock( Aug 02 10:04:30.689221 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 02 10:04:30.689221 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application return self._enter() Aug 02 10:04:30.689221 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 02 10:04:30.689221 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 02 10:04:30.690155 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 02 10:04:30.690155 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application return self.creator() Aug 02 10:04:30.690155 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 02 10:04:30.690155 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application created_value = creator( Aug 02 10:04:30.690155 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 02 10:04:30.690155 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 02 10:04:30.690155 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 02 10:04:30.690155 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 02 10:04:30.690155 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 02 10:04:30.690155 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 02 10:04:30.690155 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 02 10:04:30.690155 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application Aug 02 10:04:30.690155 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: [pid: 80377|app: 0|req: 603/1188] 10.222.0.82 () {56 vars in 1096 bytes} [Tue Aug 2 10:04:30 2022] GET /identity/v3/roles/admin => generated 83 bytes in 177 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 02 10:04:30.695973 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe891bec-daf2-4a20-8889-47524bc1d529 None None] Authenticating user token {{(pid=80378) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:04:30.795163 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG dbcounter [-] [80378] Writing DB stats keystone:SELECT=100 {{(pid=80378) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 10:04:30.798416 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe891bec-daf2-4a20-8889-47524bc1d529 None None] Validating token access rules against request {{(pid=80378) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:04:30.834958 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1caae69-eb26-4737-9d59-d9f08d42878a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '891005fa5ff74041b75747d16cc66798', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80378) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:04:30.835584 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1caae69-eb26-4737-9d59-d9f08d42878a None admin] REQUEST_METHOD: `GET` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:04:30.835866 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1caae69-eb26-4737-9d59-d9f08d42878a None admin] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:04:30.836140 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1caae69-eb26-4737-9d59-d9f08d42878a None admin] PATH_INFO: `/v3/roles` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:04:30.836861 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1caae69-eb26-4737-9d59-d9f08d42878a None admin] RBAC: Adding query filter params (name=admin) {{(pid=80378) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 02 10:04:30.837166 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1caae69-eb26-4737-9d59-d9f08d42878a None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:04:30.838571 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1caae69-eb26-4737-9d59-d9f08d42878a None admin] RBAC: Authorization granted {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:04:30.847564 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 586/1189] 10.222.0.82 () {56 vars in 1099 bytes} [Tue Aug 2 10:04:30 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 02 10:04:30.853471 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c53d3494-e403-4ab5-adfd-891471b5feac None admin] Authenticating user token {{(pid=80377) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:04:30.916684 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG dbcounter [-] [80377] Writing DB stats keystone:SELECT=100 {{(pid=80377) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 10:04:30.977878 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c53d3494-e403-4ab5-adfd-891471b5feac None admin] Validating token access rules against request {{(pid=80377) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:04:31.011657 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3419209-2712-4940-8da0-7f2ab97eb987 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '891005fa5ff74041b75747d16cc66798', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80377) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:04:31.012300 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3419209-2712-4940-8da0-7f2ab97eb987 None admin] REQUEST_METHOD: `GET` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:04:31.012580 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3419209-2712-4940-8da0-7f2ab97eb987 None admin] SCRIPT_NAME: `/identity` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:04:31.012853 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3419209-2712-4940-8da0-7f2ab97eb987 None admin] PATH_INFO: `/v3/users/b5d506bd78e44c038492ac8d6a337d0d` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:04:31.014055 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3419209-2712-4940-8da0-7f2ab97eb987 None admin] RBAC: Authorizing `identity:get_user(user_id=b5d506bd78e44c038492ac8d6a337d0d)` {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:04:31.015488 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3419209-2712-4940-8da0-7f2ab97eb987 None admin] RBAC: Authorization granted {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:04:31.017107 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: [pid: 80377|app: 0|req: 604/1190] 10.222.0.82 () {56 vars in 1177 bytes} [Tue Aug 2 10:04:30 2022] GET /identity/v3/users/b5d506bd78e44c038492ac8d6a337d0d => generated 311 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 02 10:04:31.024552 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1caae69-eb26-4737-9d59-d9f08d42878a None admin] Authenticating user token {{(pid=80378) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:04:31.124367 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1caae69-eb26-4737-9d59-d9f08d42878a None admin] Validating token access rules against request {{(pid=80378) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:04:31.158669 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db5292f9-c55d-4ef1-ba27-fa97f747874c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '891005fa5ff74041b75747d16cc66798', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80378) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:04:31.159306 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db5292f9-c55d-4ef1-ba27-fa97f747874c None admin] REQUEST_METHOD: `GET` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:04:31.159625 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db5292f9-c55d-4ef1-ba27-fa97f747874c None admin] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:04:31.159899 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db5292f9-c55d-4ef1-ba27-fa97f747874c None admin] PATH_INFO: `/v3/projects/c7080bd1bd284215bc7f023dbe7f17bf` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:04:31.161061 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db5292f9-c55d-4ef1-ba27-fa97f747874c None admin] RBAC: Authorizing `identity:get_project(project_id=c7080bd1bd284215bc7f023dbe7f17bf)` {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:04:31.162523 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db5292f9-c55d-4ef1-ba27-fa97f747874c None admin] RBAC: Authorization granted {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:04:31.164498 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 587/1191] 10.222.0.82 () {56 vars in 1186 bytes} [Tue Aug 2 10:04:31 2022] GET /identity/v3/projects/c7080bd1bd284215bc7f023dbe7f17bf => generated 361 bytes in 141 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 02 10:04:31.170624 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3419209-2712-4940-8da0-7f2ab97eb987 None admin] Authenticating user token {{(pid=80377) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:04:31.272459 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3419209-2712-4940-8da0-7f2ab97eb987 None admin] Validating token access rules against request {{(pid=80377) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:04:31.296052 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG dbcounter [-] [80377] Writing DB stats keystone:SELECT=100 {{(pid=80377) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 10:04:31.309364 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35433afc-90ce-4cf3-8d03-e1ae799f7ed9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '891005fa5ff74041b75747d16cc66798', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80377) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:04:31.310079 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35433afc-90ce-4cf3-8d03-e1ae799f7ed9 None admin] REQUEST_METHOD: `PUT` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:04:31.310379 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35433afc-90ce-4cf3-8d03-e1ae799f7ed9 None admin] SCRIPT_NAME: `/identity` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:04:31.310677 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35433afc-90ce-4cf3-8d03-e1ae799f7ed9 None admin] PATH_INFO: `/v3/projects/c7080bd1bd284215bc7f023dbe7f17bf/users/b5d506bd78e44c038492ac8d6a337d0d/roles/237966442f05411191105bf336bc14f3` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:04:31.312681 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35433afc-90ce-4cf3-8d03-e1ae799f7ed9 None admin] RBAC: Authorizing `identity:create_grant(project_id=c7080bd1bd284215bc7f023dbe7f17bf, user_id=b5d506bd78e44c038492ac8d6a337d0d, role_id=237966442f05411191105bf336bc14f3)` {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:04:31.314214 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35433afc-90ce-4cf3-8d03-e1ae799f7ed9 None admin] RBAC: Authorization granted {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:04:31.327277 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: [pid: 80377|app: 0|req: 605/1192] 10.222.0.82 () {58 vars in 1439 bytes} [Tue Aug 2 10:04:31 2022] PUT /identity/v3/projects/c7080bd1bd284215bc7f023dbe7f17bf/users/b5d506bd78e44c038492ac8d6a337d0d/roles/237966442f05411191105bf336bc14f3 => generated 0 bytes in 157 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 02 10:04:33.407734 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bbf64b7-7858-459d-9eea-302168bcc99d None None] REQUEST_METHOD: `GET` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:04:33.408561 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bbf64b7-7858-459d-9eea-302168bcc99d None None] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:04:33.408561 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bbf64b7-7858-459d-9eea-302168bcc99d None None] PATH_INFO: `/` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:04:33.409666 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 588/1193] 10.222.0.82 () {54 vars in 926 bytes} [Tue Aug 2 10:04:33 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 02 10:04:33.415257 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8360481a-fe6f-4394-b42a-5cf970de156a None None] REQUEST_METHOD: `POST` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:04:33.415681 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8360481a-fe6f-4394-b42a-5cf970de156a None None] SCRIPT_NAME: `/identity` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:04:33.415817 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8360481a-fe6f-4394-b42a-5cf970de156a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:04:33.433947 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.auth.core [None req-8360481a-fe6f-4394-b42a-5cf970de156a None None] MFA Rules not processed for user `891005fa5ff74041b75747d16cc66798`. Rule list: `[]` (Enabled: `True`). {{(pid=80377) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 02 10:04:33.435143 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.fernet_utils [None req-8360481a-fe6f-4394-b42a-5cf970de156a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80377) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 10:04:33.506884 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: [pid: 80377|app: 0|req: 606/1194] 10.222.0.82 () {58 vars in 1023 bytes} [Tue Aug 2 10:04:33 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Aug 02 10:04:33.530720 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bbf64b7-7858-459d-9eea-302168bcc99d None None] Authenticating user token {{(pid=80378) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:04:33.564056 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG dbcounter [-] [80378] Writing DB stats keystone:SELECT=100 {{(pid=80378) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 10:04:33.669404 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bbf64b7-7858-459d-9eea-302168bcc99d None None] Validating token access rules against request {{(pid=80378) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:04:33.727527 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de91a14f-5da9-488d-a5a4-dfae246e9990 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '891005fa5ff74041b75747d16cc66798', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80378) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:04:33.728602 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de91a14f-5da9-488d-a5a4-dfae246e9990 None admin] REQUEST_METHOD: `GET` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:04:33.729094 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de91a14f-5da9-488d-a5a4-dfae246e9990 None admin] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:04:33.729575 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de91a14f-5da9-488d-a5a4-dfae246e9990 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:04:33.750315 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de91a14f-5da9-488d-a5a4-dfae246e9990 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 02 10:04:33.760713 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application [None req-de91a14f-5da9-488d-a5a4-dfae246e9990 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 02 10:04:33.760713 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 02 10:04:33.760713 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 02 10:04:33.760713 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 02 10:04:33.760713 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 02 10:04:33.760713 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 02 10:04:33.760713 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 02 10:04:33.760713 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 02 10:04:33.760713 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 02 10:04:33.760713 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 02 10:04:33.760713 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 02 10:04:33.760713 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 02 10:04:33.760713 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 02 10:04:33.760713 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 02 10:04:33.760713 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 02 10:04:33.760713 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 02 10:04:33.762144 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 02 10:04:33.762144 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 02 10:04:33.762144 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 02 10:04:33.762144 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 02 10:04:33.762144 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 02 10:04:33.762144 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 02 10:04:33.762144 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 02 10:04:33.762144 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 02 10:04:33.762144 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 02 10:04:33.762144 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application return self.get_or_create( Aug 02 10:04:33.762144 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 02 10:04:33.762144 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application with Lock( Aug 02 10:04:33.762144 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 02 10:04:33.762144 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application return self._enter() Aug 02 10:04:33.762144 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 02 10:04:33.762144 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 02 10:04:33.770358 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 02 10:04:33.770358 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application return self.creator() Aug 02 10:04:33.770358 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 02 10:04:33.770358 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application created_value = creator( Aug 02 10:04:33.770358 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 02 10:04:33.770358 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 02 10:04:33.770358 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 02 10:04:33.770358 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 02 10:04:33.770358 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 02 10:04:33.770358 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 02 10:04:33.770358 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 02 10:04:33.770358 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application Aug 02 10:04:33.770358 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 589/1195] 10.222.0.82 () {56 vars in 1096 bytes} [Tue Aug 2 10:04:33 2022] GET /identity/v3/roles/admin => generated 83 bytes in 233 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 02 10:04:33.772016 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8360481a-fe6f-4394-b42a-5cf970de156a None None] Authenticating user token {{(pid=80377) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:04:33.901146 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8360481a-fe6f-4394-b42a-5cf970de156a None None] Validating token access rules against request {{(pid=80377) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:04:33.917122 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG dbcounter [-] [80377] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80377) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 10:04:33.940438 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ac9d85c-0ad5-4bf3-b97d-918143bbc2ba None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '891005fa5ff74041b75747d16cc66798', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80377) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:04:33.941007 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ac9d85c-0ad5-4bf3-b97d-918143bbc2ba None admin] REQUEST_METHOD: `GET` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:04:33.941290 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ac9d85c-0ad5-4bf3-b97d-918143bbc2ba None admin] SCRIPT_NAME: `/identity` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:04:33.941565 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ac9d85c-0ad5-4bf3-b97d-918143bbc2ba None admin] PATH_INFO: `/v3/roles` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:04:33.942324 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ac9d85c-0ad5-4bf3-b97d-918143bbc2ba None admin] RBAC: Adding query filter params (name=admin) {{(pid=80377) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 02 10:04:33.942630 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ac9d85c-0ad5-4bf3-b97d-918143bbc2ba None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:04:33.944003 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ac9d85c-0ad5-4bf3-b97d-918143bbc2ba None admin] RBAC: Authorization granted {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:04:33.952603 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: [pid: 80377|app: 0|req: 607/1196] 10.222.0.82 () {56 vars in 1099 bytes} [Tue Aug 2 10:04:33 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 182 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 02 10:04:33.960819 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de91a14f-5da9-488d-a5a4-dfae246e9990 None admin] Authenticating user token {{(pid=80378) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:04:34.089022 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG dbcounter [-] [80378] Writing DB stats keystone:SELECT=100 {{(pid=80378) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 10:04:34.105890 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de91a14f-5da9-488d-a5a4-dfae246e9990 None admin] Validating token access rules against request {{(pid=80378) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:04:34.153155 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-574f68db-96cf-4610-9517-c2cad92f1f16 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '891005fa5ff74041b75747d16cc66798', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80378) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:04:34.154192 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-574f68db-96cf-4610-9517-c2cad92f1f16 None admin] REQUEST_METHOD: `GET` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:04:34.154617 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-574f68db-96cf-4610-9517-c2cad92f1f16 None admin] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:04:34.154900 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-574f68db-96cf-4610-9517-c2cad92f1f16 None admin] PATH_INFO: `/v3/users/b5d506bd78e44c038492ac8d6a337d0d` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:04:34.156688 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-574f68db-96cf-4610-9517-c2cad92f1f16 None admin] RBAC: Authorizing `identity:get_user(user_id=b5d506bd78e44c038492ac8d6a337d0d)` {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:04:34.158724 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-574f68db-96cf-4610-9517-c2cad92f1f16 None admin] RBAC: Authorization granted {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:04:34.160542 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 590/1197] 10.222.0.82 () {56 vars in 1177 bytes} [Tue Aug 2 10:04:33 2022] GET /identity/v3/users/b5d506bd78e44c038492ac8d6a337d0d => generated 311 bytes in 202 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 02 10:04:34.169232 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ac9d85c-0ad5-4bf3-b97d-918143bbc2ba None admin] Authenticating user token {{(pid=80377) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:04:34.293547 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ac9d85c-0ad5-4bf3-b97d-918143bbc2ba None admin] Validating token access rules against request {{(pid=80377) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:04:34.328062 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2ce3c09-50d0-4026-ad95-7e4de3dfa706 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '891005fa5ff74041b75747d16cc66798', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80377) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:04:34.328691 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2ce3c09-50d0-4026-ad95-7e4de3dfa706 None admin] REQUEST_METHOD: `GET` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:04:34.328972 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2ce3c09-50d0-4026-ad95-7e4de3dfa706 None admin] SCRIPT_NAME: `/identity` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:04:34.329247 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2ce3c09-50d0-4026-ad95-7e4de3dfa706 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:04:34.379350 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG dbcounter [-] [80377] Writing DB stats keystone:SELECT=100 {{(pid=80377) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 10:04:34.408881 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2ce3c09-50d0-4026-ad95-7e4de3dfa706 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:04:34.411239 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2ce3c09-50d0-4026-ad95-7e4de3dfa706 None admin] RBAC: Authorization granted {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:04:34.548067 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: [pid: 80377|app: 0|req: 608/1198] 10.222.0.82 () {58 vars in 1282 bytes} [Tue Aug 2 10:04:34 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 379 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Aug 02 10:04:34.554532 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-574f68db-96cf-4610-9517-c2cad92f1f16 None admin] Authenticating user token {{(pid=80378) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:04:34.672403 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-574f68db-96cf-4610-9517-c2cad92f1f16 None admin] Validating token access rules against request {{(pid=80378) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:04:34.711648 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f389216-341d-4751-8e0b-1068840c32e5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '891005fa5ff74041b75747d16cc66798', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80378) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:04:34.712323 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f389216-341d-4751-8e0b-1068840c32e5 None admin] REQUEST_METHOD: `GET` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:04:34.712607 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f389216-341d-4751-8e0b-1068840c32e5 None admin] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:04:34.712887 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f389216-341d-4751-8e0b-1068840c32e5 None admin] PATH_INFO: `/v3/projects/c7080bd1bd284215bc7f023dbe7f17bf` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:04:34.714283 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f389216-341d-4751-8e0b-1068840c32e5 None admin] RBAC: Authorizing `identity:get_project(project_id=c7080bd1bd284215bc7f023dbe7f17bf)` {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:04:34.715799 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f389216-341d-4751-8e0b-1068840c32e5 None admin] RBAC: Authorization granted {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:04:34.717629 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 591/1199] 10.222.0.82 () {56 vars in 1186 bytes} [Tue Aug 2 10:04:34 2022] GET /identity/v3/projects/c7080bd1bd284215bc7f023dbe7f17bf => generated 361 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 02 10:04:34.724679 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2ce3c09-50d0-4026-ad95-7e4de3dfa706 None admin] Authenticating user token {{(pid=80377) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:04:34.812732 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG dbcounter [-] [80377] Writing DB stats keystone:SELECT=100 {{(pid=80377) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 10:04:34.843975 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2ce3c09-50d0-4026-ad95-7e4de3dfa706 None admin] Validating token access rules against request {{(pid=80377) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:04:34.883611 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-107b1a44-940f-4e1a-ae39-aa35bc265e4c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '891005fa5ff74041b75747d16cc66798', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80377) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:04:34.884245 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-107b1a44-940f-4e1a-ae39-aa35bc265e4c None admin] REQUEST_METHOD: `GET` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:04:34.884526 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-107b1a44-940f-4e1a-ae39-aa35bc265e4c None admin] SCRIPT_NAME: `/identity` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:04:34.884864 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-107b1a44-940f-4e1a-ae39-aa35bc265e4c None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:04:34.885668 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-107b1a44-940f-4e1a-ae39-aa35bc265e4c None admin] RBAC: Adding query filter params (role.id=237966442f05411191105bf336bc14f3, scope.project.id=c7080bd1bd284215bc7f023dbe7f17bf, user.id=b5d506bd78e44c038492ac8d6a337d0d) {{(pid=80377) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 02 10:04:34.886006 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-107b1a44-940f-4e1a-ae39-aa35bc265e4c None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:04:34.887454 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-107b1a44-940f-4e1a-ae39-aa35bc265e4c None admin] RBAC: Authorization granted {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:04:34.895755 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: [pid: 80377|app: 0|req: 609/1200] 10.222.0.82 () {56 vars in 1374 bytes} [Tue Aug 2 10:04:34 2022] GET /identity/v3/role_assignments?user.id=b5d506bd78e44c038492ac8d6a337d0d&scope.project.id=c7080bd1bd284215bc7f023dbe7f17bf&role.id=237966442f05411191105bf336bc14f3 => generated 608 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 02 10:04:44.706295 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG dbcounter [-] [80378] Writing DB stats keystone:SELECT=87 {{(pid=80378) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 10:04:44.891070 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG dbcounter [-] [80377] Writing DB stats keystone:SELECT=31 {{(pid=80377) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 10:05:35.485863 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3c7f850-3c1f-4380-8c83-34c313d47f4c None None] REQUEST_METHOD: `GET` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:05:35.487061 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3c7f850-3c1f-4380-8c83-34c313d47f4c None None] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:05:35.487061 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3c7f850-3c1f-4380-8c83-34c313d47f4c None None] PATH_INFO: `/` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:05:35.490200 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 592/1201] 10.222.0.82 () {54 vars in 926 bytes} [Tue Aug 2 10:05:35 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 02 10:05:35.504201 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-299d9ba6-79e0-4dee-bbfd-1b8c74a79750 None None] REQUEST_METHOD: `POST` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:05:35.507067 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-299d9ba6-79e0-4dee-bbfd-1b8c74a79750 None None] SCRIPT_NAME: `/identity` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:05:35.507067 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-299d9ba6-79e0-4dee-bbfd-1b8c74a79750 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:05:35.535815 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.auth.core [None req-299d9ba6-79e0-4dee-bbfd-1b8c74a79750 None None] MFA Rules not processed for user `891005fa5ff74041b75747d16cc66798`. Rule list: `[]` (Enabled: `True`). {{(pid=80377) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 02 10:05:35.538030 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.fernet_utils [None req-299d9ba6-79e0-4dee-bbfd-1b8c74a79750 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80377) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 10:05:35.676458 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: [pid: 80377|app: 0|req: 610/1202] 10.222.0.82 () {58 vars in 1023 bytes} [Tue Aug 2 10:05:35 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 174 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 02 10:05:35.705293 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3c7f850-3c1f-4380-8c83-34c313d47f4c None None] Authenticating user token {{(pid=80378) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:05:35.897013 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3c7f850-3c1f-4380-8c83-34c313d47f4c None None] Validating token access rules against request {{(pid=80378) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:05:35.976513 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fce690d-710f-43bb-8b7a-effe3065c496 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '891005fa5ff74041b75747d16cc66798', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80378) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:05:35.977454 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fce690d-710f-43bb-8b7a-effe3065c496 None admin] REQUEST_METHOD: `GET` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:05:35.977993 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fce690d-710f-43bb-8b7a-effe3065c496 None admin] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:05:35.978452 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fce690d-710f-43bb-8b7a-effe3065c496 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:05:36.002814 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1fce690d-710f-43bb-8b7a-effe3065c496 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:05:36.002814 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1fce690d-710f-43bb-8b7a-effe3065c496 None admin] RBAC: Authorization granted {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:05:36.033021 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application [None req-1fce690d-710f-43bb-8b7a-effe3065c496 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 02 10:05:36.033021 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 02 10:05:36.033021 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 02 10:05:36.033021 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 02 10:05:36.033021 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 02 10:05:36.033021 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 02 10:05:36.033021 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 02 10:05:36.033021 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 02 10:05:36.033021 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 02 10:05:36.033021 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 02 10:05:36.033021 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 02 10:05:36.033021 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 02 10:05:36.033021 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 02 10:05:36.033021 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 02 10:05:36.033021 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 02 10:05:36.035411 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 02 10:05:36.035411 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 02 10:05:36.035411 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 02 10:05:36.035411 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 02 10:05:36.035411 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 02 10:05:36.035411 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 02 10:05:36.035411 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application return self.get_or_create( Aug 02 10:05:36.035411 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 02 10:05:36.035411 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application with Lock( Aug 02 10:05:36.035411 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 02 10:05:36.035411 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application return self._enter() Aug 02 10:05:36.035411 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 02 10:05:36.035411 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 02 10:05:36.035411 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 02 10:05:36.035411 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application return self.creator() Aug 02 10:05:36.035411 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 02 10:05:36.035411 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application created_value = creator( Aug 02 10:05:36.038046 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 02 10:05:36.038046 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 02 10:05:36.038046 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 02 10:05:36.038046 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application Aug 02 10:05:36.040317 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 593/1203] 10.222.0.82 () {56 vars in 1108 bytes} [Tue Aug 2 10:05:35 2022] GET /identity/v3/domains/Default => generated 87 bytes in 336 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 02 10:05:36.051698 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-299d9ba6-79e0-4dee-bbfd-1b8c74a79750 None None] Authenticating user token {{(pid=80377) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:05:36.235071 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-299d9ba6-79e0-4dee-bbfd-1b8c74a79750 None None] Validating token access rules against request {{(pid=80377) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:05:36.297293 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG dbcounter [-] [80377] Writing DB stats keystone:SELECT=100 {{(pid=80377) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 10:05:36.306574 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-522799cc-7086-4190-9855-5ca2a8d91119 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '891005fa5ff74041b75747d16cc66798', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80377) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:05:36.307272 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-522799cc-7086-4190-9855-5ca2a8d91119 None admin] REQUEST_METHOD: `GET` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:05:36.307537 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-522799cc-7086-4190-9855-5ca2a8d91119 None admin] SCRIPT_NAME: `/identity` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:05:36.307796 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-522799cc-7086-4190-9855-5ca2a8d91119 None admin] PATH_INFO: `/v3/domains` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:05:36.308509 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-522799cc-7086-4190-9855-5ca2a8d91119 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80377) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 02 10:05:36.308967 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-522799cc-7086-4190-9855-5ca2a8d91119 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:05:36.310528 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-522799cc-7086-4190-9855-5ca2a8d91119 None admin] RBAC: Authorization granted {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:05:36.341999 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: [pid: 80377|app: 0|req: 611/1204] 10.222.0.82 () {56 vars in 1109 bytes} [Tue Aug 2 10:05:36 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 290 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 02 10:05:36.346627 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fce690d-710f-43bb-8b7a-effe3065c496 None admin] Authenticating user token {{(pid=80378) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:05:36.444921 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG dbcounter [-] [80378] Writing DB stats keystone:SELECT=100 {{(pid=80378) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 10:05:36.524378 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fce690d-710f-43bb-8b7a-effe3065c496 None admin] Validating token access rules against request {{(pid=80378) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:05:36.587176 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40581f27-1460-4753-b6cd-59e86b186184 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '891005fa5ff74041b75747d16cc66798', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80378) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:05:36.588036 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40581f27-1460-4753-b6cd-59e86b186184 None admin] REQUEST_METHOD: `POST` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:05:36.588408 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40581f27-1460-4753-b6cd-59e86b186184 None admin] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:05:36.588792 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40581f27-1460-4753-b6cd-59e86b186184 None admin] PATH_INFO: `/v3/users` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:05:36.589422 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40581f27-1460-4753-b6cd-59e86b186184 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:05:36.591074 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40581f27-1460-4753-b6cd-59e86b186184 None admin] RBAC: Authorization granted {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:05:36.629112 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 594/1205] 10.222.0.82 () {60 vars in 1132 bytes} [Tue Aug 2 10:05:36 2022] POST /identity/v3/users => generated 251 bytes in 276 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 02 10:05:39.109578 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec7f4039-aebc-44b7-b8df-f592e234290d None None] REQUEST_METHOD: `GET` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:05:39.110141 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec7f4039-aebc-44b7-b8df-f592e234290d None None] SCRIPT_NAME: `/identity` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:05:39.110141 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec7f4039-aebc-44b7-b8df-f592e234290d None None] PATH_INFO: `/` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:05:39.111643 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: [pid: 80377|app: 0|req: 612/1206] 10.222.0.82 () {54 vars in 926 bytes} [Tue Aug 2 10:05:39 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 02 10:05:39.117721 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f3e91bf-1613-41c3-ab91-c1439033c798 None None] REQUEST_METHOD: `POST` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:05:39.118304 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f3e91bf-1613-41c3-ab91-c1439033c798 None None] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:05:39.118304 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f3e91bf-1613-41c3-ab91-c1439033c798 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:05:39.141601 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.auth.core [None req-4f3e91bf-1613-41c3-ab91-c1439033c798 None None] MFA Rules not processed for user `891005fa5ff74041b75747d16cc66798`. Rule list: `[]` (Enabled: `True`). {{(pid=80378) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 02 10:05:39.143275 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.fernet_utils [None req-4f3e91bf-1613-41c3-ab91-c1439033c798 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80378) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 10:05:39.237979 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 595/1207] 10.222.0.82 () {58 vars in 1023 bytes} [Tue Aug 2 10:05:39 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 121 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 02 10:05:39.266185 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec7f4039-aebc-44b7-b8df-f592e234290d None None] Authenticating user token {{(pid=80377) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:05:39.463348 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec7f4039-aebc-44b7-b8df-f592e234290d None None] Validating token access rules against request {{(pid=80377) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:05:39.515117 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20c876a8-86d8-46ca-a0f5-97ea4fb5280c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '891005fa5ff74041b75747d16cc66798', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80377) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:05:39.516032 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20c876a8-86d8-46ca-a0f5-97ea4fb5280c None admin] REQUEST_METHOD: `GET` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:05:39.516346 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20c876a8-86d8-46ca-a0f5-97ea4fb5280c None admin] SCRIPT_NAME: `/identity` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:05:39.516847 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20c876a8-86d8-46ca-a0f5-97ea4fb5280c None admin] PATH_INFO: `/v3/roles/service` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:05:39.531989 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20c876a8-86d8-46ca-a0f5-97ea4fb5280c None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 02 10:05:39.540151 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application [None req-20c876a8-86d8-46ca-a0f5-97ea4fb5280c None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Aug 02 10:05:39.540151 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 02 10:05:39.540151 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 02 10:05:39.540151 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 02 10:05:39.540151 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 02 10:05:39.540151 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 02 10:05:39.540151 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 02 10:05:39.540151 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 02 10:05:39.540151 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 02 10:05:39.540151 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 02 10:05:39.540151 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 02 10:05:39.540151 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 02 10:05:39.540151 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 02 10:05:39.540151 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 02 10:05:39.540151 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 02 10:05:39.540151 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 02 10:05:39.541345 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 02 10:05:39.541345 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 02 10:05:39.541345 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 02 10:05:39.541345 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 02 10:05:39.541345 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 02 10:05:39.541345 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 02 10:05:39.541345 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 02 10:05:39.541345 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 02 10:05:39.541345 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 02 10:05:39.541345 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application return self.get_or_create( Aug 02 10:05:39.541345 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 02 10:05:39.541345 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application with Lock( Aug 02 10:05:39.541345 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 02 10:05:39.541345 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application return self._enter() Aug 02 10:05:39.541345 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 02 10:05:39.541345 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 02 10:05:39.542682 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 02 10:05:39.542682 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application return self.creator() Aug 02 10:05:39.542682 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 02 10:05:39.542682 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application created_value = creator( Aug 02 10:05:39.542682 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 02 10:05:39.542682 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 02 10:05:39.542682 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 02 10:05:39.542682 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 02 10:05:39.542682 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 02 10:05:39.542682 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 02 10:05:39.542682 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Aug 02 10:05:39.542682 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application Aug 02 10:05:39.542682 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: [pid: 80377|app: 0|req: 613/1208] 10.222.0.82 () {56 vars in 1102 bytes} [Tue Aug 2 10:05:39 2022] GET /identity/v3/roles/service => generated 85 bytes in 277 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 02 10:05:39.548964 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f3e91bf-1613-41c3-ab91-c1439033c798 None None] Authenticating user token {{(pid=80378) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:05:39.609819 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG dbcounter [-] [80378] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80378) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 10:05:39.702253 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f3e91bf-1613-41c3-ab91-c1439033c798 None None] Validating token access rules against request {{(pid=80378) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:05:39.749075 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-381f96f5-1d9d-497f-830d-7cbdaa161728 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '891005fa5ff74041b75747d16cc66798', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80378) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:05:39.749735 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-381f96f5-1d9d-497f-830d-7cbdaa161728 None admin] REQUEST_METHOD: `GET` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:05:39.750052 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-381f96f5-1d9d-497f-830d-7cbdaa161728 None admin] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:05:39.750331 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-381f96f5-1d9d-497f-830d-7cbdaa161728 None admin] PATH_INFO: `/v3/roles` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:05:39.751160 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-381f96f5-1d9d-497f-830d-7cbdaa161728 None admin] RBAC: Adding query filter params (name=service) {{(pid=80378) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 02 10:05:39.751489 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-381f96f5-1d9d-497f-830d-7cbdaa161728 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:05:39.752975 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-381f96f5-1d9d-497f-830d-7cbdaa161728 None admin] RBAC: Authorization granted {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:05:39.764861 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 596/1209] 10.222.0.82 () {56 vars in 1103 bytes} [Tue Aug 2 10:05:39 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 215 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 02 10:05:39.769361 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20c876a8-86d8-46ca-a0f5-97ea4fb5280c None admin] Authenticating user token {{(pid=80377) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:05:39.846317 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG dbcounter [-] [80377] Writing DB stats keystone:SELECT=100 {{(pid=80377) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 10:05:39.930407 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20c876a8-86d8-46ca-a0f5-97ea4fb5280c None admin] Validating token access rules against request {{(pid=80377) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:05:39.988679 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3da85251-bae4-403c-9df1-9d4a69ab58b8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '891005fa5ff74041b75747d16cc66798', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80377) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:05:39.989401 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3da85251-bae4-403c-9df1-9d4a69ab58b8 None admin] REQUEST_METHOD: `GET` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:05:39.989659 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3da85251-bae4-403c-9df1-9d4a69ab58b8 None admin] SCRIPT_NAME: `/identity` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:05:39.989985 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3da85251-bae4-403c-9df1-9d4a69ab58b8 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:05:40.009253 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3da85251-bae4-403c-9df1-9d4a69ab58b8 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:05:40.010777 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3da85251-bae4-403c-9df1-9d4a69ab58b8 None admin] RBAC: Authorization granted {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:05:40.037950 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application [None req-3da85251-bae4-403c-9df1-9d4a69ab58b8 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 02 10:05:40.037950 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 02 10:05:40.037950 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 02 10:05:40.037950 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 02 10:05:40.037950 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 02 10:05:40.037950 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 02 10:05:40.037950 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 02 10:05:40.037950 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 02 10:05:40.037950 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 02 10:05:40.037950 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 02 10:05:40.037950 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 02 10:05:40.037950 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 02 10:05:40.037950 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 02 10:05:40.037950 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 02 10:05:40.037950 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 02 10:05:40.039795 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 02 10:05:40.039795 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 02 10:05:40.039795 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 02 10:05:40.039795 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 02 10:05:40.039795 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 02 10:05:40.039795 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 02 10:05:40.039795 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application return self.get_or_create( Aug 02 10:05:40.039795 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 02 10:05:40.039795 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application with Lock( Aug 02 10:05:40.039795 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 02 10:05:40.039795 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application return self._enter() Aug 02 10:05:40.039795 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 02 10:05:40.039795 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 02 10:05:40.039795 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 02 10:05:40.039795 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application return self.creator() Aug 02 10:05:40.039795 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 02 10:05:40.039795 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application created_value = creator( Aug 02 10:05:40.040732 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 02 10:05:40.040732 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 02 10:05:40.040732 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 02 10:05:40.040732 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application Aug 02 10:05:40.041856 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: [pid: 80377|app: 0|req: 614/1210] 10.222.0.82 () {56 vars in 1108 bytes} [Tue Aug 2 10:05:39 2022] GET /identity/v3/domains/Default => generated 87 bytes in 273 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 02 10:05:40.045559 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-381f96f5-1d9d-497f-830d-7cbdaa161728 None admin] Authenticating user token {{(pid=80378) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:05:40.227071 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG dbcounter [-] [80378] Writing DB stats keystone:SELECT=100 {{(pid=80378) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 10:05:40.235401 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-381f96f5-1d9d-497f-830d-7cbdaa161728 None admin] Validating token access rules against request {{(pid=80378) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:05:40.303425 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2931b32c-c292-4299-a8bf-bbac7abb7ad1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '891005fa5ff74041b75747d16cc66798', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80378) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:05:40.304257 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2931b32c-c292-4299-a8bf-bbac7abb7ad1 None admin] REQUEST_METHOD: `GET` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:05:40.304662 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2931b32c-c292-4299-a8bf-bbac7abb7ad1 None admin] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:05:40.304976 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2931b32c-c292-4299-a8bf-bbac7abb7ad1 None admin] PATH_INFO: `/v3/domains` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:05:40.305857 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2931b32c-c292-4299-a8bf-bbac7abb7ad1 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80378) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 02 10:05:40.306240 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2931b32c-c292-4299-a8bf-bbac7abb7ad1 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:05:40.307780 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2931b32c-c292-4299-a8bf-bbac7abb7ad1 None admin] RBAC: Authorization granted {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:05:40.324414 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 597/1211] 10.222.0.82 () {56 vars in 1109 bytes} [Tue Aug 2 10:05:40 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 280 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 02 10:05:40.331166 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3da85251-bae4-403c-9df1-9d4a69ab58b8 None admin] Authenticating user token {{(pid=80377) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:05:40.473025 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3da85251-bae4-403c-9df1-9d4a69ab58b8 None admin] Validating token access rules against request {{(pid=80377) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:05:40.475761 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG dbcounter [-] [80377] Writing DB stats keystone:SELECT=100 {{(pid=80377) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 10:05:40.527911 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fec3c69-6f75-4be3-a7a3-ca26116a4539 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '891005fa5ff74041b75747d16cc66798', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80377) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:05:40.529156 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fec3c69-6f75-4be3-a7a3-ca26116a4539 None admin] REQUEST_METHOD: `GET` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:05:40.529740 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fec3c69-6f75-4be3-a7a3-ca26116a4539 None admin] SCRIPT_NAME: `/identity` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:05:40.530492 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fec3c69-6f75-4be3-a7a3-ca26116a4539 None admin] PATH_INFO: `/v3/users/placement` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:05:40.544969 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7fec3c69-6f75-4be3-a7a3-ca26116a4539 None admin] RBAC: Authorizing `identity:get_user(user_id=placement)` {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:05:40.546650 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7fec3c69-6f75-4be3-a7a3-ca26116a4539 None admin] RBAC: Authorization granted {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:05:40.565519 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application [None req-7fec3c69-6f75-4be3-a7a3-ca26116a4539 None admin] Could not find user: placement.: keystone.exception.UserNotFound: Could not find user: placement. Aug 02 10:05:40.565519 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 02 10:05:40.565519 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 02 10:05:40.565519 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 02 10:05:40.565519 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 02 10:05:40.565519 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 02 10:05:40.565519 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 02 10:05:40.565519 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 02 10:05:40.565519 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 02 10:05:40.565519 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 02 10:05:40.565519 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 02 10:05:40.565519 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 02 10:05:40.565519 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 02 10:05:40.565519 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 02 10:05:40.565519 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 02 10:05:40.565519 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 02 10:05:40.566801 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 02 10:05:40.566801 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 02 10:05:40.566801 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 02 10:05:40.566801 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 02 10:05:40.566801 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 02 10:05:40.566801 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 02 10:05:40.566801 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 02 10:05:40.566801 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 02 10:05:40.566801 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 02 10:05:40.566801 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application return self.get_or_create( Aug 02 10:05:40.566801 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 02 10:05:40.566801 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application with Lock( Aug 02 10:05:40.566801 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 02 10:05:40.566801 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application return self._enter() Aug 02 10:05:40.566801 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 02 10:05:40.566801 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 02 10:05:40.566801 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 02 10:05:40.566801 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application return self.creator() Aug 02 10:05:40.568549 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 02 10:05:40.568549 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application created_value = creator( Aug 02 10:05:40.568549 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 02 10:05:40.568549 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 02 10:05:40.568549 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 02 10:05:40.568549 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 02 10:05:40.568549 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 02 10:05:40.568549 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 02 10:05:40.568549 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: placement. Aug 02 10:05:40.568549 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application Aug 02 10:05:40.568549 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: [pid: 80377|app: 0|req: 615/1212] 10.222.0.82 () {56 vars in 1108 bytes} [Tue Aug 2 10:05:40 2022] GET /identity/v3/users/placement => generated 87 bytes in 236 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 02 10:05:40.575001 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2931b32c-c292-4299-a8bf-bbac7abb7ad1 None admin] Authenticating user token {{(pid=80378) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:05:40.767746 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2931b32c-c292-4299-a8bf-bbac7abb7ad1 None admin] Validating token access rules against request {{(pid=80378) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:05:40.828821 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70c37163-45fa-45fd-847f-3158d5c6e8cc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '891005fa5ff74041b75747d16cc66798', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80378) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:05:40.829386 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70c37163-45fa-45fd-847f-3158d5c6e8cc None admin] REQUEST_METHOD: `GET` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:05:40.829675 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70c37163-45fa-45fd-847f-3158d5c6e8cc None admin] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:05:40.829987 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70c37163-45fa-45fd-847f-3158d5c6e8cc None admin] PATH_INFO: `/v3/users` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:05:40.830935 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70c37163-45fa-45fd-847f-3158d5c6e8cc None admin] RBAC: Adding query filter params (domain_id=default, name=placement) {{(pid=80378) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 02 10:05:40.831223 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70c37163-45fa-45fd-847f-3158d5c6e8cc None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:05:40.832794 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70c37163-45fa-45fd-847f-3158d5c6e8cc None admin] RBAC: Authorization granted {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:05:40.862066 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 598/1213] 10.222.0.82 () {56 vars in 1143 bytes} [Tue Aug 2 10:05:40 2022] GET /identity/v3/users?domain_id=default&name=placement => generated 378 bytes in 286 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 02 10:05:40.866560 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fec3c69-6f75-4be3-a7a3-ca26116a4539 None admin] Authenticating user token {{(pid=80377) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:05:41.009754 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fec3c69-6f75-4be3-a7a3-ca26116a4539 None admin] Validating token access rules against request {{(pid=80377) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:05:41.053227 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a878199-d64d-49c0-ad65-5cb882f58d49 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '891005fa5ff74041b75747d16cc66798', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80377) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:05:41.054082 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a878199-d64d-49c0-ad65-5cb882f58d49 None admin] REQUEST_METHOD: `GET` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:05:41.054455 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a878199-d64d-49c0-ad65-5cb882f58d49 None admin] SCRIPT_NAME: `/identity` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:05:41.054785 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a878199-d64d-49c0-ad65-5cb882f58d49 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:05:41.097256 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG dbcounter [-] [80377] Writing DB stats keystone:SELECT=100 {{(pid=80377) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 10:05:41.150379 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a878199-d64d-49c0-ad65-5cb882f58d49 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:05:41.152989 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a878199-d64d-49c0-ad65-5cb882f58d49 None admin] RBAC: Authorization granted {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:05:41.314638 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: [pid: 80377|app: 0|req: 616/1214] 10.222.0.82 () {58 vars in 1282 bytes} [Tue Aug 2 10:05:40 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 450 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Aug 02 10:05:41.320832 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70c37163-45fa-45fd-847f-3158d5c6e8cc None admin] Authenticating user token {{(pid=80378) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:05:41.350433 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG dbcounter [-] [80378] Writing DB stats keystone:SELECT=100 {{(pid=80378) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 10:05:41.467555 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70c37163-45fa-45fd-847f-3158d5c6e8cc None admin] Validating token access rules against request {{(pid=80378) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:05:41.512669 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea7764b0-0733-4255-9236-ae517436761c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '891005fa5ff74041b75747d16cc66798', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80378) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:05:41.513417 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea7764b0-0733-4255-9236-ae517436761c None admin] REQUEST_METHOD: `GET` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:05:41.513653 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea7764b0-0733-4255-9236-ae517436761c None admin] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:05:41.513965 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea7764b0-0733-4255-9236-ae517436761c None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:05:41.526181 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea7764b0-0733-4255-9236-ae517436761c None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:05:41.527732 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea7764b0-0733-4255-9236-ae517436761c None admin] RBAC: Authorization granted {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:05:41.548651 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application [None req-ea7764b0-0733-4255-9236-ae517436761c None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 02 10:05:41.548651 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 02 10:05:41.548651 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 02 10:05:41.548651 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 02 10:05:41.548651 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 02 10:05:41.548651 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 02 10:05:41.548651 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 02 10:05:41.548651 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 02 10:05:41.548651 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 02 10:05:41.548651 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 02 10:05:41.548651 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 02 10:05:41.548651 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 02 10:05:41.548651 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 02 10:05:41.548651 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 02 10:05:41.548651 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 02 10:05:41.550557 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 02 10:05:41.550557 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 02 10:05:41.550557 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 02 10:05:41.550557 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 02 10:05:41.550557 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 02 10:05:41.550557 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 02 10:05:41.550557 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application return self.get_or_create( Aug 02 10:05:41.550557 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 02 10:05:41.550557 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application with Lock( Aug 02 10:05:41.550557 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 02 10:05:41.550557 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application return self._enter() Aug 02 10:05:41.550557 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 02 10:05:41.550557 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 02 10:05:41.550557 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 02 10:05:41.550557 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application return self.creator() Aug 02 10:05:41.550557 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 02 10:05:41.550557 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application created_value = creator( Aug 02 10:05:41.551495 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 02 10:05:41.551495 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 02 10:05:41.551495 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 02 10:05:41.551495 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application Aug 02 10:05:41.551495 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 599/1215] 10.222.0.82 () {56 vars in 1108 bytes} [Tue Aug 2 10:05:41 2022] GET /identity/v3/domains/Default => generated 87 bytes in 230 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 02 10:05:41.559438 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a878199-d64d-49c0-ad65-5cb882f58d49 None admin] Authenticating user token {{(pid=80377) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:05:41.655258 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG dbcounter [-] [80377] Writing DB stats keystone:SELECT=100 {{(pid=80377) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 10:05:41.710230 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a878199-d64d-49c0-ad65-5cb882f58d49 None admin] Validating token access rules against request {{(pid=80377) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:05:41.758182 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-088ba805-316f-42ed-9c8e-d10433960571 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '891005fa5ff74041b75747d16cc66798', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80377) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:05:41.759411 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-088ba805-316f-42ed-9c8e-d10433960571 None admin] REQUEST_METHOD: `GET` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:05:41.759939 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-088ba805-316f-42ed-9c8e-d10433960571 None admin] SCRIPT_NAME: `/identity` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:05:41.760427 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-088ba805-316f-42ed-9c8e-d10433960571 None admin] PATH_INFO: `/v3/domains` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:05:41.761490 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-088ba805-316f-42ed-9c8e-d10433960571 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80377) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 02 10:05:41.762048 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-088ba805-316f-42ed-9c8e-d10433960571 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:05:41.763856 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-088ba805-316f-42ed-9c8e-d10433960571 None admin] RBAC: Authorization granted {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:05:41.791657 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: [pid: 80377|app: 0|req: 617/1216] 10.222.0.82 () {56 vars in 1109 bytes} [Tue Aug 2 10:05:41 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 233 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 02 10:05:41.800100 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea7764b0-0733-4255-9236-ae517436761c None admin] Authenticating user token {{(pid=80378) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:05:41.935424 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG dbcounter [-] [80378] Writing DB stats keystone:SELECT=100 {{(pid=80378) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 10:05:41.993687 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea7764b0-0733-4255-9236-ae517436761c None admin] Validating token access rules against request {{(pid=80378) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:05:42.051685 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c161eae-0c0f-4c23-9852-41aecd1939ae None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '891005fa5ff74041b75747d16cc66798', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80378) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:05:42.053418 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c161eae-0c0f-4c23-9852-41aecd1939ae None admin] REQUEST_METHOD: `GET` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:05:42.053988 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c161eae-0c0f-4c23-9852-41aecd1939ae None admin] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:05:42.054332 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c161eae-0c0f-4c23-9852-41aecd1939ae None admin] PATH_INFO: `/v3/projects/service` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:05:42.062466 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c161eae-0c0f-4c23-9852-41aecd1939ae None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:05:42.063977 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c161eae-0c0f-4c23-9852-41aecd1939ae None admin] RBAC: Authorization granted {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:05:42.080339 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application [None req-8c161eae-0c0f-4c23-9852-41aecd1939ae None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 02 10:05:42.080339 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 02 10:05:42.080339 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 02 10:05:42.080339 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 02 10:05:42.080339 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 02 10:05:42.080339 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 02 10:05:42.080339 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 02 10:05:42.080339 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 02 10:05:42.080339 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 02 10:05:42.080339 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 02 10:05:42.080339 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 02 10:05:42.080339 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 02 10:05:42.080339 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 02 10:05:42.080339 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 02 10:05:42.080339 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 02 10:05:42.081647 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 02 10:05:42.081647 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 02 10:05:42.081647 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 02 10:05:42.081647 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 02 10:05:42.081647 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 02 10:05:42.081647 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 02 10:05:42.081647 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application return self.get_or_create( Aug 02 10:05:42.081647 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 02 10:05:42.081647 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application with Lock( Aug 02 10:05:42.081647 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 02 10:05:42.081647 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application return self._enter() Aug 02 10:05:42.081647 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 02 10:05:42.081647 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 02 10:05:42.081647 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 02 10:05:42.081647 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application return self.creator() Aug 02 10:05:42.081647 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 02 10:05:42.081647 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application created_value = creator( Aug 02 10:05:42.085728 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 02 10:05:42.085728 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 02 10:05:42.085728 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 02 10:05:42.085728 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 02 10:05:42.085728 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 02 10:05:42.085728 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 02 10:05:42.085728 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 02 10:05:42.085728 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application Aug 02 10:05:42.085728 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 600/1217] 10.222.0.82 () {56 vars in 1111 bytes} [Tue Aug 2 10:05:41 2022] GET /identity/v3/projects/service => generated 88 bytes in 283 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 02 10:05:42.095023 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-088ba805-316f-42ed-9c8e-d10433960571 None admin] Authenticating user token {{(pid=80377) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:05:42.253131 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-088ba805-316f-42ed-9c8e-d10433960571 None admin] Validating token access rules against request {{(pid=80377) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:05:42.283066 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG dbcounter [-] [80377] Writing DB stats keystone:SELECT=100 {{(pid=80377) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 10:05:42.301389 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f71552a6-2991-4e6a-88ac-efc83d3c298d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '891005fa5ff74041b75747d16cc66798', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80377) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:05:42.302359 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f71552a6-2991-4e6a-88ac-efc83d3c298d None admin] REQUEST_METHOD: `GET` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:05:42.302773 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f71552a6-2991-4e6a-88ac-efc83d3c298d None admin] SCRIPT_NAME: `/identity` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:05:42.303246 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f71552a6-2991-4e6a-88ac-efc83d3c298d None admin] PATH_INFO: `/v3/projects` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:05:42.304173 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f71552a6-2991-4e6a-88ac-efc83d3c298d None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80377) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 02 10:05:42.304710 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f71552a6-2991-4e6a-88ac-efc83d3c298d None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:05:42.306600 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f71552a6-2991-4e6a-88ac-efc83d3c298d None admin] RBAC: Authorization granted {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:05:42.330936 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: [pid: 80377|app: 0|req: 618/1218] 10.222.0.82 () {56 vars in 1148 bytes} [Tue Aug 2 10:05:42 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 231 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 02 10:05:42.333232 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c161eae-0c0f-4c23-9852-41aecd1939ae None admin] Authenticating user token {{(pid=80378) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:05:42.500864 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c161eae-0c0f-4c23-9852-41aecd1939ae None admin] Validating token access rules against request {{(pid=80378) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:05:42.535706 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG dbcounter [-] [80378] Writing DB stats keystone:SELECT=100 {{(pid=80378) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 10:05:42.545080 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6212e79a-8aa6-4b80-9cb0-cd867d507ce1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '891005fa5ff74041b75747d16cc66798', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80378) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:05:42.545762 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6212e79a-8aa6-4b80-9cb0-cd867d507ce1 None admin] REQUEST_METHOD: `GET` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:05:42.546256 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6212e79a-8aa6-4b80-9cb0-cd867d507ce1 None admin] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:05:42.546614 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6212e79a-8aa6-4b80-9cb0-cd867d507ce1 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:05:42.547708 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6212e79a-8aa6-4b80-9cb0-cd867d507ce1 None admin] RBAC: Adding query filter params (role.id=02003a0be60f47f2a8dfea17053af52c, scope.project.id=f1b38ebb804c41c69772c998aae72c41, user.id=ae6e910e50d74f29bfb89bf18d95a908) {{(pid=80378) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 02 10:05:42.548115 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6212e79a-8aa6-4b80-9cb0-cd867d507ce1 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:05:42.550018 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6212e79a-8aa6-4b80-9cb0-cd867d507ce1 None admin] RBAC: Authorization granted {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:05:42.574143 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 601/1219] 10.222.0.82 () {56 vars in 1374 bytes} [Tue Aug 2 10:05:42 2022] GET /identity/v3/role_assignments?user.id=ae6e910e50d74f29bfb89bf18d95a908&scope.project.id=f1b38ebb804c41c69772c998aae72c41&role.id=02003a0be60f47f2a8dfea17053af52c => generated 259 bytes in 242 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 02 10:05:44.880379 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4884834-60dd-4120-bc5b-aa4882ab1776 None None] REQUEST_METHOD: `GET` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:05:44.880927 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4884834-60dd-4120-bc5b-aa4882ab1776 None None] SCRIPT_NAME: `/identity` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:05:44.880927 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4884834-60dd-4120-bc5b-aa4882ab1776 None None] PATH_INFO: `/` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:05:44.882591 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: [pid: 80377|app: 0|req: 619/1220] 10.222.0.82 () {54 vars in 926 bytes} [Tue Aug 2 10:05:44 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 02 10:05:44.890019 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fd978ce-f456-4c16-b089-b7cfaed8323c None None] REQUEST_METHOD: `POST` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:05:44.890596 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fd978ce-f456-4c16-b089-b7cfaed8323c None None] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:05:44.890816 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fd978ce-f456-4c16-b089-b7cfaed8323c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:05:44.914519 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.auth.core [None req-5fd978ce-f456-4c16-b089-b7cfaed8323c None None] MFA Rules not processed for user `891005fa5ff74041b75747d16cc66798`. Rule list: `[]` (Enabled: `True`). {{(pid=80378) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 02 10:05:44.915777 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.fernet_utils [None req-5fd978ce-f456-4c16-b089-b7cfaed8323c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80378) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 10:05:45.013169 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 602/1221] 10.222.0.82 () {58 vars in 1023 bytes} [Tue Aug 2 10:05:44 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 125 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 02 10:05:45.036489 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4884834-60dd-4120-bc5b-aa4882ab1776 None None] Authenticating user token {{(pid=80377) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:05:45.171997 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4884834-60dd-4120-bc5b-aa4882ab1776 None None] Validating token access rules against request {{(pid=80377) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:05:45.219856 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a374d97-8a68-4d4c-a622-64383739a7d5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '891005fa5ff74041b75747d16cc66798', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80377) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:05:45.220500 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a374d97-8a68-4d4c-a622-64383739a7d5 None admin] REQUEST_METHOD: `GET` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:05:45.220775 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a374d97-8a68-4d4c-a622-64383739a7d5 None admin] SCRIPT_NAME: `/identity` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:05:45.221043 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a374d97-8a68-4d4c-a622-64383739a7d5 None admin] PATH_INFO: `/v3/roles/service` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:05:45.234024 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a374d97-8a68-4d4c-a622-64383739a7d5 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 02 10:05:45.239957 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application [None req-4a374d97-8a68-4d4c-a622-64383739a7d5 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Aug 02 10:05:45.239957 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 02 10:05:45.239957 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 02 10:05:45.239957 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 02 10:05:45.239957 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 02 10:05:45.239957 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 02 10:05:45.239957 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 02 10:05:45.239957 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 02 10:05:45.239957 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 02 10:05:45.239957 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 02 10:05:45.239957 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 02 10:05:45.239957 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 02 10:05:45.239957 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 02 10:05:45.239957 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 02 10:05:45.239957 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 02 10:05:45.239957 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 02 10:05:45.241941 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 02 10:05:45.241941 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 02 10:05:45.241941 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 02 10:05:45.241941 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 02 10:05:45.241941 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 02 10:05:45.241941 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 02 10:05:45.241941 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 02 10:05:45.241941 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 02 10:05:45.241941 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 02 10:05:45.241941 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application return self.get_or_create( Aug 02 10:05:45.241941 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 02 10:05:45.241941 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application with Lock( Aug 02 10:05:45.241941 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 02 10:05:45.241941 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application return self._enter() Aug 02 10:05:45.241941 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 02 10:05:45.241941 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 02 10:05:45.243535 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 02 10:05:45.243535 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application return self.creator() Aug 02 10:05:45.243535 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 02 10:05:45.243535 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application created_value = creator( Aug 02 10:05:45.243535 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 02 10:05:45.243535 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 02 10:05:45.243535 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 02 10:05:45.243535 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 02 10:05:45.243535 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 02 10:05:45.243535 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 02 10:05:45.243535 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Aug 02 10:05:45.243535 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application Aug 02 10:05:45.248300 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fd978ce-f456-4c16-b089-b7cfaed8323c None None] Authenticating user token {{(pid=80378) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:05:45.249958 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: [pid: 80377|app: 0|req: 620/1222] 10.222.0.82 () {56 vars in 1102 bytes} [Tue Aug 2 10:05:45 2022] GET /identity/v3/roles/service => generated 85 bytes in 214 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 02 10:05:45.362657 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fd978ce-f456-4c16-b089-b7cfaed8323c None None] Validating token access rules against request {{(pid=80378) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:05:45.388455 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG dbcounter [-] [80378] Writing DB stats keystone:SELECT=100 {{(pid=80378) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 10:05:45.402034 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57bd12a3-bc6c-4335-a511-aa0853a1b0fd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '891005fa5ff74041b75747d16cc66798', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80378) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:05:45.402730 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57bd12a3-bc6c-4335-a511-aa0853a1b0fd None admin] REQUEST_METHOD: `GET` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:05:45.403052 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57bd12a3-bc6c-4335-a511-aa0853a1b0fd None admin] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:05:45.403401 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57bd12a3-bc6c-4335-a511-aa0853a1b0fd None admin] PATH_INFO: `/v3/roles` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:05:45.404195 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57bd12a3-bc6c-4335-a511-aa0853a1b0fd None admin] RBAC: Adding query filter params (name=service) {{(pid=80378) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 02 10:05:45.404534 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57bd12a3-bc6c-4335-a511-aa0853a1b0fd None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:05:45.406101 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57bd12a3-bc6c-4335-a511-aa0853a1b0fd None admin] RBAC: Authorization granted {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:05:45.416208 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 603/1223] 10.222.0.82 () {56 vars in 1103 bytes} [Tue Aug 2 10:05:45 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 02 10:05:45.422912 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a374d97-8a68-4d4c-a622-64383739a7d5 None admin] Authenticating user token {{(pid=80377) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:05:45.474821 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG dbcounter [-] [80377] Writing DB stats keystone:SELECT=100 {{(pid=80377) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 10:05:45.541871 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a374d97-8a68-4d4c-a622-64383739a7d5 None admin] Validating token access rules against request {{(pid=80377) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:05:45.590455 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a27fa873-0209-4e50-b053-5354c6ef71bb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '891005fa5ff74041b75747d16cc66798', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80377) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:05:45.591111 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a27fa873-0209-4e50-b053-5354c6ef71bb None admin] REQUEST_METHOD: `GET` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:05:45.591385 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a27fa873-0209-4e50-b053-5354c6ef71bb None admin] SCRIPT_NAME: `/identity` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:05:45.591659 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a27fa873-0209-4e50-b053-5354c6ef71bb None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:05:45.604012 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a27fa873-0209-4e50-b053-5354c6ef71bb None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:05:45.605500 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a27fa873-0209-4e50-b053-5354c6ef71bb None admin] RBAC: Authorization granted {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:05:45.623208 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application [None req-a27fa873-0209-4e50-b053-5354c6ef71bb None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 02 10:05:45.623208 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 02 10:05:45.623208 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 02 10:05:45.623208 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 02 10:05:45.623208 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 02 10:05:45.623208 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 02 10:05:45.623208 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 02 10:05:45.623208 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 02 10:05:45.623208 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 02 10:05:45.623208 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 02 10:05:45.623208 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 02 10:05:45.623208 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 02 10:05:45.623208 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 02 10:05:45.623208 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 02 10:05:45.623208 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 02 10:05:45.624329 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 02 10:05:45.624329 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 02 10:05:45.624329 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 02 10:05:45.624329 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 02 10:05:45.624329 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 02 10:05:45.624329 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 02 10:05:45.624329 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application return self.get_or_create( Aug 02 10:05:45.624329 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 02 10:05:45.624329 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application with Lock( Aug 02 10:05:45.624329 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 02 10:05:45.624329 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application return self._enter() Aug 02 10:05:45.624329 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 02 10:05:45.624329 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 02 10:05:45.624329 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 02 10:05:45.624329 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application return self.creator() Aug 02 10:05:45.624329 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 02 10:05:45.624329 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application created_value = creator( Aug 02 10:05:45.625369 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 02 10:05:45.625369 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 02 10:05:45.625369 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 02 10:05:45.625369 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application Aug 02 10:05:45.625369 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: [pid: 80377|app: 0|req: 621/1224] 10.222.0.82 () {56 vars in 1108 bytes} [Tue Aug 2 10:05:45 2022] GET /identity/v3/domains/Default => generated 87 bytes in 202 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 02 10:05:45.631520 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57bd12a3-bc6c-4335-a511-aa0853a1b0fd None admin] Authenticating user token {{(pid=80378) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:05:45.755358 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57bd12a3-bc6c-4335-a511-aa0853a1b0fd None admin] Validating token access rules against request {{(pid=80378) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:05:45.805261 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d4c5fda-2f9f-47de-b2a5-7bb6561cd165 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '891005fa5ff74041b75747d16cc66798', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80378) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:05:45.805963 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d4c5fda-2f9f-47de-b2a5-7bb6561cd165 None admin] REQUEST_METHOD: `GET` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:05:45.806310 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d4c5fda-2f9f-47de-b2a5-7bb6561cd165 None admin] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:05:45.806657 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d4c5fda-2f9f-47de-b2a5-7bb6561cd165 None admin] PATH_INFO: `/v3/domains` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:05:45.807523 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d4c5fda-2f9f-47de-b2a5-7bb6561cd165 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80378) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 02 10:05:45.807830 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d4c5fda-2f9f-47de-b2a5-7bb6561cd165 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:05:45.809223 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d4c5fda-2f9f-47de-b2a5-7bb6561cd165 None admin] RBAC: Authorization granted {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:05:45.823570 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 604/1225] 10.222.0.82 () {56 vars in 1109 bytes} [Tue Aug 2 10:05:45 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 193 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 02 10:05:45.829117 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a27fa873-0209-4e50-b053-5354c6ef71bb None admin] Authenticating user token {{(pid=80377) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:05:45.946103 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG dbcounter [-] [80377] Writing DB stats keystone:SELECT=100 {{(pid=80377) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 10:05:45.953669 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a27fa873-0209-4e50-b053-5354c6ef71bb None admin] Validating token access rules against request {{(pid=80377) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:05:45.992273 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d548347-83a9-4b0b-b034-793aec8e0638 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '891005fa5ff74041b75747d16cc66798', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80377) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:05:45.993306 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d548347-83a9-4b0b-b034-793aec8e0638 None admin] REQUEST_METHOD: `GET` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:05:45.993606 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d548347-83a9-4b0b-b034-793aec8e0638 None admin] SCRIPT_NAME: `/identity` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:05:45.993986 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d548347-83a9-4b0b-b034-793aec8e0638 None admin] PATH_INFO: `/v3/users/placement` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:05:46.003004 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d548347-83a9-4b0b-b034-793aec8e0638 None admin] RBAC: Authorizing `identity:get_user(user_id=placement)` {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:05:46.004620 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d548347-83a9-4b0b-b034-793aec8e0638 None admin] RBAC: Authorization granted {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:05:46.017337 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application [None req-5d548347-83a9-4b0b-b034-793aec8e0638 None admin] Could not find user: placement.: keystone.exception.UserNotFound: Could not find user: placement. Aug 02 10:05:46.017337 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 02 10:05:46.017337 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 02 10:05:46.017337 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 02 10:05:46.017337 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 02 10:05:46.017337 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 02 10:05:46.017337 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 02 10:05:46.017337 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 02 10:05:46.017337 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 02 10:05:46.017337 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 02 10:05:46.017337 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 02 10:05:46.017337 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 02 10:05:46.017337 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 02 10:05:46.017337 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 02 10:05:46.017337 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 02 10:05:46.017337 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 02 10:05:46.019283 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 02 10:05:46.019283 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 02 10:05:46.019283 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 02 10:05:46.019283 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 02 10:05:46.019283 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 02 10:05:46.019283 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 02 10:05:46.019283 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 02 10:05:46.019283 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 02 10:05:46.019283 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 02 10:05:46.019283 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application return self.get_or_create( Aug 02 10:05:46.019283 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 02 10:05:46.019283 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application with Lock( Aug 02 10:05:46.019283 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 02 10:05:46.019283 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application return self._enter() Aug 02 10:05:46.019283 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 02 10:05:46.019283 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 02 10:05:46.019283 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 02 10:05:46.019283 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application return self.creator() Aug 02 10:05:46.021157 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 02 10:05:46.021157 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application created_value = creator( Aug 02 10:05:46.021157 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 02 10:05:46.021157 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 02 10:05:46.021157 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 02 10:05:46.021157 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 02 10:05:46.021157 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 02 10:05:46.021157 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 02 10:05:46.021157 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: placement. Aug 02 10:05:46.021157 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application Aug 02 10:05:46.021157 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: [pid: 80377|app: 0|req: 622/1226] 10.222.0.82 () {56 vars in 1108 bytes} [Tue Aug 2 10:05:45 2022] GET /identity/v3/users/placement => generated 87 bytes in 190 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 02 10:05:46.028899 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d4c5fda-2f9f-47de-b2a5-7bb6561cd165 None admin] Authenticating user token {{(pid=80378) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:05:46.090380 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG dbcounter [-] [80378] Writing DB stats keystone:SELECT=100 {{(pid=80378) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 10:05:46.157799 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d4c5fda-2f9f-47de-b2a5-7bb6561cd165 None admin] Validating token access rules against request {{(pid=80378) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:05:46.203606 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-633c8969-0caf-4bf6-9763-61ec4162e5e6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '891005fa5ff74041b75747d16cc66798', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80378) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:05:46.204286 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-633c8969-0caf-4bf6-9763-61ec4162e5e6 None admin] REQUEST_METHOD: `GET` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:05:46.204595 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-633c8969-0caf-4bf6-9763-61ec4162e5e6 None admin] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:05:46.204910 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-633c8969-0caf-4bf6-9763-61ec4162e5e6 None admin] PATH_INFO: `/v3/users` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:05:46.205970 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-633c8969-0caf-4bf6-9763-61ec4162e5e6 None admin] RBAC: Adding query filter params (domain_id=default, name=placement) {{(pid=80378) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 02 10:05:46.206381 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-633c8969-0caf-4bf6-9763-61ec4162e5e6 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:05:46.208114 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-633c8969-0caf-4bf6-9763-61ec4162e5e6 None admin] RBAC: Authorization granted {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:05:46.241580 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 605/1227] 10.222.0.82 () {56 vars in 1143 bytes} [Tue Aug 2 10:05:46 2022] GET /identity/v3/users?domain_id=default&name=placement => generated 378 bytes in 214 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 02 10:05:46.247617 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d548347-83a9-4b0b-b034-793aec8e0638 None admin] Authenticating user token {{(pid=80377) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:05:46.366711 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d548347-83a9-4b0b-b034-793aec8e0638 None admin] Validating token access rules against request {{(pid=80377) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:05:46.408361 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dcaa1e99-6096-463d-82c2-280ca522d6bb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '891005fa5ff74041b75747d16cc66798', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80377) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:05:46.409016 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcaa1e99-6096-463d-82c2-280ca522d6bb None admin] REQUEST_METHOD: `GET` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:05:46.409303 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcaa1e99-6096-463d-82c2-280ca522d6bb None admin] SCRIPT_NAME: `/identity` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:05:46.409624 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcaa1e99-6096-463d-82c2-280ca522d6bb None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:05:46.425204 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dcaa1e99-6096-463d-82c2-280ca522d6bb None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:05:46.427083 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dcaa1e99-6096-463d-82c2-280ca522d6bb None admin] RBAC: Authorization granted {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:05:46.456756 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application [None req-dcaa1e99-6096-463d-82c2-280ca522d6bb None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 02 10:05:46.456756 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 02 10:05:46.456756 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 02 10:05:46.456756 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 02 10:05:46.456756 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 02 10:05:46.456756 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 02 10:05:46.456756 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 02 10:05:46.456756 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 02 10:05:46.456756 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 02 10:05:46.456756 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 02 10:05:46.456756 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 02 10:05:46.456756 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 02 10:05:46.456756 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 02 10:05:46.456756 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 02 10:05:46.456756 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 02 10:05:46.458085 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 02 10:05:46.458085 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 02 10:05:46.458085 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 02 10:05:46.458085 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 02 10:05:46.458085 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 02 10:05:46.458085 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 02 10:05:46.458085 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application return self.get_or_create( Aug 02 10:05:46.458085 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 02 10:05:46.458085 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application with Lock( Aug 02 10:05:46.458085 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 02 10:05:46.458085 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application return self._enter() Aug 02 10:05:46.458085 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 02 10:05:46.458085 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 02 10:05:46.458085 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 02 10:05:46.458085 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application return self.creator() Aug 02 10:05:46.458085 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 02 10:05:46.458085 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application created_value = creator( Aug 02 10:05:46.460336 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 02 10:05:46.460336 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 02 10:05:46.460336 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 02 10:05:46.460336 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application Aug 02 10:05:46.462342 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: [pid: 80377|app: 0|req: 623/1228] 10.222.0.82 () {56 vars in 1108 bytes} [Tue Aug 2 10:05:46 2022] GET /identity/v3/domains/Default => generated 87 bytes in 215 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 02 10:05:46.473175 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-633c8969-0caf-4bf6-9763-61ec4162e5e6 None admin] Authenticating user token {{(pid=80378) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:05:46.625747 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-633c8969-0caf-4bf6-9763-61ec4162e5e6 None admin] Validating token access rules against request {{(pid=80378) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:05:46.629001 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG dbcounter [-] [80378] Writing DB stats keystone:SELECT=100 {{(pid=80378) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 10:05:46.667105 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39aa54c2-0410-4107-9fdf-752273d93557 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '891005fa5ff74041b75747d16cc66798', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80378) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:05:46.668250 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39aa54c2-0410-4107-9fdf-752273d93557 None admin] REQUEST_METHOD: `GET` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:05:46.668833 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39aa54c2-0410-4107-9fdf-752273d93557 None admin] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:05:46.669380 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39aa54c2-0410-4107-9fdf-752273d93557 None admin] PATH_INFO: `/v3/domains` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:05:46.670661 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39aa54c2-0410-4107-9fdf-752273d93557 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80378) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 02 10:05:46.671118 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39aa54c2-0410-4107-9fdf-752273d93557 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:05:46.673093 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39aa54c2-0410-4107-9fdf-752273d93557 None admin] RBAC: Authorization granted {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:05:46.694583 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 606/1229] 10.222.0.82 () {56 vars in 1109 bytes} [Tue Aug 2 10:05:46 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 222 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 02 10:05:46.703527 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dcaa1e99-6096-463d-82c2-280ca522d6bb None admin] Authenticating user token {{(pid=80377) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:05:46.715611 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG dbcounter [-] [80377] Writing DB stats keystone:SELECT=100 {{(pid=80377) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 10:05:46.827102 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dcaa1e99-6096-463d-82c2-280ca522d6bb None admin] Validating token access rules against request {{(pid=80377) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:05:46.873637 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e898202-9f54-4c53-a444-79946849d264 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '891005fa5ff74041b75747d16cc66798', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80377) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:05:46.874870 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e898202-9f54-4c53-a444-79946849d264 None admin] REQUEST_METHOD: `GET` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:05:46.875320 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e898202-9f54-4c53-a444-79946849d264 None admin] SCRIPT_NAME: `/identity` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:05:46.875971 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e898202-9f54-4c53-a444-79946849d264 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:05:46.888978 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e898202-9f54-4c53-a444-79946849d264 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:05:46.891721 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e898202-9f54-4c53-a444-79946849d264 None admin] RBAC: Authorization granted {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:05:46.905659 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application [None req-8e898202-9f54-4c53-a444-79946849d264 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 02 10:05:46.905659 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 02 10:05:46.905659 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 02 10:05:46.905659 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 02 10:05:46.905659 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 02 10:05:46.905659 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 02 10:05:46.905659 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 02 10:05:46.905659 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 02 10:05:46.905659 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 02 10:05:46.905659 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 02 10:05:46.905659 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 02 10:05:46.905659 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 02 10:05:46.905659 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 02 10:05:46.905659 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 02 10:05:46.905659 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 02 10:05:46.907024 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 02 10:05:46.907024 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 02 10:05:46.907024 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 02 10:05:46.907024 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 02 10:05:46.907024 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 02 10:05:46.907024 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 02 10:05:46.907024 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application return self.get_or_create( Aug 02 10:05:46.907024 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 02 10:05:46.907024 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application with Lock( Aug 02 10:05:46.907024 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 02 10:05:46.907024 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application return self._enter() Aug 02 10:05:46.907024 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 02 10:05:46.907024 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 02 10:05:46.907024 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 02 10:05:46.907024 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application return self.creator() Aug 02 10:05:46.907024 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 02 10:05:46.907024 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application created_value = creator( Aug 02 10:05:46.908019 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 02 10:05:46.908019 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 02 10:05:46.908019 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 02 10:05:46.908019 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 02 10:05:46.908019 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 02 10:05:46.908019 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 02 10:05:46.908019 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 02 10:05:46.908019 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application Aug 02 10:05:46.908019 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: [pid: 80377|app: 0|req: 624/1230] 10.222.0.82 () {56 vars in 1111 bytes} [Tue Aug 2 10:05:46 2022] GET /identity/v3/projects/service => generated 88 bytes in 204 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 02 10:05:46.915809 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39aa54c2-0410-4107-9fdf-752273d93557 None admin] Authenticating user token {{(pid=80378) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:05:47.037810 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39aa54c2-0410-4107-9fdf-752273d93557 None admin] Validating token access rules against request {{(pid=80378) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:05:47.075700 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3353c37-3052-495f-9341-2497bd24cbb9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '891005fa5ff74041b75747d16cc66798', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80378) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:05:47.076329 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3353c37-3052-495f-9341-2497bd24cbb9 None admin] REQUEST_METHOD: `GET` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:05:47.076612 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3353c37-3052-495f-9341-2497bd24cbb9 None admin] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:05:47.076908 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3353c37-3052-495f-9341-2497bd24cbb9 None admin] PATH_INFO: `/v3/projects` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:05:47.077635 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3353c37-3052-495f-9341-2497bd24cbb9 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80378) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 02 10:05:47.077981 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3353c37-3052-495f-9341-2497bd24cbb9 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:05:47.079458 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3353c37-3052-495f-9341-2497bd24cbb9 None admin] RBAC: Authorization granted {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:05:47.093648 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 607/1231] 10.222.0.82 () {56 vars in 1148 bytes} [Tue Aug 2 10:05:46 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 179 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 02 10:05:47.100544 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e898202-9f54-4c53-a444-79946849d264 None admin] Authenticating user token {{(pid=80377) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:05:47.179030 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG dbcounter [-] [80377] Writing DB stats keystone:SELECT=100 {{(pid=80377) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 10:05:47.218562 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e898202-9f54-4c53-a444-79946849d264 None admin] Validating token access rules against request {{(pid=80377) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:05:47.258811 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0f691c2-a709-42a0-b3b3-4970eed14279 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '891005fa5ff74041b75747d16cc66798', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80377) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:05:47.259502 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0f691c2-a709-42a0-b3b3-4970eed14279 None admin] REQUEST_METHOD: `PUT` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:05:47.259989 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0f691c2-a709-42a0-b3b3-4970eed14279 None admin] SCRIPT_NAME: `/identity` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:05:47.260301 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0f691c2-a709-42a0-b3b3-4970eed14279 None admin] PATH_INFO: `/v3/projects/f1b38ebb804c41c69772c998aae72c41/users/ae6e910e50d74f29bfb89bf18d95a908/roles/02003a0be60f47f2a8dfea17053af52c` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:05:47.278272 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0f691c2-a709-42a0-b3b3-4970eed14279 None admin] RBAC: Authorizing `identity:create_grant(project_id=f1b38ebb804c41c69772c998aae72c41, user_id=ae6e910e50d74f29bfb89bf18d95a908, role_id=02003a0be60f47f2a8dfea17053af52c)` {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:05:47.279759 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0f691c2-a709-42a0-b3b3-4970eed14279 None admin] RBAC: Authorization granted {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:05:47.301023 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: [pid: 80377|app: 0|req: 625/1232] 10.222.0.82 () {58 vars in 1439 bytes} [Tue Aug 2 10:05:47 2022] PUT /identity/v3/projects/f1b38ebb804c41c69772c998aae72c41/users/ae6e910e50d74f29bfb89bf18d95a908/roles/02003a0be60f47f2a8dfea17053af52c => generated 0 bytes in 202 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 02 10:05:49.408834 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd12dd61-0b28-4389-b237-bde7002bc128 None None] REQUEST_METHOD: `GET` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:05:49.409483 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd12dd61-0b28-4389-b237-bde7002bc128 None None] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:05:49.409483 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd12dd61-0b28-4389-b237-bde7002bc128 None None] PATH_INFO: `/` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:05:49.410890 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 608/1233] 10.222.0.82 () {54 vars in 926 bytes} [Tue Aug 2 10:05:49 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 02 10:05:49.417517 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28db4086-8b25-4d95-8971-c4899c2b19e2 None None] REQUEST_METHOD: `POST` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:05:49.418397 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28db4086-8b25-4d95-8971-c4899c2b19e2 None None] SCRIPT_NAME: `/identity` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:05:49.418824 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28db4086-8b25-4d95-8971-c4899c2b19e2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:05:49.440547 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.auth.core [None req-28db4086-8b25-4d95-8971-c4899c2b19e2 None None] MFA Rules not processed for user `891005fa5ff74041b75747d16cc66798`. Rule list: `[]` (Enabled: `True`). {{(pid=80377) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 02 10:05:49.442352 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.fernet_utils [None req-28db4086-8b25-4d95-8971-c4899c2b19e2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80377) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 10:05:49.560728 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: [pid: 80377|app: 0|req: 626/1234] 10.222.0.82 () {58 vars in 1023 bytes} [Tue Aug 2 10:05:49 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 144 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 02 10:05:49.585145 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd12dd61-0b28-4389-b237-bde7002bc128 None None] Authenticating user token {{(pid=80378) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:05:49.612999 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG dbcounter [-] [80378] Writing DB stats keystone:SELECT=100 {{(pid=80378) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 10:05:49.710795 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd12dd61-0b28-4389-b237-bde7002bc128 None None] Validating token access rules against request {{(pid=80378) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:05:49.751855 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8ad3b14-5be3-45ab-bd25-eb5e71fc82dc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '891005fa5ff74041b75747d16cc66798', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80378) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:05:49.752546 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8ad3b14-5be3-45ab-bd25-eb5e71fc82dc None admin] REQUEST_METHOD: `GET` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:05:49.752830 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8ad3b14-5be3-45ab-bd25-eb5e71fc82dc None admin] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:05:49.753101 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8ad3b14-5be3-45ab-bd25-eb5e71fc82dc None admin] PATH_INFO: `/v3/roles/service` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:05:49.764113 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8ad3b14-5be3-45ab-bd25-eb5e71fc82dc None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 02 10:05:49.769066 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application [None req-c8ad3b14-5be3-45ab-bd25-eb5e71fc82dc None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Aug 02 10:05:49.769066 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 02 10:05:49.769066 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 02 10:05:49.769066 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 02 10:05:49.769066 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 02 10:05:49.769066 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 02 10:05:49.769066 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 02 10:05:49.769066 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 02 10:05:49.769066 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 02 10:05:49.769066 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 02 10:05:49.769066 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 02 10:05:49.769066 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 02 10:05:49.769066 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 02 10:05:49.769066 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 02 10:05:49.769066 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 02 10:05:49.769066 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 02 10:05:49.771239 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 02 10:05:49.771239 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 02 10:05:49.771239 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 02 10:05:49.771239 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 02 10:05:49.771239 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 02 10:05:49.771239 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 02 10:05:49.771239 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 02 10:05:49.771239 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 02 10:05:49.771239 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 02 10:05:49.771239 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application return self.get_or_create( Aug 02 10:05:49.771239 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 02 10:05:49.771239 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application with Lock( Aug 02 10:05:49.771239 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 02 10:05:49.771239 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application return self._enter() Aug 02 10:05:49.771239 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 02 10:05:49.771239 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 02 10:05:49.772996 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 02 10:05:49.772996 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application return self.creator() Aug 02 10:05:49.772996 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 02 10:05:49.772996 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application created_value = creator( Aug 02 10:05:49.772996 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 02 10:05:49.772996 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 02 10:05:49.772996 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 02 10:05:49.772996 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 02 10:05:49.772996 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 02 10:05:49.772996 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 02 10:05:49.772996 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Aug 02 10:05:49.772996 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application Aug 02 10:05:49.772996 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 609/1235] 10.222.0.82 () {56 vars in 1102 bytes} [Tue Aug 2 10:05:49 2022] GET /identity/v3/roles/service => generated 85 bytes in 186 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 02 10:05:49.780852 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28db4086-8b25-4d95-8971-c4899c2b19e2 None None] Authenticating user token {{(pid=80377) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:05:49.832204 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG dbcounter [-] [80377] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80377) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 10:05:49.896377 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28db4086-8b25-4d95-8971-c4899c2b19e2 None None] Validating token access rules against request {{(pid=80377) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:05:49.935427 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bee59509-4495-4140-a498-86b220dca3e4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '891005fa5ff74041b75747d16cc66798', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80377) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:05:49.936322 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bee59509-4495-4140-a498-86b220dca3e4 None admin] REQUEST_METHOD: `GET` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:05:49.936692 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bee59509-4495-4140-a498-86b220dca3e4 None admin] SCRIPT_NAME: `/identity` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:05:49.936883 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bee59509-4495-4140-a498-86b220dca3e4 None admin] PATH_INFO: `/v3/roles` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:05:49.937625 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bee59509-4495-4140-a498-86b220dca3e4 None admin] RBAC: Adding query filter params (name=service) {{(pid=80377) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 02 10:05:49.937961 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bee59509-4495-4140-a498-86b220dca3e4 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:05:49.939465 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bee59509-4495-4140-a498-86b220dca3e4 None admin] RBAC: Authorization granted {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:05:49.948333 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: [pid: 80377|app: 0|req: 627/1236] 10.222.0.82 () {56 vars in 1103 bytes} [Tue Aug 2 10:05:49 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 02 10:05:49.954972 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8ad3b14-5be3-45ab-bd25-eb5e71fc82dc None admin] Authenticating user token {{(pid=80378) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:05:50.057557 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG dbcounter [-] [80378] Writing DB stats keystone:SELECT=100 {{(pid=80378) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 10:05:50.079388 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8ad3b14-5be3-45ab-bd25-eb5e71fc82dc None admin] Validating token access rules against request {{(pid=80378) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:05:50.118584 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-597e5945-7806-4017-b598-20aa06815ccb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '891005fa5ff74041b75747d16cc66798', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80378) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:05:50.119235 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-597e5945-7806-4017-b598-20aa06815ccb None admin] REQUEST_METHOD: `GET` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:05:50.119539 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-597e5945-7806-4017-b598-20aa06815ccb None admin] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:05:50.119788 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-597e5945-7806-4017-b598-20aa06815ccb None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:05:50.131779 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-597e5945-7806-4017-b598-20aa06815ccb None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:05:50.133450 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-597e5945-7806-4017-b598-20aa06815ccb None admin] RBAC: Authorization granted {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:05:50.154123 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application [None req-597e5945-7806-4017-b598-20aa06815ccb None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 02 10:05:50.154123 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 02 10:05:50.154123 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 02 10:05:50.154123 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 02 10:05:50.154123 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 02 10:05:50.154123 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 02 10:05:50.154123 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 02 10:05:50.154123 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 02 10:05:50.154123 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 02 10:05:50.154123 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 02 10:05:50.154123 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 02 10:05:50.154123 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 02 10:05:50.154123 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 02 10:05:50.154123 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 02 10:05:50.154123 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 02 10:05:50.155284 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 02 10:05:50.155284 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 02 10:05:50.155284 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 02 10:05:50.155284 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 02 10:05:50.155284 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 02 10:05:50.155284 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 02 10:05:50.155284 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application return self.get_or_create( Aug 02 10:05:50.155284 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 02 10:05:50.155284 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application with Lock( Aug 02 10:05:50.155284 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 02 10:05:50.155284 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application return self._enter() Aug 02 10:05:50.155284 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 02 10:05:50.155284 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 02 10:05:50.155284 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 02 10:05:50.155284 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application return self.creator() Aug 02 10:05:50.155284 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 02 10:05:50.155284 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application created_value = creator( Aug 02 10:05:50.156317 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 02 10:05:50.156317 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 02 10:05:50.156317 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 02 10:05:50.156317 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application Aug 02 10:05:50.156317 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 610/1237] 10.222.0.82 () {56 vars in 1108 bytes} [Tue Aug 2 10:05:49 2022] GET /identity/v3/domains/Default => generated 87 bytes in 202 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 02 10:05:50.161528 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bee59509-4495-4140-a498-86b220dca3e4 None admin] Authenticating user token {{(pid=80377) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:05:50.277410 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bee59509-4495-4140-a498-86b220dca3e4 None admin] Validating token access rules against request {{(pid=80377) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:05:50.287220 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG dbcounter [-] [80377] Writing DB stats keystone:SELECT=100 {{(pid=80377) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 10:05:50.314800 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f74e141-bad5-4d35-b231-d4cc1a3f392a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '891005fa5ff74041b75747d16cc66798', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80377) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:05:50.315295 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f74e141-bad5-4d35-b231-d4cc1a3f392a None admin] REQUEST_METHOD: `GET` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:05:50.315578 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f74e141-bad5-4d35-b231-d4cc1a3f392a None admin] SCRIPT_NAME: `/identity` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:05:50.315873 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f74e141-bad5-4d35-b231-d4cc1a3f392a None admin] PATH_INFO: `/v3/domains` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:05:50.316552 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f74e141-bad5-4d35-b231-d4cc1a3f392a None admin] RBAC: Adding query filter params (name=Default) {{(pid=80377) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 02 10:05:50.316865 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f74e141-bad5-4d35-b231-d4cc1a3f392a None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:05:50.318305 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f74e141-bad5-4d35-b231-d4cc1a3f392a None admin] RBAC: Authorization granted {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:05:50.331699 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: [pid: 80377|app: 0|req: 628/1238] 10.222.0.82 () {56 vars in 1109 bytes} [Tue Aug 2 10:05:50 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 02 10:05:50.337485 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-597e5945-7806-4017-b598-20aa06815ccb None admin] Authenticating user token {{(pid=80378) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:05:50.469317 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-597e5945-7806-4017-b598-20aa06815ccb None admin] Validating token access rules against request {{(pid=80378) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:05:50.509398 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4857ea99-61e3-4e56-875f-5bf27a3b4768 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '891005fa5ff74041b75747d16cc66798', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80378) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:05:50.510486 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4857ea99-61e3-4e56-875f-5bf27a3b4768 None admin] REQUEST_METHOD: `GET` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:05:50.510927 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4857ea99-61e3-4e56-875f-5bf27a3b4768 None admin] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:05:50.511352 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4857ea99-61e3-4e56-875f-5bf27a3b4768 None admin] PATH_INFO: `/v3/users/placement` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:05:50.516188 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG dbcounter [-] [80378] Writing DB stats keystone:SELECT=100 {{(pid=80378) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 10:05:50.524484 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4857ea99-61e3-4e56-875f-5bf27a3b4768 None admin] RBAC: Authorizing `identity:get_user(user_id=placement)` {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:05:50.525992 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4857ea99-61e3-4e56-875f-5bf27a3b4768 None admin] RBAC: Authorization granted {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:05:50.538895 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application [None req-4857ea99-61e3-4e56-875f-5bf27a3b4768 None admin] Could not find user: placement.: keystone.exception.UserNotFound: Could not find user: placement. Aug 02 10:05:50.538895 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 02 10:05:50.538895 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 02 10:05:50.538895 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 02 10:05:50.538895 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 02 10:05:50.538895 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 02 10:05:50.538895 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 02 10:05:50.538895 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 02 10:05:50.538895 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 02 10:05:50.538895 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 02 10:05:50.538895 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 02 10:05:50.538895 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 02 10:05:50.538895 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 02 10:05:50.538895 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 02 10:05:50.538895 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 02 10:05:50.538895 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 02 10:05:50.541018 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 02 10:05:50.541018 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 02 10:05:50.541018 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 02 10:05:50.541018 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 02 10:05:50.541018 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 02 10:05:50.541018 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 02 10:05:50.541018 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 02 10:05:50.541018 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 02 10:05:50.541018 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 02 10:05:50.541018 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application return self.get_or_create( Aug 02 10:05:50.541018 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 02 10:05:50.541018 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application with Lock( Aug 02 10:05:50.541018 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 02 10:05:50.541018 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application return self._enter() Aug 02 10:05:50.541018 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 02 10:05:50.541018 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 02 10:05:50.541018 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 02 10:05:50.541018 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application return self.creator() Aug 02 10:05:50.543282 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 02 10:05:50.543282 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application created_value = creator( Aug 02 10:05:50.543282 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 02 10:05:50.543282 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 02 10:05:50.543282 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 02 10:05:50.543282 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 02 10:05:50.543282 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 02 10:05:50.543282 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 02 10:05:50.543282 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: placement. Aug 02 10:05:50.543282 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application Aug 02 10:05:50.545400 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 611/1239] 10.222.0.82 () {56 vars in 1108 bytes} [Tue Aug 2 10:05:50 2022] GET /identity/v3/users/placement => generated 87 bytes in 208 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 02 10:05:50.546442 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f74e141-bad5-4d35-b231-d4cc1a3f392a None admin] Authenticating user token {{(pid=80377) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:05:50.675430 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f74e141-bad5-4d35-b231-d4cc1a3f392a None admin] Validating token access rules against request {{(pid=80377) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:05:50.716972 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aff56bef-367a-473c-a4d5-8a6e9c2a0470 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '891005fa5ff74041b75747d16cc66798', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80377) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:05:50.717748 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aff56bef-367a-473c-a4d5-8a6e9c2a0470 None admin] REQUEST_METHOD: `GET` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:05:50.718062 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aff56bef-367a-473c-a4d5-8a6e9c2a0470 None admin] SCRIPT_NAME: `/identity` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:05:50.718350 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aff56bef-367a-473c-a4d5-8a6e9c2a0470 None admin] PATH_INFO: `/v3/users` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:05:50.719294 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aff56bef-367a-473c-a4d5-8a6e9c2a0470 None admin] RBAC: Adding query filter params (domain_id=default, name=placement) {{(pid=80377) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 02 10:05:50.719653 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aff56bef-367a-473c-a4d5-8a6e9c2a0470 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:05:50.721209 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aff56bef-367a-473c-a4d5-8a6e9c2a0470 None admin] RBAC: Authorization granted {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:05:50.740113 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: [pid: 80377|app: 0|req: 629/1240] 10.222.0.82 () {56 vars in 1143 bytes} [Tue Aug 2 10:05:50 2022] GET /identity/v3/users?domain_id=default&name=placement => generated 378 bytes in 194 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 02 10:05:50.746475 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4857ea99-61e3-4e56-875f-5bf27a3b4768 None admin] Authenticating user token {{(pid=80378) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:05:50.867844 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4857ea99-61e3-4e56-875f-5bf27a3b4768 None admin] Validating token access rules against request {{(pid=80378) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:05:50.912919 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90d61c4e-e05d-4c0f-aff3-62cfec599ac4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '891005fa5ff74041b75747d16cc66798', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80378) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:05:50.913640 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90d61c4e-e05d-4c0f-aff3-62cfec599ac4 None admin] REQUEST_METHOD: `GET` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:05:50.913995 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90d61c4e-e05d-4c0f-aff3-62cfec599ac4 None admin] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:05:50.914307 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90d61c4e-e05d-4c0f-aff3-62cfec599ac4 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:05:50.988598 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG dbcounter [-] [80378] Writing DB stats keystone:SELECT=100 {{(pid=80378) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 10:05:50.998276 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90d61c4e-e05d-4c0f-aff3-62cfec599ac4 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:05:50.999951 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90d61c4e-e05d-4c0f-aff3-62cfec599ac4 None admin] RBAC: Authorization granted {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:05:51.125149 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 612/1241] 10.222.0.82 () {58 vars in 1282 bytes} [Tue Aug 2 10:05:50 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 379 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Aug 02 10:05:51.131731 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aff56bef-367a-473c-a4d5-8a6e9c2a0470 None admin] Authenticating user token {{(pid=80377) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:05:51.164041 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG dbcounter [-] [80377] Writing DB stats keystone:SELECT=100 {{(pid=80377) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 10:05:51.244052 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aff56bef-367a-473c-a4d5-8a6e9c2a0470 None admin] Validating token access rules against request {{(pid=80377) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:05:51.288948 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6c5d32d-582d-4127-adaa-7d366fb84a4a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '891005fa5ff74041b75747d16cc66798', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80377) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:05:51.289611 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6c5d32d-582d-4127-adaa-7d366fb84a4a None admin] REQUEST_METHOD: `GET` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:05:51.289948 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6c5d32d-582d-4127-adaa-7d366fb84a4a None admin] SCRIPT_NAME: `/identity` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:05:51.290220 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6c5d32d-582d-4127-adaa-7d366fb84a4a None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:05:51.309242 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6c5d32d-582d-4127-adaa-7d366fb84a4a None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:05:51.310746 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6c5d32d-582d-4127-adaa-7d366fb84a4a None admin] RBAC: Authorization granted {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:05:51.327209 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application [None req-c6c5d32d-582d-4127-adaa-7d366fb84a4a None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 02 10:05:51.327209 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 02 10:05:51.327209 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 02 10:05:51.327209 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 02 10:05:51.327209 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 02 10:05:51.327209 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 02 10:05:51.327209 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 02 10:05:51.327209 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 02 10:05:51.327209 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 02 10:05:51.327209 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 02 10:05:51.327209 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 02 10:05:51.327209 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 02 10:05:51.327209 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 02 10:05:51.327209 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 02 10:05:51.327209 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 02 10:05:51.328381 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 02 10:05:51.328381 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 02 10:05:51.328381 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 02 10:05:51.328381 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 02 10:05:51.328381 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 02 10:05:51.328381 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 02 10:05:51.328381 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application return self.get_or_create( Aug 02 10:05:51.328381 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 02 10:05:51.328381 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application with Lock( Aug 02 10:05:51.328381 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 02 10:05:51.328381 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application return self._enter() Aug 02 10:05:51.328381 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 02 10:05:51.328381 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 02 10:05:51.328381 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 02 10:05:51.328381 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application return self.creator() Aug 02 10:05:51.328381 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 02 10:05:51.328381 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application created_value = creator( Aug 02 10:05:51.329460 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 02 10:05:51.329460 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 02 10:05:51.329460 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 02 10:05:51.329460 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application Aug 02 10:05:51.329460 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: [pid: 80377|app: 0|req: 630/1242] 10.222.0.82 () {56 vars in 1108 bytes} [Tue Aug 2 10:05:51 2022] GET /identity/v3/domains/Default => generated 87 bytes in 197 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 02 10:05:51.336347 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90d61c4e-e05d-4c0f-aff3-62cfec599ac4 None admin] Authenticating user token {{(pid=80378) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:05:51.464314 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG dbcounter [-] [80378] Writing DB stats keystone:SELECT=100 {{(pid=80378) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 10:05:51.471875 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90d61c4e-e05d-4c0f-aff3-62cfec599ac4 None admin] Validating token access rules against request {{(pid=80378) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:05:51.515094 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82505fe5-73b3-4d5d-98a5-c52d95266a43 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '891005fa5ff74041b75747d16cc66798', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80378) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:05:51.515859 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82505fe5-73b3-4d5d-98a5-c52d95266a43 None admin] REQUEST_METHOD: `GET` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:05:51.516325 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82505fe5-73b3-4d5d-98a5-c52d95266a43 None admin] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:05:51.516697 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82505fe5-73b3-4d5d-98a5-c52d95266a43 None admin] PATH_INFO: `/v3/domains` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:05:51.517745 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82505fe5-73b3-4d5d-98a5-c52d95266a43 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80378) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 02 10:05:51.518258 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82505fe5-73b3-4d5d-98a5-c52d95266a43 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:05:51.520331 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82505fe5-73b3-4d5d-98a5-c52d95266a43 None admin] RBAC: Authorization granted {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:05:51.535019 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 613/1243] 10.222.0.82 () {56 vars in 1109 bytes} [Tue Aug 2 10:05:51 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 200 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 02 10:05:51.542571 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6c5d32d-582d-4127-adaa-7d366fb84a4a None admin] Authenticating user token {{(pid=80377) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:05:51.638024 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG dbcounter [-] [80377] Writing DB stats keystone:SELECT=100 {{(pid=80377) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 10:05:51.668088 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6c5d32d-582d-4127-adaa-7d366fb84a4a None admin] Validating token access rules against request {{(pid=80377) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:05:51.711979 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1a631b2-2962-477f-84a2-fa7c7492a367 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '891005fa5ff74041b75747d16cc66798', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80377) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:05:51.712622 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1a631b2-2962-477f-84a2-fa7c7492a367 None admin] REQUEST_METHOD: `GET` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:05:51.712986 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1a631b2-2962-477f-84a2-fa7c7492a367 None admin] SCRIPT_NAME: `/identity` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:05:51.713164 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1a631b2-2962-477f-84a2-fa7c7492a367 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:05:51.720173 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1a631b2-2962-477f-84a2-fa7c7492a367 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:05:51.721654 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1a631b2-2962-477f-84a2-fa7c7492a367 None admin] RBAC: Authorization granted {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:05:51.734372 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application [None req-c1a631b2-2962-477f-84a2-fa7c7492a367 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 02 10:05:51.734372 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 02 10:05:51.734372 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 02 10:05:51.734372 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 02 10:05:51.734372 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 02 10:05:51.734372 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 02 10:05:51.734372 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 02 10:05:51.734372 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 02 10:05:51.734372 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 02 10:05:51.734372 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 02 10:05:51.734372 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 02 10:05:51.734372 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 02 10:05:51.734372 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 02 10:05:51.734372 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 02 10:05:51.734372 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 02 10:05:51.735555 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 02 10:05:51.735555 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 02 10:05:51.735555 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 02 10:05:51.735555 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 02 10:05:51.735555 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 02 10:05:51.735555 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 02 10:05:51.735555 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application return self.get_or_create( Aug 02 10:05:51.735555 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 02 10:05:51.735555 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application with Lock( Aug 02 10:05:51.735555 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 02 10:05:51.735555 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application return self._enter() Aug 02 10:05:51.735555 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 02 10:05:51.735555 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 02 10:05:51.735555 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 02 10:05:51.735555 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application return self.creator() Aug 02 10:05:51.735555 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 02 10:05:51.735555 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application created_value = creator( Aug 02 10:05:51.736798 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 02 10:05:51.736798 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 02 10:05:51.736798 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 02 10:05:51.736798 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 02 10:05:51.736798 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 02 10:05:51.736798 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 02 10:05:51.736798 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 02 10:05:51.736798 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application Aug 02 10:05:51.736798 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: [pid: 80377|app: 0|req: 631/1244] 10.222.0.82 () {56 vars in 1111 bytes} [Tue Aug 2 10:05:51 2022] GET /identity/v3/projects/service => generated 88 bytes in 194 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 02 10:05:51.742821 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82505fe5-73b3-4d5d-98a5-c52d95266a43 None admin] Authenticating user token {{(pid=80378) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:05:51.874821 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82505fe5-73b3-4d5d-98a5-c52d95266a43 None admin] Validating token access rules against request {{(pid=80378) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:05:51.916294 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b708566d-8ad1-4945-bcd4-db53cf1a7070 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '891005fa5ff74041b75747d16cc66798', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80378) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:05:51.917121 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b708566d-8ad1-4945-bcd4-db53cf1a7070 None admin] REQUEST_METHOD: `GET` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:05:51.917465 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b708566d-8ad1-4945-bcd4-db53cf1a7070 None admin] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:05:51.917740 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b708566d-8ad1-4945-bcd4-db53cf1a7070 None admin] PATH_INFO: `/v3/projects` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:05:51.918927 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b708566d-8ad1-4945-bcd4-db53cf1a7070 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80378) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 02 10:05:51.919373 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b708566d-8ad1-4945-bcd4-db53cf1a7070 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:05:51.921089 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b708566d-8ad1-4945-bcd4-db53cf1a7070 None admin] RBAC: Authorization granted {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:05:51.936739 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 614/1245] 10.222.0.82 () {56 vars in 1148 bytes} [Tue Aug 2 10:05:51 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 195 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 02 10:05:51.944138 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1a631b2-2962-477f-84a2-fa7c7492a367 None admin] Authenticating user token {{(pid=80377) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:05:52.068515 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1a631b2-2962-477f-84a2-fa7c7492a367 None admin] Validating token access rules against request {{(pid=80377) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:05:52.108099 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7c481ab-4424-4734-aa89-e7b85c3067fe None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '891005fa5ff74041b75747d16cc66798', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80377) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:05:52.108644 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7c481ab-4424-4734-aa89-e7b85c3067fe None admin] REQUEST_METHOD: `GET` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:05:52.108923 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7c481ab-4424-4734-aa89-e7b85c3067fe None admin] SCRIPT_NAME: `/identity` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:05:52.109192 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7c481ab-4424-4734-aa89-e7b85c3067fe None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:05:52.110029 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7c481ab-4424-4734-aa89-e7b85c3067fe None admin] RBAC: Adding query filter params (role.id=02003a0be60f47f2a8dfea17053af52c, scope.project.id=f1b38ebb804c41c69772c998aae72c41, user.id=ae6e910e50d74f29bfb89bf18d95a908) {{(pid=80377) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 02 10:05:52.110343 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7c481ab-4424-4734-aa89-e7b85c3067fe None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:05:52.111788 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7c481ab-4424-4734-aa89-e7b85c3067fe None admin] RBAC: Authorization granted {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:05:52.119452 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: [pid: 80377|app: 0|req: 632/1246] 10.222.0.82 () {56 vars in 1374 bytes} [Tue Aug 2 10:05:51 2022] GET /identity/v3/role_assignments?user.id=ae6e910e50d74f29bfb89bf18d95a908&scope.project.id=f1b38ebb804c41c69772c998aae72c41&role.id=02003a0be60f47f2a8dfea17053af52c => generated 608 bytes in 176 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 02 10:05:54.359256 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-884230f6-a072-4bdf-bd0f-0b893de384cc None None] REQUEST_METHOD: `GET` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:05:54.359962 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-884230f6-a072-4bdf-bd0f-0b893de384cc None None] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:05:54.359962 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-884230f6-a072-4bdf-bd0f-0b893de384cc None None] PATH_INFO: `/` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:05:54.361170 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 615/1247] 10.222.0.82 () {54 vars in 926 bytes} [Tue Aug 2 10:05:54 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 02 10:05:54.367661 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad9713ae-5da1-48f1-9139-8958fe0a0f0b None None] REQUEST_METHOD: `POST` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:05:54.368258 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad9713ae-5da1-48f1-9139-8958fe0a0f0b None None] SCRIPT_NAME: `/identity` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:05:54.368258 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad9713ae-5da1-48f1-9139-8958fe0a0f0b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:05:54.378515 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG dbcounter [-] [80377] Writing DB stats keystone:SELECT=100 {{(pid=80377) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 10:05:54.399125 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.auth.core [None req-ad9713ae-5da1-48f1-9139-8958fe0a0f0b None None] MFA Rules not processed for user `891005fa5ff74041b75747d16cc66798`. Rule list: `[]` (Enabled: `True`). {{(pid=80377) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 02 10:05:54.400358 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.fernet_utils [None req-ad9713ae-5da1-48f1-9139-8958fe0a0f0b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80377) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 10:05:54.483870 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: [pid: 80377|app: 0|req: 633/1248] 10.222.0.82 () {58 vars in 1023 bytes} [Tue Aug 2 10:05:54 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 117 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 02 10:05:54.508550 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-884230f6-a072-4bdf-bd0f-0b893de384cc None None] Authenticating user token {{(pid=80378) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:05:54.531257 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG dbcounter [-] [80378] Writing DB stats keystone:SELECT=100 {{(pid=80378) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 10:05:54.644110 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-884230f6-a072-4bdf-bd0f-0b893de384cc None None] Validating token access rules against request {{(pid=80378) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:05:54.688671 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6aa8655a-f316-4eed-acf9-1df345f3cbb9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '891005fa5ff74041b75747d16cc66798', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80378) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:05:54.689324 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6aa8655a-f316-4eed-acf9-1df345f3cbb9 None admin] REQUEST_METHOD: `GET` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:05:54.689604 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6aa8655a-f316-4eed-acf9-1df345f3cbb9 None admin] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:05:54.689901 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6aa8655a-f316-4eed-acf9-1df345f3cbb9 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:05:54.700880 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6aa8655a-f316-4eed-acf9-1df345f3cbb9 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 02 10:05:54.706465 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application [None req-6aa8655a-f316-4eed-acf9-1df345f3cbb9 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 02 10:05:54.706465 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 02 10:05:54.706465 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 02 10:05:54.706465 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 02 10:05:54.706465 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 02 10:05:54.706465 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 02 10:05:54.706465 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 02 10:05:54.706465 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 02 10:05:54.706465 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 02 10:05:54.706465 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 02 10:05:54.706465 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 02 10:05:54.706465 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 02 10:05:54.706465 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 02 10:05:54.706465 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 02 10:05:54.706465 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 02 10:05:54.706465 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 02 10:05:54.707679 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 02 10:05:54.707679 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 02 10:05:54.707679 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 02 10:05:54.707679 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 02 10:05:54.707679 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 02 10:05:54.707679 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 02 10:05:54.707679 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 02 10:05:54.707679 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 02 10:05:54.707679 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 02 10:05:54.707679 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application return self.get_or_create( Aug 02 10:05:54.707679 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 02 10:05:54.707679 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application with Lock( Aug 02 10:05:54.707679 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 02 10:05:54.707679 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application return self._enter() Aug 02 10:05:54.707679 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 02 10:05:54.707679 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 02 10:05:54.709678 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 02 10:05:54.709678 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application return self.creator() Aug 02 10:05:54.709678 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 02 10:05:54.709678 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application created_value = creator( Aug 02 10:05:54.709678 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 02 10:05:54.709678 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 02 10:05:54.709678 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 02 10:05:54.709678 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 02 10:05:54.709678 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 02 10:05:54.709678 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 02 10:05:54.709678 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 02 10:05:54.709678 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application Aug 02 10:05:54.709678 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 616/1249] 10.222.0.82 () {56 vars in 1096 bytes} [Tue Aug 2 10:05:54 2022] GET /identity/v3/roles/admin => generated 83 bytes in 200 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 02 10:05:54.715380 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad9713ae-5da1-48f1-9139-8958fe0a0f0b None None] Authenticating user token {{(pid=80377) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:05:54.850441 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad9713ae-5da1-48f1-9139-8958fe0a0f0b None None] Validating token access rules against request {{(pid=80377) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:05:54.890291 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG dbcounter [-] [80377] Writing DB stats keystone:SELECT=100 {{(pid=80377) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 10:05:54.895966 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af58a4c0-3b36-4958-a78f-12fe7a5c4e43 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '891005fa5ff74041b75747d16cc66798', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80377) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:05:54.896825 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af58a4c0-3b36-4958-a78f-12fe7a5c4e43 None admin] REQUEST_METHOD: `GET` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:05:54.897636 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af58a4c0-3b36-4958-a78f-12fe7a5c4e43 None admin] SCRIPT_NAME: `/identity` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:05:54.898325 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af58a4c0-3b36-4958-a78f-12fe7a5c4e43 None admin] PATH_INFO: `/v3/roles` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:05:54.899316 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af58a4c0-3b36-4958-a78f-12fe7a5c4e43 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80377) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 02 10:05:54.899768 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af58a4c0-3b36-4958-a78f-12fe7a5c4e43 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:05:54.902153 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af58a4c0-3b36-4958-a78f-12fe7a5c4e43 None admin] RBAC: Authorization granted {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:05:54.914991 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: [pid: 80377|app: 0|req: 634/1250] 10.222.0.82 () {56 vars in 1099 bytes} [Tue Aug 2 10:05:54 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 200 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 02 10:05:54.921558 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6aa8655a-f316-4eed-acf9-1df345f3cbb9 None admin] Authenticating user token {{(pid=80378) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:05:55.018849 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG dbcounter [-] [80378] Writing DB stats keystone:SELECT=100 {{(pid=80378) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 10:05:55.046191 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6aa8655a-f316-4eed-acf9-1df345f3cbb9 None admin] Validating token access rules against request {{(pid=80378) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:05:55.097743 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0a7752d-718e-4448-87f0-29a2b453f95b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '891005fa5ff74041b75747d16cc66798', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80378) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:05:55.098562 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0a7752d-718e-4448-87f0-29a2b453f95b None admin] REQUEST_METHOD: `GET` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:05:55.098845 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0a7752d-718e-4448-87f0-29a2b453f95b None admin] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:05:55.099148 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0a7752d-718e-4448-87f0-29a2b453f95b None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:05:55.111941 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0a7752d-718e-4448-87f0-29a2b453f95b None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:05:55.113421 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0a7752d-718e-4448-87f0-29a2b453f95b None admin] RBAC: Authorization granted {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:05:55.132610 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application [None req-f0a7752d-718e-4448-87f0-29a2b453f95b None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 02 10:05:55.132610 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 02 10:05:55.132610 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 02 10:05:55.132610 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 02 10:05:55.132610 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 02 10:05:55.132610 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 02 10:05:55.132610 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 02 10:05:55.132610 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 02 10:05:55.132610 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 02 10:05:55.132610 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 02 10:05:55.132610 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 02 10:05:55.132610 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 02 10:05:55.132610 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 02 10:05:55.132610 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 02 10:05:55.132610 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 02 10:05:55.133929 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 02 10:05:55.133929 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 02 10:05:55.133929 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 02 10:05:55.133929 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 02 10:05:55.133929 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 02 10:05:55.133929 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 02 10:05:55.133929 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application return self.get_or_create( Aug 02 10:05:55.133929 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 02 10:05:55.133929 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application with Lock( Aug 02 10:05:55.133929 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 02 10:05:55.133929 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application return self._enter() Aug 02 10:05:55.133929 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 02 10:05:55.133929 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 02 10:05:55.133929 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 02 10:05:55.133929 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application return self.creator() Aug 02 10:05:55.133929 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 02 10:05:55.133929 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application created_value = creator( Aug 02 10:05:55.134880 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 02 10:05:55.134880 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 02 10:05:55.134880 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 02 10:05:55.134880 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application Aug 02 10:05:55.134880 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 617/1251] 10.222.0.82 () {56 vars in 1108 bytes} [Tue Aug 2 10:05:54 2022] GET /identity/v3/domains/Default => generated 87 bytes in 213 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 02 10:05:55.140545 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af58a4c0-3b36-4958-a78f-12fe7a5c4e43 None admin] Authenticating user token {{(pid=80377) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:05:55.283887 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af58a4c0-3b36-4958-a78f-12fe7a5c4e43 None admin] Validating token access rules against request {{(pid=80377) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:05:55.332612 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51f34e26-585f-4f75-b5f2-2ffde2fc5b01 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '891005fa5ff74041b75747d16cc66798', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80377) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:05:55.333105 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51f34e26-585f-4f75-b5f2-2ffde2fc5b01 None admin] REQUEST_METHOD: `GET` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:05:55.333595 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51f34e26-585f-4f75-b5f2-2ffde2fc5b01 None admin] SCRIPT_NAME: `/identity` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:05:55.334042 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51f34e26-585f-4f75-b5f2-2ffde2fc5b01 None admin] PATH_INFO: `/v3/domains` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:05:55.334635 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51f34e26-585f-4f75-b5f2-2ffde2fc5b01 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80377) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 02 10:05:55.335563 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51f34e26-585f-4f75-b5f2-2ffde2fc5b01 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:05:55.337402 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51f34e26-585f-4f75-b5f2-2ffde2fc5b01 None admin] RBAC: Authorization granted {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:05:55.355165 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: [pid: 80377|app: 0|req: 635/1252] 10.222.0.82 () {56 vars in 1109 bytes} [Tue Aug 2 10:05:55 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 215 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 02 10:05:55.361172 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0a7752d-718e-4448-87f0-29a2b453f95b None admin] Authenticating user token {{(pid=80378) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:05:55.488406 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0a7752d-718e-4448-87f0-29a2b453f95b None admin] Validating token access rules against request {{(pid=80378) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:05:55.520931 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG dbcounter [-] [80378] Writing DB stats keystone:SELECT=100 {{(pid=80378) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 10:05:55.529858 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90fd432a-c632-4e65-93da-408fcab5988b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '891005fa5ff74041b75747d16cc66798', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80378) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:05:55.530517 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90fd432a-c632-4e65-93da-408fcab5988b None admin] REQUEST_METHOD: `GET` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:05:55.530800 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90fd432a-c632-4e65-93da-408fcab5988b None admin] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:05:55.531079 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90fd432a-c632-4e65-93da-408fcab5988b None admin] PATH_INFO: `/v3/users/placement` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:05:55.541849 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90fd432a-c632-4e65-93da-408fcab5988b None admin] RBAC: Authorizing `identity:get_user(user_id=placement)` {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:05:55.543540 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90fd432a-c632-4e65-93da-408fcab5988b None admin] RBAC: Authorization granted {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:05:55.558216 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application [None req-90fd432a-c632-4e65-93da-408fcab5988b None admin] Could not find user: placement.: keystone.exception.UserNotFound: Could not find user: placement. Aug 02 10:05:55.558216 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 02 10:05:55.558216 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 02 10:05:55.558216 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 02 10:05:55.558216 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 02 10:05:55.558216 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 02 10:05:55.558216 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 02 10:05:55.558216 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 02 10:05:55.558216 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 02 10:05:55.558216 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 02 10:05:55.558216 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 02 10:05:55.558216 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 02 10:05:55.558216 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 02 10:05:55.558216 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 02 10:05:55.558216 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 02 10:05:55.558216 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 02 10:05:55.565169 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 02 10:05:55.565169 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 02 10:05:55.565169 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 02 10:05:55.565169 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 02 10:05:55.565169 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 02 10:05:55.565169 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 02 10:05:55.565169 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 02 10:05:55.565169 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 02 10:05:55.565169 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 02 10:05:55.565169 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application return self.get_or_create( Aug 02 10:05:55.565169 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 02 10:05:55.565169 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application with Lock( Aug 02 10:05:55.565169 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 02 10:05:55.565169 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application return self._enter() Aug 02 10:05:55.565169 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 02 10:05:55.565169 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 02 10:05:55.565169 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 02 10:05:55.565169 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application return self.creator() Aug 02 10:05:55.566329 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 02 10:05:55.566329 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application created_value = creator( Aug 02 10:05:55.566329 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 02 10:05:55.566329 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 02 10:05:55.566329 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 02 10:05:55.566329 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 02 10:05:55.566329 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 02 10:05:55.566329 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 02 10:05:55.566329 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: placement. Aug 02 10:05:55.566329 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application Aug 02 10:05:55.566329 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 618/1253] 10.222.0.82 () {56 vars in 1108 bytes} [Tue Aug 2 10:05:55 2022] GET /identity/v3/users/placement => generated 87 bytes in 199 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 02 10:05:55.566946 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51f34e26-585f-4f75-b5f2-2ffde2fc5b01 None admin] Authenticating user token {{(pid=80377) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:05:55.643151 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG dbcounter [-] [80377] Writing DB stats keystone:SELECT=100 {{(pid=80377) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 10:05:55.703980 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51f34e26-585f-4f75-b5f2-2ffde2fc5b01 None admin] Validating token access rules against request {{(pid=80377) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:05:55.743733 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6464fd96-62b8-4cb4-9efc-e051340499a4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '891005fa5ff74041b75747d16cc66798', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80377) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:05:55.744495 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6464fd96-62b8-4cb4-9efc-e051340499a4 None admin] REQUEST_METHOD: `GET` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:05:55.744754 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6464fd96-62b8-4cb4-9efc-e051340499a4 None admin] SCRIPT_NAME: `/identity` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:05:55.745033 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6464fd96-62b8-4cb4-9efc-e051340499a4 None admin] PATH_INFO: `/v3/users` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:05:55.745880 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6464fd96-62b8-4cb4-9efc-e051340499a4 None admin] RBAC: Adding query filter params (domain_id=default, name=placement) {{(pid=80377) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 02 10:05:55.746195 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6464fd96-62b8-4cb4-9efc-e051340499a4 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:05:55.747651 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6464fd96-62b8-4cb4-9efc-e051340499a4 None admin] RBAC: Authorization granted {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:05:55.770193 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: [pid: 80377|app: 0|req: 636/1254] 10.222.0.82 () {56 vars in 1143 bytes} [Tue Aug 2 10:05:55 2022] GET /identity/v3/users?domain_id=default&name=placement => generated 378 bytes in 205 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 02 10:05:55.776809 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90fd432a-c632-4e65-93da-408fcab5988b None admin] Authenticating user token {{(pid=80378) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:05:55.909519 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90fd432a-c632-4e65-93da-408fcab5988b None admin] Validating token access rules against request {{(pid=80378) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:05:55.947723 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce18b407-803f-4a0f-a10a-2a5efd55db46 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '891005fa5ff74041b75747d16cc66798', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80378) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:05:55.949248 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce18b407-803f-4a0f-a10a-2a5efd55db46 None admin] REQUEST_METHOD: `GET` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:05:55.949764 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce18b407-803f-4a0f-a10a-2a5efd55db46 None admin] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:05:55.950342 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce18b407-803f-4a0f-a10a-2a5efd55db46 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:05:56.027568 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG dbcounter [-] [80378] Writing DB stats keystone:SELECT=100 {{(pid=80378) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 10:05:56.041936 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce18b407-803f-4a0f-a10a-2a5efd55db46 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:05:56.043858 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce18b407-803f-4a0f-a10a-2a5efd55db46 None admin] RBAC: Authorization granted {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:05:56.200118 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 619/1255] 10.222.0.82 () {58 vars in 1282 bytes} [Tue Aug 2 10:05:55 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 424 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Aug 02 10:05:56.206366 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6464fd96-62b8-4cb4-9efc-e051340499a4 None admin] Authenticating user token {{(pid=80377) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:05:56.336019 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6464fd96-62b8-4cb4-9efc-e051340499a4 None admin] Validating token access rules against request {{(pid=80377) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:05:56.354001 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG dbcounter [-] [80377] Writing DB stats keystone:SELECT=100 {{(pid=80377) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 10:05:56.386113 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e66835ba-e350-491e-b5e9-5656fac10116 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '891005fa5ff74041b75747d16cc66798', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80377) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:05:56.386790 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e66835ba-e350-491e-b5e9-5656fac10116 None admin] REQUEST_METHOD: `GET` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:05:56.387074 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e66835ba-e350-491e-b5e9-5656fac10116 None admin] SCRIPT_NAME: `/identity` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:05:56.387351 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e66835ba-e350-491e-b5e9-5656fac10116 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:05:56.401143 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e66835ba-e350-491e-b5e9-5656fac10116 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:05:56.402603 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e66835ba-e350-491e-b5e9-5656fac10116 None admin] RBAC: Authorization granted {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:05:56.423406 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application [None req-e66835ba-e350-491e-b5e9-5656fac10116 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 02 10:05:56.423406 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 02 10:05:56.423406 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 02 10:05:56.423406 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 02 10:05:56.423406 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 02 10:05:56.423406 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 02 10:05:56.423406 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 02 10:05:56.423406 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 02 10:05:56.423406 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 02 10:05:56.423406 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 02 10:05:56.423406 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 02 10:05:56.423406 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 02 10:05:56.423406 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 02 10:05:56.423406 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 02 10:05:56.423406 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 02 10:05:56.424809 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 02 10:05:56.424809 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 02 10:05:56.424809 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 02 10:05:56.424809 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 02 10:05:56.424809 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 02 10:05:56.424809 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 02 10:05:56.424809 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application return self.get_or_create( Aug 02 10:05:56.424809 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 02 10:05:56.424809 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application with Lock( Aug 02 10:05:56.424809 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 02 10:05:56.424809 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application return self._enter() Aug 02 10:05:56.424809 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 02 10:05:56.424809 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 02 10:05:56.424809 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 02 10:05:56.424809 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application return self.creator() Aug 02 10:05:56.424809 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 02 10:05:56.424809 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application created_value = creator( Aug 02 10:05:56.425943 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 02 10:05:56.425943 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 02 10:05:56.425943 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 02 10:05:56.425943 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application Aug 02 10:05:56.425943 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: [pid: 80377|app: 0|req: 637/1256] 10.222.0.82 () {56 vars in 1108 bytes} [Tue Aug 2 10:05:56 2022] GET /identity/v3/domains/Default => generated 87 bytes in 220 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 02 10:05:56.433365 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce18b407-803f-4a0f-a10a-2a5efd55db46 None admin] Authenticating user token {{(pid=80378) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:05:56.569300 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG dbcounter [-] [80378] Writing DB stats keystone:SELECT=100 {{(pid=80378) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 10:05:56.594059 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce18b407-803f-4a0f-a10a-2a5efd55db46 None admin] Validating token access rules against request {{(pid=80378) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:05:56.635992 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70982863-c3c5-4808-90f7-b832326b88fe None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '891005fa5ff74041b75747d16cc66798', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80378) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:05:56.636674 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70982863-c3c5-4808-90f7-b832326b88fe None admin] REQUEST_METHOD: `GET` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:05:56.637020 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70982863-c3c5-4808-90f7-b832326b88fe None admin] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:05:56.637406 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70982863-c3c5-4808-90f7-b832326b88fe None admin] PATH_INFO: `/v3/domains` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:05:56.638111 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70982863-c3c5-4808-90f7-b832326b88fe None admin] RBAC: Adding query filter params (name=Default) {{(pid=80378) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 02 10:05:56.638466 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70982863-c3c5-4808-90f7-b832326b88fe None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:05:56.640677 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70982863-c3c5-4808-90f7-b832326b88fe None admin] RBAC: Authorization granted {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:05:56.655115 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 620/1257] 10.222.0.82 () {56 vars in 1109 bytes} [Tue Aug 2 10:05:56 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 222 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 02 10:05:56.661129 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e66835ba-e350-491e-b5e9-5656fac10116 None admin] Authenticating user token {{(pid=80377) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:05:56.790523 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e66835ba-e350-491e-b5e9-5656fac10116 None admin] Validating token access rules against request {{(pid=80377) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:05:56.838682 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4225f70b-800d-4c16-bc91-ff5bffad3f71 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '891005fa5ff74041b75747d16cc66798', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80377) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:05:56.839307 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4225f70b-800d-4c16-bc91-ff5bffad3f71 None admin] REQUEST_METHOD: `GET` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:05:56.839613 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4225f70b-800d-4c16-bc91-ff5bffad3f71 None admin] SCRIPT_NAME: `/identity` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:05:56.842578 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4225f70b-800d-4c16-bc91-ff5bffad3f71 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:05:56.860651 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4225f70b-800d-4c16-bc91-ff5bffad3f71 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:05:56.862437 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4225f70b-800d-4c16-bc91-ff5bffad3f71 None admin] RBAC: Authorization granted {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:05:56.877298 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application [None req-4225f70b-800d-4c16-bc91-ff5bffad3f71 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 02 10:05:56.877298 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 02 10:05:56.877298 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 02 10:05:56.877298 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 02 10:05:56.877298 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 02 10:05:56.877298 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 02 10:05:56.877298 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 02 10:05:56.877298 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 02 10:05:56.877298 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 02 10:05:56.877298 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 02 10:05:56.877298 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 02 10:05:56.877298 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 02 10:05:56.877298 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 02 10:05:56.877298 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 02 10:05:56.877298 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 02 10:05:56.878402 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 02 10:05:56.878402 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 02 10:05:56.878402 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 02 10:05:56.878402 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 02 10:05:56.878402 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 02 10:05:56.878402 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 02 10:05:56.878402 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application return self.get_or_create( Aug 02 10:05:56.878402 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 02 10:05:56.878402 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application with Lock( Aug 02 10:05:56.878402 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 02 10:05:56.878402 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application return self._enter() Aug 02 10:05:56.878402 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 02 10:05:56.878402 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 02 10:05:56.878402 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 02 10:05:56.878402 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application return self.creator() Aug 02 10:05:56.878402 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 02 10:05:56.878402 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application created_value = creator( Aug 02 10:05:56.879558 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 02 10:05:56.879558 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 02 10:05:56.879558 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 02 10:05:56.879558 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 02 10:05:56.879558 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 02 10:05:56.879558 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 02 10:05:56.879558 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 02 10:05:56.879558 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application Aug 02 10:05:56.879558 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: [pid: 80377|app: 0|req: 638/1258] 10.222.0.82 () {56 vars in 1111 bytes} [Tue Aug 2 10:05:56 2022] GET /identity/v3/projects/service => generated 88 bytes in 219 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 02 10:05:56.886878 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70982863-c3c5-4808-90f7-b832326b88fe None admin] Authenticating user token {{(pid=80378) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:05:57.013893 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70982863-c3c5-4808-90f7-b832326b88fe None admin] Validating token access rules against request {{(pid=80378) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:05:57.057625 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c330dac-d8f2-4143-86a8-53f8dcab864f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '891005fa5ff74041b75747d16cc66798', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80378) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:05:57.058319 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c330dac-d8f2-4143-86a8-53f8dcab864f None admin] REQUEST_METHOD: `GET` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:05:57.058560 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c330dac-d8f2-4143-86a8-53f8dcab864f None admin] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:05:57.058838 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c330dac-d8f2-4143-86a8-53f8dcab864f None admin] PATH_INFO: `/v3/projects` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:05:57.059601 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c330dac-d8f2-4143-86a8-53f8dcab864f None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80378) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 02 10:05:57.059908 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c330dac-d8f2-4143-86a8-53f8dcab864f None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:05:57.061379 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c330dac-d8f2-4143-86a8-53f8dcab864f None admin] RBAC: Authorization granted {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:05:57.088928 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 621/1259] 10.222.0.82 () {56 vars in 1148 bytes} [Tue Aug 2 10:05:56 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 197 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 02 10:05:57.090074 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4225f70b-800d-4c16-bc91-ff5bffad3f71 None admin] Authenticating user token {{(pid=80377) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:05:57.123191 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG dbcounter [-] [80377] Writing DB stats keystone:SELECT=100 {{(pid=80377) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 10:05:57.232330 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4225f70b-800d-4c16-bc91-ff5bffad3f71 None admin] Validating token access rules against request {{(pid=80377) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:05:57.279277 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c8f3215-098b-4f19-ac3b-c3c7be913e73 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '891005fa5ff74041b75747d16cc66798', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80377) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:05:57.280047 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c8f3215-098b-4f19-ac3b-c3c7be913e73 None admin] REQUEST_METHOD: `GET` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:05:57.280348 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c8f3215-098b-4f19-ac3b-c3c7be913e73 None admin] SCRIPT_NAME: `/identity` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:05:57.280639 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c8f3215-098b-4f19-ac3b-c3c7be913e73 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:05:57.281606 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c8f3215-098b-4f19-ac3b-c3c7be913e73 None admin] RBAC: Adding query filter params (role.id=237966442f05411191105bf336bc14f3, scope.project.id=f1b38ebb804c41c69772c998aae72c41, user.id=ae6e910e50d74f29bfb89bf18d95a908) {{(pid=80377) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 02 10:05:57.282073 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c8f3215-098b-4f19-ac3b-c3c7be913e73 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:05:57.284093 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c8f3215-098b-4f19-ac3b-c3c7be913e73 None admin] RBAC: Authorization granted {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:05:57.295195 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: [pid: 80377|app: 0|req: 639/1260] 10.222.0.82 () {56 vars in 1374 bytes} [Tue Aug 2 10:05:57 2022] GET /identity/v3/role_assignments?user.id=ae6e910e50d74f29bfb89bf18d95a908&scope.project.id=f1b38ebb804c41c69772c998aae72c41&role.id=237966442f05411191105bf336bc14f3 => generated 259 bytes in 205 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 02 10:05:59.399787 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e24d3448-356d-4ab2-9b00-adceeccabbfb None None] REQUEST_METHOD: `GET` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:05:59.400570 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e24d3448-356d-4ab2-9b00-adceeccabbfb None None] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:05:59.400570 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e24d3448-356d-4ab2-9b00-adceeccabbfb None None] PATH_INFO: `/` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:05:59.401944 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 622/1261] 10.222.0.82 () {54 vars in 926 bytes} [Tue Aug 2 10:05:59 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 02 10:05:59.407801 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0c00ea6-2795-4793-9e9d-18acf9e09383 None None] REQUEST_METHOD: `POST` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:05:59.408367 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0c00ea6-2795-4793-9e9d-18acf9e09383 None None] SCRIPT_NAME: `/identity` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:05:59.408367 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0c00ea6-2795-4793-9e9d-18acf9e09383 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:05:59.429871 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.auth.core [None req-c0c00ea6-2795-4793-9e9d-18acf9e09383 None None] MFA Rules not processed for user `891005fa5ff74041b75747d16cc66798`. Rule list: `[]` (Enabled: `True`). {{(pid=80377) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 02 10:05:59.431119 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.fernet_utils [None req-c0c00ea6-2795-4793-9e9d-18acf9e09383 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80377) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 10:05:59.510951 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: [pid: 80377|app: 0|req: 640/1262] 10.222.0.82 () {58 vars in 1023 bytes} [Tue Aug 2 10:05:59 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 104 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 02 10:05:59.535944 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e24d3448-356d-4ab2-9b00-adceeccabbfb None None] Authenticating user token {{(pid=80378) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:05:59.551439 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG dbcounter [-] [80378] Writing DB stats keystone:SELECT=100 {{(pid=80378) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 10:05:59.661412 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e24d3448-356d-4ab2-9b00-adceeccabbfb None None] Validating token access rules against request {{(pid=80378) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:05:59.701827 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96a74282-04b9-48f8-ad68-ea73957f3f41 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '891005fa5ff74041b75747d16cc66798', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80378) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:05:59.702498 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96a74282-04b9-48f8-ad68-ea73957f3f41 None admin] REQUEST_METHOD: `GET` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:05:59.702845 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96a74282-04b9-48f8-ad68-ea73957f3f41 None admin] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:05:59.703075 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96a74282-04b9-48f8-ad68-ea73957f3f41 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:05:59.714248 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96a74282-04b9-48f8-ad68-ea73957f3f41 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 02 10:05:59.719329 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application [None req-96a74282-04b9-48f8-ad68-ea73957f3f41 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 02 10:05:59.719329 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 02 10:05:59.719329 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 02 10:05:59.719329 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 02 10:05:59.719329 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 02 10:05:59.719329 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 02 10:05:59.719329 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 02 10:05:59.719329 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 02 10:05:59.719329 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 02 10:05:59.719329 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 02 10:05:59.719329 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 02 10:05:59.719329 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 02 10:05:59.719329 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 02 10:05:59.719329 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 02 10:05:59.719329 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 02 10:05:59.719329 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 02 10:05:59.720590 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 02 10:05:59.720590 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 02 10:05:59.720590 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 02 10:05:59.720590 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 02 10:05:59.720590 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 02 10:05:59.720590 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 02 10:05:59.720590 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 02 10:05:59.720590 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 02 10:05:59.720590 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 02 10:05:59.720590 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application return self.get_or_create( Aug 02 10:05:59.720590 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 02 10:05:59.720590 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application with Lock( Aug 02 10:05:59.720590 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 02 10:05:59.720590 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application return self._enter() Aug 02 10:05:59.720590 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 02 10:05:59.720590 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 02 10:05:59.721701 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 02 10:05:59.721701 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application return self.creator() Aug 02 10:05:59.721701 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 02 10:05:59.721701 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application created_value = creator( Aug 02 10:05:59.721701 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 02 10:05:59.721701 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 02 10:05:59.721701 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 02 10:05:59.721701 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 02 10:05:59.721701 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 02 10:05:59.721701 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 02 10:05:59.721701 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 02 10:05:59.721701 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application Aug 02 10:05:59.721701 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 623/1263] 10.222.0.82 () {56 vars in 1096 bytes} [Tue Aug 2 10:05:59 2022] GET /identity/v3/roles/admin => generated 83 bytes in 185 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 02 10:05:59.726741 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0c00ea6-2795-4793-9e9d-18acf9e09383 None None] Authenticating user token {{(pid=80377) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:05:59.746690 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG dbcounter [-] [80377] Writing DB stats keystone:SELECT=100 {{(pid=80377) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 10:05:59.838200 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0c00ea6-2795-4793-9e9d-18acf9e09383 None None] Validating token access rules against request {{(pid=80377) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:05:59.877307 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89ff84b1-88f4-4032-ba96-ba8807994198 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '891005fa5ff74041b75747d16cc66798', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80377) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:05:59.877971 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89ff84b1-88f4-4032-ba96-ba8807994198 None admin] REQUEST_METHOD: `GET` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:05:59.878256 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89ff84b1-88f4-4032-ba96-ba8807994198 None admin] SCRIPT_NAME: `/identity` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:05:59.878533 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89ff84b1-88f4-4032-ba96-ba8807994198 None admin] PATH_INFO: `/v3/roles` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:05:59.879256 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89ff84b1-88f4-4032-ba96-ba8807994198 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80377) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 02 10:05:59.879561 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89ff84b1-88f4-4032-ba96-ba8807994198 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:05:59.881521 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89ff84b1-88f4-4032-ba96-ba8807994198 None admin] RBAC: Authorization granted {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:05:59.890301 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: [pid: 80377|app: 0|req: 641/1264] 10.222.0.82 () {56 vars in 1099 bytes} [Tue Aug 2 10:05:59 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 02 10:05:59.896125 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96a74282-04b9-48f8-ad68-ea73957f3f41 None admin] Authenticating user token {{(pid=80378) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:05:59.964362 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG dbcounter [-] [80378] Writing DB stats keystone:SELECT=100 {{(pid=80378) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 10:05:59.999076 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96a74282-04b9-48f8-ad68-ea73957f3f41 None admin] Validating token access rules against request {{(pid=80378) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:06:00.035419 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fda0779-cf58-4e19-a5be-e4b959c23eb8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '891005fa5ff74041b75747d16cc66798', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80378) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:06:00.036058 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fda0779-cf58-4e19-a5be-e4b959c23eb8 None admin] REQUEST_METHOD: `GET` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:06:00.036490 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fda0779-cf58-4e19-a5be-e4b959c23eb8 None admin] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:06:00.037362 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fda0779-cf58-4e19-a5be-e4b959c23eb8 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:06:00.049624 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9fda0779-cf58-4e19-a5be-e4b959c23eb8 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:06:00.051255 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9fda0779-cf58-4e19-a5be-e4b959c23eb8 None admin] RBAC: Authorization granted {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:06:00.068793 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application [None req-9fda0779-cf58-4e19-a5be-e4b959c23eb8 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 02 10:06:00.068793 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 02 10:06:00.068793 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 02 10:06:00.068793 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 02 10:06:00.068793 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 02 10:06:00.068793 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 02 10:06:00.068793 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 02 10:06:00.068793 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 02 10:06:00.068793 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 02 10:06:00.068793 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 02 10:06:00.068793 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 02 10:06:00.068793 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 02 10:06:00.068793 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 02 10:06:00.068793 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 02 10:06:00.068793 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 02 10:06:00.069959 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 02 10:06:00.069959 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 02 10:06:00.069959 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 02 10:06:00.069959 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 02 10:06:00.069959 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 02 10:06:00.069959 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 02 10:06:00.069959 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application return self.get_or_create( Aug 02 10:06:00.069959 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 02 10:06:00.069959 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application with Lock( Aug 02 10:06:00.069959 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 02 10:06:00.069959 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application return self._enter() Aug 02 10:06:00.069959 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 02 10:06:00.069959 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 02 10:06:00.069959 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 02 10:06:00.069959 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application return self.creator() Aug 02 10:06:00.069959 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 02 10:06:00.069959 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application created_value = creator( Aug 02 10:06:00.070968 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 02 10:06:00.070968 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 02 10:06:00.070968 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 02 10:06:00.070968 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application Aug 02 10:06:00.073989 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 624/1265] 10.222.0.82 () {56 vars in 1108 bytes} [Tue Aug 2 10:05:59 2022] GET /identity/v3/domains/Default => generated 87 bytes in 178 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 02 10:06:00.076293 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89ff84b1-88f4-4032-ba96-ba8807994198 None admin] Authenticating user token {{(pid=80377) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:06:00.162730 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG dbcounter [-] [80377] Writing DB stats keystone:SELECT=100 {{(pid=80377) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 10:06:00.190424 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89ff84b1-88f4-4032-ba96-ba8807994198 None admin] Validating token access rules against request {{(pid=80377) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:06:00.224149 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-123cc143-6614-485d-9d33-b5f2d51dea5a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '891005fa5ff74041b75747d16cc66798', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80377) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:06:00.225324 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-123cc143-6614-485d-9d33-b5f2d51dea5a None admin] REQUEST_METHOD: `GET` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:06:00.225605 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-123cc143-6614-485d-9d33-b5f2d51dea5a None admin] SCRIPT_NAME: `/identity` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:06:00.225906 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-123cc143-6614-485d-9d33-b5f2d51dea5a None admin] PATH_INFO: `/v3/domains` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:06:00.226633 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-123cc143-6614-485d-9d33-b5f2d51dea5a None admin] RBAC: Adding query filter params (name=Default) {{(pid=80377) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 02 10:06:00.226936 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-123cc143-6614-485d-9d33-b5f2d51dea5a None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:06:00.228322 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-123cc143-6614-485d-9d33-b5f2d51dea5a None admin] RBAC: Authorization granted {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:06:00.240940 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: [pid: 80377|app: 0|req: 642/1266] 10.222.0.82 () {56 vars in 1109 bytes} [Tue Aug 2 10:06:00 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 02 10:06:00.247899 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fda0779-cf58-4e19-a5be-e4b959c23eb8 None admin] Authenticating user token {{(pid=80378) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:06:00.349431 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fda0779-cf58-4e19-a5be-e4b959c23eb8 None admin] Validating token access rules against request {{(pid=80378) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:06:00.368722 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG dbcounter [-] [80378] Writing DB stats keystone:SELECT=100 {{(pid=80378) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 10:06:00.382987 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15aab12c-9ebd-46c8-8c1f-56819c662435 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '891005fa5ff74041b75747d16cc66798', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80378) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:06:00.383630 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15aab12c-9ebd-46c8-8c1f-56819c662435 None admin] REQUEST_METHOD: `GET` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:06:00.383913 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15aab12c-9ebd-46c8-8c1f-56819c662435 None admin] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:06:00.384188 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15aab12c-9ebd-46c8-8c1f-56819c662435 None admin] PATH_INFO: `/v3/users/placement` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:06:00.391853 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15aab12c-9ebd-46c8-8c1f-56819c662435 None admin] RBAC: Authorizing `identity:get_user(user_id=placement)` {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:06:00.393321 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15aab12c-9ebd-46c8-8c1f-56819c662435 None admin] RBAC: Authorization granted {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:06:00.405272 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application [None req-15aab12c-9ebd-46c8-8c1f-56819c662435 None admin] Could not find user: placement.: keystone.exception.UserNotFound: Could not find user: placement. Aug 02 10:06:00.405272 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 02 10:06:00.405272 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 02 10:06:00.405272 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 02 10:06:00.405272 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 02 10:06:00.405272 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 02 10:06:00.405272 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 02 10:06:00.405272 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 02 10:06:00.405272 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 02 10:06:00.405272 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 02 10:06:00.405272 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 02 10:06:00.405272 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 02 10:06:00.405272 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 02 10:06:00.405272 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 02 10:06:00.405272 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 02 10:06:00.405272 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 02 10:06:00.406961 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 02 10:06:00.406961 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 02 10:06:00.406961 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 02 10:06:00.406961 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 02 10:06:00.406961 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 02 10:06:00.406961 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 02 10:06:00.406961 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 02 10:06:00.406961 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 02 10:06:00.406961 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 02 10:06:00.406961 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application return self.get_or_create( Aug 02 10:06:00.406961 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 02 10:06:00.406961 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application with Lock( Aug 02 10:06:00.406961 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 02 10:06:00.406961 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application return self._enter() Aug 02 10:06:00.406961 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 02 10:06:00.406961 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 02 10:06:00.406961 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 02 10:06:00.406961 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application return self.creator() Aug 02 10:06:00.408047 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 02 10:06:00.408047 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application created_value = creator( Aug 02 10:06:00.408047 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 02 10:06:00.408047 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 02 10:06:00.408047 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 02 10:06:00.408047 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 02 10:06:00.408047 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 02 10:06:00.408047 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 02 10:06:00.408047 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: placement. Aug 02 10:06:00.408047 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application Aug 02 10:06:00.408047 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 625/1267] 10.222.0.82 () {56 vars in 1108 bytes} [Tue Aug 2 10:06:00 2022] GET /identity/v3/users/placement => generated 87 bytes in 159 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 02 10:06:00.412309 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-123cc143-6614-485d-9d33-b5f2d51dea5a None admin] Authenticating user token {{(pid=80377) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:06:00.513015 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-123cc143-6614-485d-9d33-b5f2d51dea5a None admin] Validating token access rules against request {{(pid=80377) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:06:00.547845 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c9836a8-4f28-4dae-a2ef-8608911fe3ee None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '891005fa5ff74041b75747d16cc66798', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80377) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:06:00.548481 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c9836a8-4f28-4dae-a2ef-8608911fe3ee None admin] REQUEST_METHOD: `GET` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:06:00.548767 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c9836a8-4f28-4dae-a2ef-8608911fe3ee None admin] SCRIPT_NAME: `/identity` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:06:00.549042 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c9836a8-4f28-4dae-a2ef-8608911fe3ee None admin] PATH_INFO: `/v3/users` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:06:00.549873 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c9836a8-4f28-4dae-a2ef-8608911fe3ee None admin] RBAC: Adding query filter params (domain_id=default, name=placement) {{(pid=80377) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 02 10:06:00.550190 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c9836a8-4f28-4dae-a2ef-8608911fe3ee None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:06:00.551634 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c9836a8-4f28-4dae-a2ef-8608911fe3ee None admin] RBAC: Authorization granted {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:06:00.561655 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG dbcounter [-] [80377] Writing DB stats keystone:SELECT=100 {{(pid=80377) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 10:06:00.572768 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: [pid: 80377|app: 0|req: 643/1268] 10.222.0.82 () {56 vars in 1143 bytes} [Tue Aug 2 10:06:00 2022] GET /identity/v3/users?domain_id=default&name=placement => generated 378 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 02 10:06:00.578691 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15aab12c-9ebd-46c8-8c1f-56819c662435 None admin] Authenticating user token {{(pid=80378) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:06:00.677919 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15aab12c-9ebd-46c8-8c1f-56819c662435 None admin] Validating token access rules against request {{(pid=80378) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:06:00.711580 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7c69955-3171-4468-b457-3425682300b4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '891005fa5ff74041b75747d16cc66798', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80378) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:06:00.712222 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7c69955-3171-4468-b457-3425682300b4 None admin] REQUEST_METHOD: `GET` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:06:00.712504 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7c69955-3171-4468-b457-3425682300b4 None admin] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:06:00.712778 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7c69955-3171-4468-b457-3425682300b4 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:06:00.723492 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7c69955-3171-4468-b457-3425682300b4 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:06:00.725540 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7c69955-3171-4468-b457-3425682300b4 None admin] RBAC: Authorization granted {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:06:00.739696 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application [None req-a7c69955-3171-4468-b457-3425682300b4 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 02 10:06:00.739696 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 02 10:06:00.739696 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 02 10:06:00.739696 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 02 10:06:00.739696 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 02 10:06:00.739696 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 02 10:06:00.739696 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 02 10:06:00.739696 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 02 10:06:00.739696 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 02 10:06:00.739696 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 02 10:06:00.739696 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 02 10:06:00.739696 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 02 10:06:00.739696 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 02 10:06:00.739696 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 02 10:06:00.739696 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 02 10:06:00.741387 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 02 10:06:00.741387 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 02 10:06:00.741387 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 02 10:06:00.741387 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 02 10:06:00.741387 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 02 10:06:00.741387 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 02 10:06:00.741387 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application return self.get_or_create( Aug 02 10:06:00.741387 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 02 10:06:00.741387 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application with Lock( Aug 02 10:06:00.741387 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 02 10:06:00.741387 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application return self._enter() Aug 02 10:06:00.741387 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 02 10:06:00.741387 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 02 10:06:00.741387 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 02 10:06:00.741387 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application return self.creator() Aug 02 10:06:00.741387 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 02 10:06:00.741387 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application created_value = creator( Aug 02 10:06:00.742528 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 02 10:06:00.742528 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 02 10:06:00.742528 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 02 10:06:00.742528 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application Aug 02 10:06:00.742528 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 626/1269] 10.222.0.82 () {56 vars in 1108 bytes} [Tue Aug 2 10:06:00 2022] GET /identity/v3/domains/Default => generated 87 bytes in 163 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 02 10:06:00.747108 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c9836a8-4f28-4dae-a2ef-8608911fe3ee None admin] Authenticating user token {{(pid=80377) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:06:00.843631 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c9836a8-4f28-4dae-a2ef-8608911fe3ee None admin] Validating token access rules against request {{(pid=80377) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:06:00.877389 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a73b7da3-df06-4406-87cf-1b3f3cd897fd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '891005fa5ff74041b75747d16cc66798', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80377) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:06:00.878046 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a73b7da3-df06-4406-87cf-1b3f3cd897fd None admin] REQUEST_METHOD: `GET` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:06:00.878328 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a73b7da3-df06-4406-87cf-1b3f3cd897fd None admin] SCRIPT_NAME: `/identity` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:06:00.878602 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a73b7da3-df06-4406-87cf-1b3f3cd897fd None admin] PATH_INFO: `/v3/domains` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:06:00.879301 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a73b7da3-df06-4406-87cf-1b3f3cd897fd None admin] RBAC: Adding query filter params (name=Default) {{(pid=80377) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 02 10:06:00.879630 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a73b7da3-df06-4406-87cf-1b3f3cd897fd None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:06:00.881043 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a73b7da3-df06-4406-87cf-1b3f3cd897fd None admin] RBAC: Authorization granted {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:06:00.892418 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: [pid: 80377|app: 0|req: 644/1270] 10.222.0.82 () {56 vars in 1109 bytes} [Tue Aug 2 10:06:00 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 146 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 02 10:06:00.898394 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7c69955-3171-4468-b457-3425682300b4 None admin] Authenticating user token {{(pid=80378) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:06:00.930250 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG dbcounter [-] [80378] Writing DB stats keystone:SELECT=100 {{(pid=80378) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 10:06:00.997445 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7c69955-3171-4468-b457-3425682300b4 None admin] Validating token access rules against request {{(pid=80378) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:06:01.030021 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-491fe941-3eb3-4a53-8895-5562731507e7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '891005fa5ff74041b75747d16cc66798', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80378) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:06:01.030657 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-491fe941-3eb3-4a53-8895-5562731507e7 None admin] REQUEST_METHOD: `GET` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:06:01.030941 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-491fe941-3eb3-4a53-8895-5562731507e7 None admin] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:06:01.031218 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-491fe941-3eb3-4a53-8895-5562731507e7 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:06:01.038393 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-491fe941-3eb3-4a53-8895-5562731507e7 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:06:01.039835 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-491fe941-3eb3-4a53-8895-5562731507e7 None admin] RBAC: Authorization granted {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:06:01.051274 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application [None req-491fe941-3eb3-4a53-8895-5562731507e7 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 02 10:06:01.051274 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 02 10:06:01.051274 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 02 10:06:01.051274 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 02 10:06:01.051274 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 02 10:06:01.051274 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 02 10:06:01.051274 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 02 10:06:01.051274 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 02 10:06:01.051274 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 02 10:06:01.051274 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 02 10:06:01.051274 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 02 10:06:01.051274 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 02 10:06:01.051274 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 02 10:06:01.051274 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 02 10:06:01.051274 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 02 10:06:01.052990 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 02 10:06:01.052990 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 02 10:06:01.052990 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 02 10:06:01.052990 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 02 10:06:01.052990 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 02 10:06:01.052990 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 02 10:06:01.052990 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application return self.get_or_create( Aug 02 10:06:01.052990 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 02 10:06:01.052990 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application with Lock( Aug 02 10:06:01.052990 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 02 10:06:01.052990 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application return self._enter() Aug 02 10:06:01.052990 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 02 10:06:01.052990 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 02 10:06:01.052990 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 02 10:06:01.052990 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application return self.creator() Aug 02 10:06:01.052990 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 02 10:06:01.052990 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application created_value = creator( Aug 02 10:06:01.054414 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 02 10:06:01.054414 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 02 10:06:01.054414 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 02 10:06:01.054414 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 02 10:06:01.054414 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 02 10:06:01.054414 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 02 10:06:01.054414 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 02 10:06:01.054414 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application Aug 02 10:06:01.054414 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 627/1271] 10.222.0.82 () {56 vars in 1111 bytes} [Tue Aug 2 10:06:00 2022] GET /identity/v3/projects/service => generated 88 bytes in 155 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 02 10:06:01.058803 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a73b7da3-df06-4406-87cf-1b3f3cd897fd None admin] Authenticating user token {{(pid=80377) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:06:01.128192 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG dbcounter [-] [80377] Writing DB stats keystone:SELECT=100 {{(pid=80377) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 10:06:01.172810 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a73b7da3-df06-4406-87cf-1b3f3cd897fd None admin] Validating token access rules against request {{(pid=80377) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:06:01.205008 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4044b5a6-9a58-43fc-b0cb-8b241a7b0adb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '891005fa5ff74041b75747d16cc66798', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80377) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:06:01.205630 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4044b5a6-9a58-43fc-b0cb-8b241a7b0adb None admin] REQUEST_METHOD: `GET` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:06:01.205950 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4044b5a6-9a58-43fc-b0cb-8b241a7b0adb None admin] SCRIPT_NAME: `/identity` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:06:01.206222 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4044b5a6-9a58-43fc-b0cb-8b241a7b0adb None admin] PATH_INFO: `/v3/projects` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:06:01.207030 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4044b5a6-9a58-43fc-b0cb-8b241a7b0adb None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80377) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 02 10:06:01.207284 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4044b5a6-9a58-43fc-b0cb-8b241a7b0adb None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:06:01.209303 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4044b5a6-9a58-43fc-b0cb-8b241a7b0adb None admin] RBAC: Authorization granted {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:06:01.221901 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: [pid: 80377|app: 0|req: 645/1272] 10.222.0.82 () {56 vars in 1148 bytes} [Tue Aug 2 10:06:01 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 02 10:06:01.228427 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-491fe941-3eb3-4a53-8895-5562731507e7 None admin] Authenticating user token {{(pid=80378) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:06:01.329522 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG dbcounter [-] [80378] Writing DB stats keystone:SELECT=100 {{(pid=80378) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 10:06:01.338989 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-491fe941-3eb3-4a53-8895-5562731507e7 None admin] Validating token access rules against request {{(pid=80378) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:06:01.372636 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ef78e76-5932-456e-9034-abedf9f7721f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '891005fa5ff74041b75747d16cc66798', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80378) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:06:01.373318 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ef78e76-5932-456e-9034-abedf9f7721f None admin] REQUEST_METHOD: `PUT` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:06:01.373603 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ef78e76-5932-456e-9034-abedf9f7721f None admin] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:06:01.373917 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ef78e76-5932-456e-9034-abedf9f7721f None admin] PATH_INFO: `/v3/projects/f1b38ebb804c41c69772c998aae72c41/users/ae6e910e50d74f29bfb89bf18d95a908/roles/237966442f05411191105bf336bc14f3` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:06:01.376023 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ef78e76-5932-456e-9034-abedf9f7721f None admin] RBAC: Authorizing `identity:create_grant(project_id=f1b38ebb804c41c69772c998aae72c41, user_id=ae6e910e50d74f29bfb89bf18d95a908, role_id=237966442f05411191105bf336bc14f3)` {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:06:01.377548 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ef78e76-5932-456e-9034-abedf9f7721f None admin] RBAC: Authorization granted {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:06:01.391867 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 628/1273] 10.222.0.82 () {58 vars in 1439 bytes} [Tue Aug 2 10:06:01 2022] PUT /identity/v3/projects/f1b38ebb804c41c69772c998aae72c41/users/ae6e910e50d74f29bfb89bf18d95a908/roles/237966442f05411191105bf336bc14f3 => generated 0 bytes in 164 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 02 10:06:03.409694 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46b756a3-2cda-498d-9b0b-969f5d5f683c None None] REQUEST_METHOD: `GET` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:06:03.410282 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46b756a3-2cda-498d-9b0b-969f5d5f683c None None] SCRIPT_NAME: `/identity` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:06:03.410282 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46b756a3-2cda-498d-9b0b-969f5d5f683c None None] PATH_INFO: `/` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:06:03.411631 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: [pid: 80377|app: 0|req: 646/1274] 10.222.0.82 () {54 vars in 926 bytes} [Tue Aug 2 10:06:03 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 02 10:06:03.416657 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-216e317f-1e40-4ddd-b3f7-07567adb8dad None None] REQUEST_METHOD: `POST` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:06:03.417061 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-216e317f-1e40-4ddd-b3f7-07567adb8dad None None] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:06:03.417225 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-216e317f-1e40-4ddd-b3f7-07567adb8dad None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:06:03.435805 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.auth.core [None req-216e317f-1e40-4ddd-b3f7-07567adb8dad None None] MFA Rules not processed for user `891005fa5ff74041b75747d16cc66798`. Rule list: `[]` (Enabled: `True`). {{(pid=80378) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 02 10:06:03.436996 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.fernet_utils [None req-216e317f-1e40-4ddd-b3f7-07567adb8dad None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80378) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 10:06:03.502000 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 629/1275] 10.222.0.82 () {58 vars in 1023 bytes} [Tue Aug 2 10:06:03 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 02 10:06:03.525150 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46b756a3-2cda-498d-9b0b-969f5d5f683c None None] Authenticating user token {{(pid=80377) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:06:03.635033 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46b756a3-2cda-498d-9b0b-969f5d5f683c None None] Validating token access rules against request {{(pid=80377) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:06:03.653280 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG dbcounter [-] [80377] Writing DB stats keystone:SELECT=100 {{(pid=80377) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 10:06:03.671283 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6542202b-3f54-4ea4-b432-ca2c3b16b480 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '891005fa5ff74041b75747d16cc66798', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80377) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:06:03.671850 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6542202b-3f54-4ea4-b432-ca2c3b16b480 None admin] REQUEST_METHOD: `GET` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:06:03.672131 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6542202b-3f54-4ea4-b432-ca2c3b16b480 None admin] SCRIPT_NAME: `/identity` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:06:03.672405 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6542202b-3f54-4ea4-b432-ca2c3b16b480 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:06:03.683243 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6542202b-3f54-4ea4-b432-ca2c3b16b480 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 02 10:06:03.688138 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application [None req-6542202b-3f54-4ea4-b432-ca2c3b16b480 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 02 10:06:03.688138 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 02 10:06:03.688138 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 02 10:06:03.688138 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 02 10:06:03.688138 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 02 10:06:03.688138 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 02 10:06:03.688138 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 02 10:06:03.688138 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 02 10:06:03.688138 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 02 10:06:03.688138 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 02 10:06:03.688138 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 02 10:06:03.688138 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 02 10:06:03.688138 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 02 10:06:03.688138 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 02 10:06:03.688138 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 02 10:06:03.688138 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 02 10:06:03.689291 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 02 10:06:03.689291 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 02 10:06:03.689291 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 02 10:06:03.689291 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 02 10:06:03.689291 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 02 10:06:03.689291 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 02 10:06:03.689291 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 02 10:06:03.689291 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 02 10:06:03.689291 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 02 10:06:03.689291 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application return self.get_or_create( Aug 02 10:06:03.689291 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 02 10:06:03.689291 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application with Lock( Aug 02 10:06:03.689291 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 02 10:06:03.689291 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application return self._enter() Aug 02 10:06:03.689291 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 02 10:06:03.689291 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 02 10:06:03.690479 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 02 10:06:03.690479 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application return self.creator() Aug 02 10:06:03.690479 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 02 10:06:03.690479 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application created_value = creator( Aug 02 10:06:03.690479 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 02 10:06:03.690479 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 02 10:06:03.690479 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 02 10:06:03.690479 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 02 10:06:03.690479 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 02 10:06:03.690479 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 02 10:06:03.690479 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 02 10:06:03.690479 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application Aug 02 10:06:03.690479 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: [pid: 80377|app: 0|req: 647/1276] 10.222.0.82 () {56 vars in 1096 bytes} [Tue Aug 2 10:06:03 2022] GET /identity/v3/roles/admin => generated 83 bytes in 165 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 02 10:06:03.696034 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-216e317f-1e40-4ddd-b3f7-07567adb8dad None None] Authenticating user token {{(pid=80378) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:06:03.776677 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG dbcounter [-] [80378] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80378) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 10:06:03.795610 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-216e317f-1e40-4ddd-b3f7-07567adb8dad None None] Validating token access rules against request {{(pid=80378) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:06:03.828678 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f65813a-866b-4fe1-b0d6-0a9b88be3be8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '891005fa5ff74041b75747d16cc66798', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80378) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:06:03.829296 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f65813a-866b-4fe1-b0d6-0a9b88be3be8 None admin] REQUEST_METHOD: `GET` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:06:03.829581 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f65813a-866b-4fe1-b0d6-0a9b88be3be8 None admin] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:06:03.829903 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f65813a-866b-4fe1-b0d6-0a9b88be3be8 None admin] PATH_INFO: `/v3/roles` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:06:03.830637 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f65813a-866b-4fe1-b0d6-0a9b88be3be8 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80378) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 02 10:06:03.830944 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f65813a-866b-4fe1-b0d6-0a9b88be3be8 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:06:03.832316 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f65813a-866b-4fe1-b0d6-0a9b88be3be8 None admin] RBAC: Authorization granted {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:06:03.840183 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 630/1277] 10.222.0.82 () {56 vars in 1099 bytes} [Tue Aug 2 10:06:03 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 145 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 02 10:06:03.845916 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6542202b-3f54-4ea4-b432-ca2c3b16b480 None admin] Authenticating user token {{(pid=80377) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:06:03.946063 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6542202b-3f54-4ea4-b432-ca2c3b16b480 None admin] Validating token access rules against request {{(pid=80377) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:06:03.978466 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93a387c6-2b82-4e01-b756-38d3fa5ec981 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '891005fa5ff74041b75747d16cc66798', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80377) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:06:03.979106 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93a387c6-2b82-4e01-b756-38d3fa5ec981 None admin] REQUEST_METHOD: `GET` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:06:03.979389 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93a387c6-2b82-4e01-b756-38d3fa5ec981 None admin] SCRIPT_NAME: `/identity` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:06:03.979667 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93a387c6-2b82-4e01-b756-38d3fa5ec981 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:06:03.989975 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93a387c6-2b82-4e01-b756-38d3fa5ec981 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:06:03.991946 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93a387c6-2b82-4e01-b756-38d3fa5ec981 None admin] RBAC: Authorization granted {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:06:04.005758 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application [None req-93a387c6-2b82-4e01-b756-38d3fa5ec981 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 02 10:06:04.005758 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 02 10:06:04.005758 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 02 10:06:04.005758 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 02 10:06:04.005758 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 02 10:06:04.005758 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 02 10:06:04.005758 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 02 10:06:04.005758 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 02 10:06:04.005758 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 02 10:06:04.005758 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 02 10:06:04.005758 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 02 10:06:04.005758 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 02 10:06:04.005758 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 02 10:06:04.005758 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 02 10:06:04.005758 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 02 10:06:04.006843 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 02 10:06:04.006843 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 02 10:06:04.006843 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 02 10:06:04.006843 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 02 10:06:04.006843 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 02 10:06:04.006843 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 02 10:06:04.006843 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application return self.get_or_create( Aug 02 10:06:04.006843 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 02 10:06:04.006843 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application with Lock( Aug 02 10:06:04.006843 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 02 10:06:04.006843 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application return self._enter() Aug 02 10:06:04.006843 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 02 10:06:04.006843 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 02 10:06:04.006843 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 02 10:06:04.006843 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application return self.creator() Aug 02 10:06:04.006843 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 02 10:06:04.006843 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application created_value = creator( Aug 02 10:06:04.007810 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 02 10:06:04.007810 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 02 10:06:04.007810 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 02 10:06:04.007810 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application Aug 02 10:06:04.007810 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: [pid: 80377|app: 0|req: 648/1278] 10.222.0.82 () {56 vars in 1108 bytes} [Tue Aug 2 10:06:03 2022] GET /identity/v3/domains/Default => generated 87 bytes in 162 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 02 10:06:04.012799 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f65813a-866b-4fe1-b0d6-0a9b88be3be8 None admin] Authenticating user token {{(pid=80378) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:06:04.119010 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f65813a-866b-4fe1-b0d6-0a9b88be3be8 None admin] Validating token access rules against request {{(pid=80378) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:06:04.152105 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef365854-68b8-4b58-b2c6-ffbee14f4dda None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '891005fa5ff74041b75747d16cc66798', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80378) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:06:04.152725 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef365854-68b8-4b58-b2c6-ffbee14f4dda None admin] REQUEST_METHOD: `GET` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:06:04.153008 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef365854-68b8-4b58-b2c6-ffbee14f4dda None admin] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:06:04.153283 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef365854-68b8-4b58-b2c6-ffbee14f4dda None admin] PATH_INFO: `/v3/domains` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:06:04.154120 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef365854-68b8-4b58-b2c6-ffbee14f4dda None admin] RBAC: Adding query filter params (name=Default) {{(pid=80378) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 02 10:06:04.154426 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef365854-68b8-4b58-b2c6-ffbee14f4dda None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:06:04.155793 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef365854-68b8-4b58-b2c6-ffbee14f4dda None admin] RBAC: Authorization granted {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:06:04.167750 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 631/1279] 10.222.0.82 () {56 vars in 1109 bytes} [Tue Aug 2 10:06:04 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 02 10:06:04.173855 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93a387c6-2b82-4e01-b756-38d3fa5ec981 None admin] Authenticating user token {{(pid=80377) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:06:04.203916 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG dbcounter [-] [80377] Writing DB stats keystone:SELECT=100 {{(pid=80377) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 10:06:04.273633 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93a387c6-2b82-4e01-b756-38d3fa5ec981 None admin] Validating token access rules against request {{(pid=80377) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:06:04.307241 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d17cddba-703e-4e05-9480-47fe188e09ed None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '891005fa5ff74041b75747d16cc66798', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80377) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:06:04.307892 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d17cddba-703e-4e05-9480-47fe188e09ed None admin] REQUEST_METHOD: `GET` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:06:04.308171 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d17cddba-703e-4e05-9480-47fe188e09ed None admin] SCRIPT_NAME: `/identity` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:06:04.308450 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d17cddba-703e-4e05-9480-47fe188e09ed None admin] PATH_INFO: `/v3/users/placement` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:06:04.316238 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d17cddba-703e-4e05-9480-47fe188e09ed None admin] RBAC: Authorizing `identity:get_user(user_id=placement)` {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:06:04.317686 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d17cddba-703e-4e05-9480-47fe188e09ed None admin] RBAC: Authorization granted {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:06:04.329182 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application [None req-d17cddba-703e-4e05-9480-47fe188e09ed None admin] Could not find user: placement.: keystone.exception.UserNotFound: Could not find user: placement. Aug 02 10:06:04.329182 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 02 10:06:04.329182 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 02 10:06:04.329182 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 02 10:06:04.329182 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 02 10:06:04.329182 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 02 10:06:04.329182 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 02 10:06:04.329182 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 02 10:06:04.329182 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 02 10:06:04.329182 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 02 10:06:04.329182 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 02 10:06:04.329182 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 02 10:06:04.329182 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 02 10:06:04.329182 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 02 10:06:04.329182 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 02 10:06:04.329182 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 02 10:06:04.330417 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 02 10:06:04.330417 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 02 10:06:04.330417 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 02 10:06:04.330417 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 02 10:06:04.330417 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 02 10:06:04.330417 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 02 10:06:04.330417 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 02 10:06:04.330417 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 02 10:06:04.330417 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 02 10:06:04.330417 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application return self.get_or_create( Aug 02 10:06:04.330417 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 02 10:06:04.330417 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application with Lock( Aug 02 10:06:04.330417 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 02 10:06:04.330417 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application return self._enter() Aug 02 10:06:04.330417 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 02 10:06:04.330417 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 02 10:06:04.330417 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 02 10:06:04.330417 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application return self.creator() Aug 02 10:06:04.331770 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 02 10:06:04.331770 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application created_value = creator( Aug 02 10:06:04.331770 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 02 10:06:04.331770 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 02 10:06:04.331770 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 02 10:06:04.331770 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 02 10:06:04.331770 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 02 10:06:04.331770 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 02 10:06:04.331770 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: placement. Aug 02 10:06:04.331770 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application Aug 02 10:06:04.331770 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: [pid: 80377|app: 0|req: 649/1280] 10.222.0.82 () {56 vars in 1108 bytes} [Tue Aug 2 10:06:04 2022] GET /identity/v3/users/placement => generated 87 bytes in 157 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 02 10:06:04.336723 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef365854-68b8-4b58-b2c6-ffbee14f4dda None admin] Authenticating user token {{(pid=80378) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:06:04.340510 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG dbcounter [-] [80378] Writing DB stats keystone:SELECT=100 {{(pid=80378) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 10:06:04.436304 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef365854-68b8-4b58-b2c6-ffbee14f4dda None admin] Validating token access rules against request {{(pid=80378) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:06:04.469906 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd5d638e-e35f-4f57-b13d-a11b06f1af22 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '891005fa5ff74041b75747d16cc66798', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80378) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:06:04.470454 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd5d638e-e35f-4f57-b13d-a11b06f1af22 None admin] REQUEST_METHOD: `GET` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:06:04.470737 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd5d638e-e35f-4f57-b13d-a11b06f1af22 None admin] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:06:04.471012 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd5d638e-e35f-4f57-b13d-a11b06f1af22 None admin] PATH_INFO: `/v3/users` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:06:04.471802 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd5d638e-e35f-4f57-b13d-a11b06f1af22 None admin] RBAC: Adding query filter params (domain_id=default, name=placement) {{(pid=80378) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 02 10:06:04.472111 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd5d638e-e35f-4f57-b13d-a11b06f1af22 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:06:04.473567 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd5d638e-e35f-4f57-b13d-a11b06f1af22 None admin] RBAC: Authorization granted {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:06:04.494371 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 632/1281] 10.222.0.82 () {56 vars in 1143 bytes} [Tue Aug 2 10:06:04 2022] GET /identity/v3/users?domain_id=default&name=placement => generated 378 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 02 10:06:04.500165 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d17cddba-703e-4e05-9480-47fe188e09ed None admin] Authenticating user token {{(pid=80377) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:06:04.608594 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG dbcounter [-] [80377] Writing DB stats keystone:SELECT=100 {{(pid=80377) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 10:06:04.620244 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d17cddba-703e-4e05-9480-47fe188e09ed None admin] Validating token access rules against request {{(pid=80377) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:06:04.652776 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39e3ba1d-1b74-41fb-a322-b412117d302d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '891005fa5ff74041b75747d16cc66798', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80377) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:06:04.653389 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39e3ba1d-1b74-41fb-a322-b412117d302d None admin] REQUEST_METHOD: `GET` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:06:04.653671 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39e3ba1d-1b74-41fb-a322-b412117d302d None admin] SCRIPT_NAME: `/identity` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:06:04.653975 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39e3ba1d-1b74-41fb-a322-b412117d302d None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:06:04.728462 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39e3ba1d-1b74-41fb-a322-b412117d302d None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:06:04.729954 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39e3ba1d-1b74-41fb-a322-b412117d302d None admin] RBAC: Authorization granted {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:06:04.827313 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG dbcounter [-] [80377] Writing DB stats keystone:SELECT=100 {{(pid=80377) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 10:06:04.839043 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: [pid: 80377|app: 0|req: 650/1282] 10.222.0.82 () {58 vars in 1282 bytes} [Tue Aug 2 10:06:04 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 339 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Aug 02 10:06:04.845355 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd5d638e-e35f-4f57-b13d-a11b06f1af22 None admin] Authenticating user token {{(pid=80378) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:06:04.909105 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG dbcounter [-] [80378] Writing DB stats keystone:SELECT=100 {{(pid=80378) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 10:06:04.949691 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd5d638e-e35f-4f57-b13d-a11b06f1af22 None admin] Validating token access rules against request {{(pid=80378) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:06:04.988591 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78084072-72ef-4be3-a372-4df988284ab3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '891005fa5ff74041b75747d16cc66798', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80378) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:06:04.989117 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78084072-72ef-4be3-a372-4df988284ab3 None admin] REQUEST_METHOD: `GET` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:06:04.989399 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78084072-72ef-4be3-a372-4df988284ab3 None admin] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:06:04.989673 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78084072-72ef-4be3-a372-4df988284ab3 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:06:05.000563 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78084072-72ef-4be3-a372-4df988284ab3 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:06:05.002051 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78084072-72ef-4be3-a372-4df988284ab3 None admin] RBAC: Authorization granted {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:06:05.017326 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application [None req-78084072-72ef-4be3-a372-4df988284ab3 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 02 10:06:05.017326 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 02 10:06:05.017326 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 02 10:06:05.017326 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 02 10:06:05.017326 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 02 10:06:05.017326 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 02 10:06:05.017326 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 02 10:06:05.017326 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 02 10:06:05.017326 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 02 10:06:05.017326 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 02 10:06:05.017326 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 02 10:06:05.017326 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 02 10:06:05.017326 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 02 10:06:05.017326 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 02 10:06:05.017326 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 02 10:06:05.018439 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 02 10:06:05.018439 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 02 10:06:05.018439 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 02 10:06:05.018439 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 02 10:06:05.018439 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 02 10:06:05.018439 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 02 10:06:05.018439 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application return self.get_or_create( Aug 02 10:06:05.018439 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 02 10:06:05.018439 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application with Lock( Aug 02 10:06:05.018439 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 02 10:06:05.018439 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application return self._enter() Aug 02 10:06:05.018439 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 02 10:06:05.018439 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 02 10:06:05.018439 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 02 10:06:05.018439 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application return self.creator() Aug 02 10:06:05.018439 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 02 10:06:05.018439 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application created_value = creator( Aug 02 10:06:05.019385 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 02 10:06:05.019385 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 02 10:06:05.019385 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 02 10:06:05.019385 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application Aug 02 10:06:05.019385 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 633/1283] 10.222.0.82 () {56 vars in 1108 bytes} [Tue Aug 2 10:06:04 2022] GET /identity/v3/domains/Default => generated 87 bytes in 174 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 02 10:06:05.024808 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39e3ba1d-1b74-41fb-a322-b412117d302d None admin] Authenticating user token {{(pid=80377) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:06:05.126150 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39e3ba1d-1b74-41fb-a322-b412117d302d None admin] Validating token access rules against request {{(pid=80377) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:06:05.169238 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5ba0457-aeb1-464b-bcd1-21fd72dfcd9a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '891005fa5ff74041b75747d16cc66798', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80377) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:06:05.169892 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5ba0457-aeb1-464b-bcd1-21fd72dfcd9a None admin] REQUEST_METHOD: `GET` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:06:05.170179 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5ba0457-aeb1-464b-bcd1-21fd72dfcd9a None admin] SCRIPT_NAME: `/identity` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:06:05.170451 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5ba0457-aeb1-464b-bcd1-21fd72dfcd9a None admin] PATH_INFO: `/v3/domains` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:06:05.171152 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5ba0457-aeb1-464b-bcd1-21fd72dfcd9a None admin] RBAC: Adding query filter params (name=Default) {{(pid=80377) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 02 10:06:05.171460 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5ba0457-aeb1-464b-bcd1-21fd72dfcd9a None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:06:05.172835 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5ba0457-aeb1-464b-bcd1-21fd72dfcd9a None admin] RBAC: Authorization granted {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:06:05.185108 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: [pid: 80377|app: 0|req: 651/1284] 10.222.0.82 () {56 vars in 1109 bytes} [Tue Aug 2 10:06:05 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 02 10:06:05.191312 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78084072-72ef-4be3-a372-4df988284ab3 None admin] Authenticating user token {{(pid=80378) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:06:05.298337 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78084072-72ef-4be3-a372-4df988284ab3 None admin] Validating token access rules against request {{(pid=80378) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:06:05.313533 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG dbcounter [-] [80378] Writing DB stats keystone:SELECT=100 {{(pid=80378) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 10:06:05.337583 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0283046-ef00-43d6-b073-b17011859b9b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '891005fa5ff74041b75747d16cc66798', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80378) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:06:05.338263 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0283046-ef00-43d6-b073-b17011859b9b None admin] REQUEST_METHOD: `GET` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:06:05.338546 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0283046-ef00-43d6-b073-b17011859b9b None admin] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:06:05.338820 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0283046-ef00-43d6-b073-b17011859b9b None admin] PATH_INFO: `/v3/projects/service` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:06:05.345762 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0283046-ef00-43d6-b073-b17011859b9b None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:06:05.347240 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0283046-ef00-43d6-b073-b17011859b9b None admin] RBAC: Authorization granted {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:06:05.359281 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application [None req-e0283046-ef00-43d6-b073-b17011859b9b None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 02 10:06:05.359281 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 02 10:06:05.359281 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 02 10:06:05.359281 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 02 10:06:05.359281 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 02 10:06:05.359281 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 02 10:06:05.359281 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 02 10:06:05.359281 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 02 10:06:05.359281 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 02 10:06:05.359281 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 02 10:06:05.359281 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 02 10:06:05.359281 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 02 10:06:05.359281 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 02 10:06:05.359281 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 02 10:06:05.359281 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 02 10:06:05.360660 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 02 10:06:05.360660 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 02 10:06:05.360660 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 02 10:06:05.360660 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 02 10:06:05.360660 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 02 10:06:05.360660 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 02 10:06:05.360660 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application return self.get_or_create( Aug 02 10:06:05.360660 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 02 10:06:05.360660 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application with Lock( Aug 02 10:06:05.360660 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 02 10:06:05.360660 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application return self._enter() Aug 02 10:06:05.360660 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 02 10:06:05.360660 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 02 10:06:05.360660 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 02 10:06:05.360660 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application return self.creator() Aug 02 10:06:05.360660 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 02 10:06:05.360660 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application created_value = creator( Aug 02 10:06:05.361557 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 02 10:06:05.361557 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 02 10:06:05.361557 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 02 10:06:05.361557 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 02 10:06:05.361557 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 02 10:06:05.361557 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 02 10:06:05.361557 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 02 10:06:05.361557 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application Aug 02 10:06:05.361557 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 634/1285] 10.222.0.82 () {56 vars in 1111 bytes} [Tue Aug 2 10:06:05 2022] GET /identity/v3/projects/service => generated 88 bytes in 170 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 02 10:06:05.367185 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5ba0457-aeb1-464b-bcd1-21fd72dfcd9a None admin] Authenticating user token {{(pid=80377) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:06:05.421108 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG dbcounter [-] [80377] Writing DB stats keystone:SELECT=100 {{(pid=80377) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 10:06:05.471429 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5ba0457-aeb1-464b-bcd1-21fd72dfcd9a None admin] Validating token access rules against request {{(pid=80377) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:06:05.505744 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ab8331c-3bb4-42b4-ac7e-f4046c8f4b18 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '891005fa5ff74041b75747d16cc66798', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80377) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:06:05.506464 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ab8331c-3bb4-42b4-ac7e-f4046c8f4b18 None admin] REQUEST_METHOD: `GET` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:06:05.506665 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ab8331c-3bb4-42b4-ac7e-f4046c8f4b18 None admin] SCRIPT_NAME: `/identity` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:06:05.506947 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ab8331c-3bb4-42b4-ac7e-f4046c8f4b18 None admin] PATH_INFO: `/v3/projects` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:06:05.507696 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ab8331c-3bb4-42b4-ac7e-f4046c8f4b18 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80377) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 02 10:06:05.508003 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ab8331c-3bb4-42b4-ac7e-f4046c8f4b18 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:06:05.509436 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ab8331c-3bb4-42b4-ac7e-f4046c8f4b18 None admin] RBAC: Authorization granted {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:06:05.522064 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: [pid: 80377|app: 0|req: 652/1286] 10.222.0.82 () {56 vars in 1148 bytes} [Tue Aug 2 10:06:05 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 02 10:06:05.528220 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0283046-ef00-43d6-b073-b17011859b9b None admin] Authenticating user token {{(pid=80378) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:06:05.634890 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0283046-ef00-43d6-b073-b17011859b9b None admin] Validating token access rules against request {{(pid=80378) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:06:05.668053 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2915ec20-5719-4cd2-b130-50f9e5fb7d19 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '891005fa5ff74041b75747d16cc66798', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80378) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:06:05.668925 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2915ec20-5719-4cd2-b130-50f9e5fb7d19 None admin] REQUEST_METHOD: `GET` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:06:05.669211 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2915ec20-5719-4cd2-b130-50f9e5fb7d19 None admin] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:06:05.669486 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2915ec20-5719-4cd2-b130-50f9e5fb7d19 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:06:05.670303 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2915ec20-5719-4cd2-b130-50f9e5fb7d19 None admin] RBAC: Adding query filter params (role.id=237966442f05411191105bf336bc14f3, scope.project.id=f1b38ebb804c41c69772c998aae72c41, user.id=ae6e910e50d74f29bfb89bf18d95a908) {{(pid=80378) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 02 10:06:05.670615 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2915ec20-5719-4cd2-b130-50f9e5fb7d19 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:06:05.672052 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2915ec20-5719-4cd2-b130-50f9e5fb7d19 None admin] RBAC: Authorization granted {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:06:05.678285 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 635/1287] 10.222.0.82 () {56 vars in 1374 bytes} [Tue Aug 2 10:06:05 2022] GET /identity/v3/role_assignments?user.id=ae6e910e50d74f29bfb89bf18d95a908&scope.project.id=f1b38ebb804c41c69772c998aae72c41&role.id=237966442f05411191105bf336bc14f3 => generated 608 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 02 10:06:07.695972 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f3a8ba4-e084-49d2-b7d0-1b0dd162c6c2 None None] REQUEST_METHOD: `GET` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:06:07.696508 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f3a8ba4-e084-49d2-b7d0-1b0dd162c6c2 None None] SCRIPT_NAME: `/identity` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:06:07.696508 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f3a8ba4-e084-49d2-b7d0-1b0dd162c6c2 None None] PATH_INFO: `/` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:06:07.697921 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: [pid: 80377|app: 0|req: 653/1288] 10.222.0.82 () {54 vars in 926 bytes} [Tue Aug 2 10:06:07 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 02 10:06:07.703210 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7ed4e8e-40b4-4025-be59-4249c915858d None None] REQUEST_METHOD: `POST` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:06:07.703656 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7ed4e8e-40b4-4025-be59-4249c915858d None None] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:06:07.703756 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7ed4e8e-40b4-4025-be59-4249c915858d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:06:07.721738 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.auth.core [None req-c7ed4e8e-40b4-4025-be59-4249c915858d None None] MFA Rules not processed for user `891005fa5ff74041b75747d16cc66798`. Rule list: `[]` (Enabled: `True`). {{(pid=80378) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 02 10:06:07.722977 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.fernet_utils [None req-c7ed4e8e-40b4-4025-be59-4249c915858d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80378) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 10:06:07.753087 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG dbcounter [-] [80378] Writing DB stats keystone:SELECT=100 {{(pid=80378) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 10:06:07.790845 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 636/1289] 10.222.0.82 () {58 vars in 1023 bytes} [Tue Aug 2 10:06:07 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 89 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 02 10:06:07.813054 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f3a8ba4-e084-49d2-b7d0-1b0dd162c6c2 None None] Authenticating user token {{(pid=80377) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:06:07.915947 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f3a8ba4-e084-49d2-b7d0-1b0dd162c6c2 None None] Validating token access rules against request {{(pid=80377) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:06:07.926786 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG dbcounter [-] [80377] Writing DB stats keystone:SELECT=100 {{(pid=80377) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 10:06:07.951037 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ccf7a27a-1f9e-4afe-b194-025efffbec1c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '891005fa5ff74041b75747d16cc66798', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80377) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:06:07.951610 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccf7a27a-1f9e-4afe-b194-025efffbec1c None admin] REQUEST_METHOD: `GET` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:06:07.951893 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccf7a27a-1f9e-4afe-b194-025efffbec1c None admin] SCRIPT_NAME: `/identity` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:06:07.952169 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccf7a27a-1f9e-4afe-b194-025efffbec1c None admin] PATH_INFO: `/v3/services/placement` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:06:07.952803 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ccf7a27a-1f9e-4afe-b194-025efffbec1c None admin] RBAC: Authorizing `identity:get_service(service_id=placement)` {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:06:07.954209 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ccf7a27a-1f9e-4afe-b194-025efffbec1c None admin] RBAC: Authorization granted {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:06:07.963112 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application [None req-ccf7a27a-1f9e-4afe-b194-025efffbec1c None admin] Could not find service: placement.: keystone.exception.ServiceNotFound: Could not find service: placement. Aug 02 10:06:07.963112 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 02 10:06:07.963112 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Aug 02 10:06:07.963112 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Aug 02 10:06:07.963112 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Aug 02 10:06:07.963112 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Aug 02 10:06:07.963112 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Aug 02 10:06:07.963112 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 02 10:06:07.963112 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: placement. Aug 02 10:06:07.963112 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application Aug 02 10:06:07.963112 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 02 10:06:07.963112 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application Aug 02 10:06:07.963112 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 02 10:06:07.963112 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 02 10:06:07.963112 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 02 10:06:07.963112 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 02 10:06:07.963112 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 02 10:06:07.964340 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 02 10:06:07.964340 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 02 10:06:07.964340 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 02 10:06:07.964340 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 02 10:06:07.964340 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 02 10:06:07.964340 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 02 10:06:07.964340 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Aug 02 10:06:07.964340 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application return self._get_service(service_id) Aug 02 10:06:07.964340 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Aug 02 10:06:07.964340 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Aug 02 10:06:07.964340 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 02 10:06:07.964340 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 02 10:06:07.964340 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 02 10:06:07.964340 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 02 10:06:07.964340 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 02 10:06:07.964340 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application return self.get_or_create( Aug 02 10:06:07.965374 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 02 10:06:07.965374 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application with Lock( Aug 02 10:06:07.965374 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 02 10:06:07.965374 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application return self._enter() Aug 02 10:06:07.965374 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 02 10:06:07.965374 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 02 10:06:07.965374 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 02 10:06:07.965374 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application return self.creator() Aug 02 10:06:07.965374 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 02 10:06:07.965374 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application created_value = creator( Aug 02 10:06:07.965374 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Aug 02 10:06:07.965374 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 02 10:06:07.965374 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: placement. Aug 02 10:06:07.965374 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application Aug 02 10:06:07.965374 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: [pid: 80377|app: 0|req: 654/1290] 10.222.0.82 () {56 vars in 1117 bytes} [Tue Aug 2 10:06:07 2022] GET /identity/v3/services/placement => generated 90 bytes in 152 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 02 10:06:07.970716 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7ed4e8e-40b4-4025-be59-4249c915858d None None] Authenticating user token {{(pid=80378) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:06:08.077146 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7ed4e8e-40b4-4025-be59-4249c915858d None None] Validating token access rules against request {{(pid=80378) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:06:08.113098 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa905a67-a47b-4e36-980c-2ac86ab1eeeb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '891005fa5ff74041b75747d16cc66798', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80378) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:06:08.113722 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa905a67-a47b-4e36-980c-2ac86ab1eeeb None admin] REQUEST_METHOD: `GET` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:06:08.114037 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa905a67-a47b-4e36-980c-2ac86ab1eeeb None admin] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:06:08.114353 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa905a67-a47b-4e36-980c-2ac86ab1eeeb None admin] PATH_INFO: `/v3/services` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:06:08.114999 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa905a67-a47b-4e36-980c-2ac86ab1eeeb None admin] RBAC: Adding query filter params (name=placement) {{(pid=80378) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 02 10:06:08.115820 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa905a67-a47b-4e36-980c-2ac86ab1eeeb None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:06:08.117252 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa905a67-a47b-4e36-980c-2ac86ab1eeeb None admin] RBAC: Authorization granted {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:06:08.122780 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 637/1291] 10.222.0.82 () {56 vars in 1116 bytes} [Tue Aug 2 10:06:07 2022] GET /identity/v3/services?name=placement => generated 126 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 02 10:06:08.130526 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ccf7a27a-1f9e-4afe-b194-025efffbec1c None admin] Authenticating user token {{(pid=80377) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:06:08.232129 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ccf7a27a-1f9e-4afe-b194-025efffbec1c None admin] Validating token access rules against request {{(pid=80377) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:06:08.266892 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97686c8b-92de-4808-b8a2-ecc78b41345a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '891005fa5ff74041b75747d16cc66798', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80377) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:06:08.267503 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97686c8b-92de-4808-b8a2-ecc78b41345a None admin] REQUEST_METHOD: `GET` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:06:08.267782 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97686c8b-92de-4808-b8a2-ecc78b41345a None admin] SCRIPT_NAME: `/identity` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:06:08.268052 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97686c8b-92de-4808-b8a2-ecc78b41345a None admin] PATH_INFO: `/v3/services` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:06:08.268725 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97686c8b-92de-4808-b8a2-ecc78b41345a None admin] RBAC: Adding query filter params (type=placement) {{(pid=80377) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 02 10:06:08.269029 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97686c8b-92de-4808-b8a2-ecc78b41345a None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:06:08.270419 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97686c8b-92de-4808-b8a2-ecc78b41345a None admin] RBAC: Authorization granted {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:06:08.274881 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: [pid: 80377|app: 0|req: 655/1292] 10.222.0.82 () {56 vars in 1116 bytes} [Tue Aug 2 10:06:08 2022] GET /identity/v3/services?type=placement => generated 126 bytes in 145 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 02 10:06:10.286074 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44acaca6-aeee-48c3-a1c5-3e226c267fd9 None None] REQUEST_METHOD: `GET` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:06:10.286619 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44acaca6-aeee-48c3-a1c5-3e226c267fd9 None None] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:06:10.286619 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44acaca6-aeee-48c3-a1c5-3e226c267fd9 None None] PATH_INFO: `/` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:06:10.288557 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 638/1293] 10.222.0.82 () {54 vars in 926 bytes} [Tue Aug 2 10:06:10 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 02 10:06:10.294310 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c63ef20b-f04f-4984-9f52-d5ef2b996175 None None] REQUEST_METHOD: `POST` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:06:10.294713 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c63ef20b-f04f-4984-9f52-d5ef2b996175 None None] SCRIPT_NAME: `/identity` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:06:10.294916 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c63ef20b-f04f-4984-9f52-d5ef2b996175 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:06:10.313066 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.auth.core [None req-c63ef20b-f04f-4984-9f52-d5ef2b996175 None None] MFA Rules not processed for user `891005fa5ff74041b75747d16cc66798`. Rule list: `[]` (Enabled: `True`). {{(pid=80377) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 02 10:06:10.314309 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.fernet_utils [None req-c63ef20b-f04f-4984-9f52-d5ef2b996175 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80377) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 10:06:10.344609 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG dbcounter [-] [80377] Writing DB stats keystone:SELECT=100 {{(pid=80377) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 10:06:10.384169 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: [pid: 80377|app: 0|req: 656/1294] 10.222.0.82 () {58 vars in 1023 bytes} [Tue Aug 2 10:06:10 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 91 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 02 10:06:10.406934 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44acaca6-aeee-48c3-a1c5-3e226c267fd9 None None] Authenticating user token {{(pid=80378) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:06:10.446194 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG dbcounter [-] [80378] Writing DB stats keystone:SELECT=100 {{(pid=80378) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 10:06:10.519436 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44acaca6-aeee-48c3-a1c5-3e226c267fd9 None None] Validating token access rules against request {{(pid=80378) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:06:10.561062 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-392317e4-5bc6-4016-b841-d7c5133b15af None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '891005fa5ff74041b75747d16cc66798', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80378) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:06:10.561745 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-392317e4-5bc6-4016-b841-d7c5133b15af None admin] REQUEST_METHOD: `POST` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:06:10.561926 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-392317e4-5bc6-4016-b841-d7c5133b15af None admin] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:06:10.562340 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-392317e4-5bc6-4016-b841-d7c5133b15af None admin] PATH_INFO: `/v3/services` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:06:10.563079 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-392317e4-5bc6-4016-b841-d7c5133b15af None admin] RBAC: Authorizing `identity:create_service()` {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:06:10.564274 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-392317e4-5bc6-4016-b841-d7c5133b15af None admin] RBAC: Authorization granted {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:06:10.579717 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 639/1295] 10.222.0.82 () {60 vars in 1141 bytes} [Tue Aug 2 10:06:10 2022] POST /identity/v3/services => generated 246 bytes in 173 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 02 10:06:12.607504 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c7582f5-054a-4a62-8ffb-3bac042ec216 None None] REQUEST_METHOD: `GET` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:06:12.608077 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c7582f5-054a-4a62-8ffb-3bac042ec216 None None] SCRIPT_NAME: `/identity` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:06:12.608077 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c7582f5-054a-4a62-8ffb-3bac042ec216 None None] PATH_INFO: `/` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:06:12.609411 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: [pid: 80377|app: 0|req: 657/1296] 10.222.0.82 () {54 vars in 926 bytes} [Tue Aug 2 10:06:12 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 02 10:06:12.614727 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a41b2bd5-1a05-406c-9db0-00c030a57c7e None None] REQUEST_METHOD: `POST` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:06:12.615316 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a41b2bd5-1a05-406c-9db0-00c030a57c7e None None] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:06:12.615316 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a41b2bd5-1a05-406c-9db0-00c030a57c7e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:06:12.634834 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.auth.core [None req-a41b2bd5-1a05-406c-9db0-00c030a57c7e None None] MFA Rules not processed for user `891005fa5ff74041b75747d16cc66798`. Rule list: `[]` (Enabled: `True`). {{(pid=80378) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 02 10:06:12.636768 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.fernet_utils [None req-a41b2bd5-1a05-406c-9db0-00c030a57c7e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80378) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 10:06:12.713677 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 640/1297] 10.222.0.82 () {58 vars in 1023 bytes} [Tue Aug 2 10:06:12 2022] POST /identity/v3/auth/tokens => generated 2221 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 02 10:06:12.737059 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c7582f5-054a-4a62-8ffb-3bac042ec216 None None] Authenticating user token {{(pid=80377) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:06:12.841064 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c7582f5-054a-4a62-8ffb-3bac042ec216 None None] Validating token access rules against request {{(pid=80377) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:06:12.877316 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ae15e00-d4d9-40d1-9043-9da7670888bc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '891005fa5ff74041b75747d16cc66798', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80377) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:06:12.877925 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ae15e00-d4d9-40d1-9043-9da7670888bc None admin] REQUEST_METHOD: `GET` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:06:12.878211 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ae15e00-d4d9-40d1-9043-9da7670888bc None admin] SCRIPT_NAME: `/identity` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:06:12.878486 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ae15e00-d4d9-40d1-9043-9da7670888bc None admin] PATH_INFO: `/v3/services/placement` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:06:12.879122 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ae15e00-d4d9-40d1-9043-9da7670888bc None admin] RBAC: Authorizing `identity:get_service(service_id=placement)` {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:06:12.880508 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ae15e00-d4d9-40d1-9043-9da7670888bc None admin] RBAC: Authorization granted {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:06:12.889900 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application [None req-8ae15e00-d4d9-40d1-9043-9da7670888bc None admin] Could not find service: placement.: keystone.exception.ServiceNotFound: Could not find service: placement. Aug 02 10:06:12.889900 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 02 10:06:12.889900 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Aug 02 10:06:12.889900 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Aug 02 10:06:12.889900 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Aug 02 10:06:12.889900 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Aug 02 10:06:12.889900 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Aug 02 10:06:12.889900 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 02 10:06:12.889900 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: placement. Aug 02 10:06:12.889900 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application Aug 02 10:06:12.889900 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 02 10:06:12.889900 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application Aug 02 10:06:12.889900 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 02 10:06:12.889900 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 02 10:06:12.889900 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 02 10:06:12.889900 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 02 10:06:12.889900 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 02 10:06:12.891178 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 02 10:06:12.891178 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 02 10:06:12.891178 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 02 10:06:12.891178 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 02 10:06:12.891178 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 02 10:06:12.891178 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 02 10:06:12.891178 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Aug 02 10:06:12.891178 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application return self._get_service(service_id) Aug 02 10:06:12.891178 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Aug 02 10:06:12.891178 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Aug 02 10:06:12.891178 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 02 10:06:12.891178 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 02 10:06:12.891178 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 02 10:06:12.891178 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 02 10:06:12.891178 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 02 10:06:12.891178 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application return self.get_or_create( Aug 02 10:06:12.892468 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 02 10:06:12.892468 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application with Lock( Aug 02 10:06:12.892468 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 02 10:06:12.892468 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application return self._enter() Aug 02 10:06:12.892468 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 02 10:06:12.892468 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 02 10:06:12.892468 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 02 10:06:12.892468 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application return self.creator() Aug 02 10:06:12.892468 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 02 10:06:12.892468 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application created_value = creator( Aug 02 10:06:12.892468 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Aug 02 10:06:12.892468 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 02 10:06:12.892468 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: placement. Aug 02 10:06:12.892468 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application Aug 02 10:06:12.892468 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: [pid: 80377|app: 0|req: 658/1298] 10.222.0.82 () {56 vars in 1117 bytes} [Tue Aug 2 10:06:12 2022] GET /identity/v3/services/placement => generated 90 bytes in 155 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 02 10:06:12.897419 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a41b2bd5-1a05-406c-9db0-00c030a57c7e None None] Authenticating user token {{(pid=80378) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:06:12.916903 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG dbcounter [-] [80378] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80378) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 10:06:13.008821 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a41b2bd5-1a05-406c-9db0-00c030a57c7e None None] Validating token access rules against request {{(pid=80378) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:06:13.043551 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24ace6af-bd1d-493f-8a4d-352ebad1c755 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '891005fa5ff74041b75747d16cc66798', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80378) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:06:13.044439 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24ace6af-bd1d-493f-8a4d-352ebad1c755 None admin] REQUEST_METHOD: `GET` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:06:13.044721 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24ace6af-bd1d-493f-8a4d-352ebad1c755 None admin] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:06:13.044996 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24ace6af-bd1d-493f-8a4d-352ebad1c755 None admin] PATH_INFO: `/v3/services` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:06:13.045688 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24ace6af-bd1d-493f-8a4d-352ebad1c755 None admin] RBAC: Adding query filter params (name=placement) {{(pid=80378) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 02 10:06:13.046080 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24ace6af-bd1d-493f-8a4d-352ebad1c755 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:06:13.047469 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24ace6af-bd1d-493f-8a4d-352ebad1c755 None admin] RBAC: Authorization granted {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:06:13.052919 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 641/1299] 10.222.0.82 () {56 vars in 1116 bytes} [Tue Aug 2 10:06:12 2022] GET /identity/v3/services?name=placement => generated 358 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 02 10:06:13.059013 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ae15e00-d4d9-40d1-9043-9da7670888bc None admin] Authenticating user token {{(pid=80377) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:06:13.091862 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG dbcounter [-] [80377] Writing DB stats keystone:SELECT=100 {{(pid=80377) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 10:06:13.167633 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ae15e00-d4d9-40d1-9043-9da7670888bc None admin] Validating token access rules against request {{(pid=80377) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:06:13.201983 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4d16fbc-799f-4ade-a17c-8177ee8d41b6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '891005fa5ff74041b75747d16cc66798', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80377) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:06:13.202611 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4d16fbc-799f-4ade-a17c-8177ee8d41b6 None admin] REQUEST_METHOD: `GET` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:06:13.202902 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4d16fbc-799f-4ade-a17c-8177ee8d41b6 None admin] SCRIPT_NAME: `/identity` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:06:13.203176 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4d16fbc-799f-4ade-a17c-8177ee8d41b6 None admin] PATH_INFO: `/v3/endpoints` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:06:13.203929 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4d16fbc-799f-4ade-a17c-8177ee8d41b6 None admin] RBAC: Adding query filter params (interface=public, service_id=0a5b2a77f3ed479da1fd36b16d562716, region_id=RegionOne) {{(pid=80377) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 02 10:06:13.204244 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4d16fbc-799f-4ade-a17c-8177ee8d41b6 None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:06:13.205616 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4d16fbc-799f-4ade-a17c-8177ee8d41b6 None admin] RBAC: Authorization granted {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:06:13.211692 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: [pid: 80377|app: 0|req: 659/1300] 10.222.0.82 () {56 vars in 1251 bytes} [Tue Aug 2 10:06:13 2022] GET /identity/v3/endpoints?service_id=0a5b2a77f3ed479da1fd36b16d562716&interface=public®ion_id=RegionOne => generated 194 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 02 10:06:13.217771 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24ace6af-bd1d-493f-8a4d-352ebad1c755 None admin] Authenticating user token {{(pid=80378) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:06:13.309579 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG dbcounter [-] [80378] Writing DB stats keystone:SELECT=100 {{(pid=80378) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 10:06:13.328491 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24ace6af-bd1d-493f-8a4d-352ebad1c755 None admin] Validating token access rules against request {{(pid=80378) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:06:13.364511 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ab9086a-9d75-496b-9dd4-b7d696ef5311 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '891005fa5ff74041b75747d16cc66798', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80378) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:06:13.365126 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ab9086a-9d75-496b-9dd4-b7d696ef5311 None admin] REQUEST_METHOD: `GET` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:06:13.365441 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ab9086a-9d75-496b-9dd4-b7d696ef5311 None admin] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:06:13.365718 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ab9086a-9d75-496b-9dd4-b7d696ef5311 None admin] PATH_INFO: `/v3/services` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:06:13.366501 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ab9086a-9d75-496b-9dd4-b7d696ef5311 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:06:13.367957 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ab9086a-9d75-496b-9dd4-b7d696ef5311 None admin] RBAC: Authorization granted {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:06:13.374877 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 642/1301] 10.222.0.82 () {56 vars in 1087 bytes} [Tue Aug 2 10:06:13 2022] GET /identity/v3/services => generated 2191 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 02 10:06:15.398380 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94173eaf-4f6d-4c1d-8fb5-65e441f74479 None None] REQUEST_METHOD: `GET` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:06:15.398953 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94173eaf-4f6d-4c1d-8fb5-65e441f74479 None None] SCRIPT_NAME: `/identity` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:06:15.398953 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94173eaf-4f6d-4c1d-8fb5-65e441f74479 None None] PATH_INFO: `/` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:06:15.400279 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: [pid: 80377|app: 0|req: 660/1302] 10.222.0.82 () {54 vars in 926 bytes} [Tue Aug 2 10:06:15 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 02 10:06:15.405490 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9aa481c1-6d1a-4876-baee-43b73a554c84 None None] REQUEST_METHOD: `POST` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:06:15.405944 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9aa481c1-6d1a-4876-baee-43b73a554c84 None None] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:06:15.406064 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9aa481c1-6d1a-4876-baee-43b73a554c84 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:06:15.424654 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.auth.core [None req-9aa481c1-6d1a-4876-baee-43b73a554c84 None None] MFA Rules not processed for user `891005fa5ff74041b75747d16cc66798`. Rule list: `[]` (Enabled: `True`). {{(pid=80378) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 02 10:06:15.425862 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.fernet_utils [None req-9aa481c1-6d1a-4876-baee-43b73a554c84 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80378) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 10:06:15.491879 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 643/1303] 10.222.0.82 () {58 vars in 1023 bytes} [Tue Aug 2 10:06:15 2022] POST /identity/v3/auth/tokens => generated 2221 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 02 10:06:15.514320 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94173eaf-4f6d-4c1d-8fb5-65e441f74479 None None] Authenticating user token {{(pid=80377) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:06:15.616962 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG dbcounter [-] [80377] Writing DB stats keystone:SELECT=100 {{(pid=80377) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 10:06:15.623882 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94173eaf-4f6d-4c1d-8fb5-65e441f74479 None None] Validating token access rules against request {{(pid=80377) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:06:15.657752 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3d7a642-0365-4a54-91a2-9cc948436d5e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '891005fa5ff74041b75747d16cc66798', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80377) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:06:15.658348 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3d7a642-0365-4a54-91a2-9cc948436d5e None admin] REQUEST_METHOD: `GET` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:06:15.658632 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3d7a642-0365-4a54-91a2-9cc948436d5e None admin] SCRIPT_NAME: `/identity` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:06:15.658909 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3d7a642-0365-4a54-91a2-9cc948436d5e None admin] PATH_INFO: `/v3/services/placement` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:06:15.659549 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3d7a642-0365-4a54-91a2-9cc948436d5e None admin] RBAC: Authorizing `identity:get_service(service_id=placement)` {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:06:15.660919 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3d7a642-0365-4a54-91a2-9cc948436d5e None admin] RBAC: Authorization granted {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:06:15.670814 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application [None req-e3d7a642-0365-4a54-91a2-9cc948436d5e None admin] Could not find service: placement.: keystone.exception.ServiceNotFound: Could not find service: placement. Aug 02 10:06:15.670814 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 02 10:06:15.670814 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Aug 02 10:06:15.670814 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Aug 02 10:06:15.670814 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Aug 02 10:06:15.670814 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Aug 02 10:06:15.670814 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Aug 02 10:06:15.670814 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 02 10:06:15.670814 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: placement. Aug 02 10:06:15.670814 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application Aug 02 10:06:15.670814 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 02 10:06:15.670814 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application Aug 02 10:06:15.670814 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 02 10:06:15.670814 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 02 10:06:15.670814 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 02 10:06:15.670814 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 02 10:06:15.670814 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 02 10:06:15.672047 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 02 10:06:15.672047 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 02 10:06:15.672047 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 02 10:06:15.672047 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 02 10:06:15.672047 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 02 10:06:15.672047 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 02 10:06:15.672047 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Aug 02 10:06:15.672047 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application return self._get_service(service_id) Aug 02 10:06:15.672047 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Aug 02 10:06:15.672047 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Aug 02 10:06:15.672047 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 02 10:06:15.672047 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 02 10:06:15.672047 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 02 10:06:15.672047 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 02 10:06:15.672047 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 02 10:06:15.672047 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application return self.get_or_create( Aug 02 10:06:15.673024 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 02 10:06:15.673024 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application with Lock( Aug 02 10:06:15.673024 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 02 10:06:15.673024 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application return self._enter() Aug 02 10:06:15.673024 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 02 10:06:15.673024 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 02 10:06:15.673024 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 02 10:06:15.673024 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application return self.creator() Aug 02 10:06:15.673024 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 02 10:06:15.673024 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application created_value = creator( Aug 02 10:06:15.673024 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Aug 02 10:06:15.673024 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 02 10:06:15.673024 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: placement. Aug 02 10:06:15.673024 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: ERROR keystone.server.flask.application Aug 02 10:06:15.673024 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: [pid: 80377|app: 0|req: 661/1304] 10.222.0.82 () {56 vars in 1117 bytes} [Tue Aug 2 10:06:15 2022] GET /identity/v3/services/placement => generated 90 bytes in 158 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 02 10:06:15.678502 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9aa481c1-6d1a-4876-baee-43b73a554c84 None None] Authenticating user token {{(pid=80378) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:06:15.754278 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG dbcounter [-] [80378] Writing DB stats keystone:SELECT=100 {{(pid=80378) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 10:06:15.778191 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9aa481c1-6d1a-4876-baee-43b73a554c84 None None] Validating token access rules against request {{(pid=80378) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:06:15.810796 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b0c5b31-a976-480c-8c7f-3045f2947754 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '891005fa5ff74041b75747d16cc66798', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80378) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:06:15.811413 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b0c5b31-a976-480c-8c7f-3045f2947754 None admin] REQUEST_METHOD: `GET` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:06:15.811695 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b0c5b31-a976-480c-8c7f-3045f2947754 None admin] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:06:15.811968 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b0c5b31-a976-480c-8c7f-3045f2947754 None admin] PATH_INFO: `/v3/services` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:06:15.812640 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b0c5b31-a976-480c-8c7f-3045f2947754 None admin] RBAC: Adding query filter params (name=placement) {{(pid=80378) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 02 10:06:15.812947 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b0c5b31-a976-480c-8c7f-3045f2947754 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:06:15.814345 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b0c5b31-a976-480c-8c7f-3045f2947754 None admin] RBAC: Authorization granted {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:06:15.819891 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 644/1305] 10.222.0.82 () {56 vars in 1116 bytes} [Tue Aug 2 10:06:15 2022] GET /identity/v3/services?name=placement => generated 358 bytes in 142 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 02 10:06:15.825773 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3d7a642-0365-4a54-91a2-9cc948436d5e None admin] Authenticating user token {{(pid=80377) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:06:15.924762 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3d7a642-0365-4a54-91a2-9cc948436d5e None admin] Validating token access rules against request {{(pid=80377) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:06:15.958637 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93105e05-95d8-47e8-af9d-3da89c4f5186 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '891005fa5ff74041b75747d16cc66798', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80377) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:06:15.959327 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93105e05-95d8-47e8-af9d-3da89c4f5186 None admin] REQUEST_METHOD: `POST` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:06:15.959681 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93105e05-95d8-47e8-af9d-3da89c4f5186 None admin] SCRIPT_NAME: `/identity` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:06:15.959958 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93105e05-95d8-47e8-af9d-3da89c4f5186 None admin] PATH_INFO: `/v3/endpoints` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:06:15.960659 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93105e05-95d8-47e8-af9d-3da89c4f5186 None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:06:15.962045 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93105e05-95d8-47e8-af9d-3da89c4f5186 None admin] RBAC: Authorization granted {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:06:15.981454 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: [pid: 80377|app: 0|req: 662/1306] 10.222.0.82 () {60 vars in 1144 bytes} [Tue Aug 2 10:06:15 2022] POST /identity/v3/endpoints => generated 331 bytes in 156 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 02 10:06:25.816351 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG dbcounter [-] [80378] Writing DB stats keystone:SELECT=30 {{(pid=80378) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 10:06:25.977626 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG dbcounter [-] [80377] Writing DB stats keystone:SELECT=91,keystone:INSERT=1 {{(pid=80377) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 10:07:23.196198 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2db9071a-6836-427b-b0e1-3a5187839ab0 None None] REQUEST_METHOD: `GET` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:07:23.196752 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2db9071a-6836-427b-b0e1-3a5187839ab0 None None] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:07:23.196752 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2db9071a-6836-427b-b0e1-3a5187839ab0 None None] PATH_INFO: `/` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:07:23.202055 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 645/1307] 10.222.0.82 () {54 vars in 926 bytes} [Tue Aug 2 10:07:23 2022] GET /identity => generated 270 bytes in 7 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 02 10:07:23.203445 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe2fac7e-7c42-44cd-869f-36eff593d860 None None] REQUEST_METHOD: `POST` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:07:23.203870 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe2fac7e-7c42-44cd-869f-36eff593d860 None None] SCRIPT_NAME: `/identity` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:07:23.204050 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe2fac7e-7c42-44cd-869f-36eff593d860 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:07:23.331759 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.auth.core [None req-fe2fac7e-7c42-44cd-869f-36eff593d860 None None] MFA Rules not processed for user `891005fa5ff74041b75747d16cc66798`. Rule list: `[]` (Enabled: `True`). {{(pid=80377) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 02 10:07:23.335560 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.fernet_utils [None req-fe2fac7e-7c42-44cd-869f-36eff593d860 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80377) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 10:07:23.595602 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: [pid: 80377|app: 0|req: 663/1308] 10.222.0.82 () {58 vars in 1023 bytes} [Tue Aug 2 10:07:23 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 393 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 02 10:07:23.650100 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8c3ccaa-eccb-4885-9781-1e6e073a1b77 None None] REQUEST_METHOD: `GET` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:07:23.650433 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8c3ccaa-eccb-4885-9781-1e6e073a1b77 None None] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:07:23.650713 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8c3ccaa-eccb-4885-9781-1e6e073a1b77 None None] PATH_INFO: `/` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:07:23.652091 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 646/1309] 10.222.0.82 () {54 vars in 965 bytes} [Tue Aug 2 10:07:23 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 02 10:07:23.658960 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b4d3c7e-6e0f-4b8d-a53a-8210aee49820 None None] REQUEST_METHOD: `POST` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:07:23.659303 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b4d3c7e-6e0f-4b8d-a53a-8210aee49820 None None] SCRIPT_NAME: `/identity` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:07:23.659734 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b4d3c7e-6e0f-4b8d-a53a-8210aee49820 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:07:23.771613 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.auth.core [None req-0b4d3c7e-6e0f-4b8d-a53a-8210aee49820 None None] MFA Rules not processed for user `33b87170f0634b67bce60750847b5cbe`. Rule list: `[]` (Enabled: `True`). {{(pid=80377) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 02 10:07:23.773697 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.fernet_utils [None req-0b4d3c7e-6e0f-4b8d-a53a-8210aee49820 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80377) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 10:07:23.799227 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG dbcounter [-] [80377] Writing DB stats keystone:SELECT=100 {{(pid=80377) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 10:07:23.869859 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: [pid: 80377|app: 0|req: 664/1310] 10.222.0.82 () {58 vars in 1062 bytes} [Tue Aug 2 10:07:23 2022] POST /identity/v3/auth/tokens => generated 3121 bytes in 212 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 02 10:07:23.881329 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8c3ccaa-eccb-4885-9781-1e6e073a1b77 None None] Authenticating user token {{(pid=80378) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:07:23.883367 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.fernet_utils [None req-a8c3ccaa-eccb-4885-9781-1e6e073a1b77 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80378) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 10:07:24.008368 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8c3ccaa-eccb-4885-9781-1e6e073a1b77 None None] Validating token access rules against request {{(pid=80378) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:07:24.091184 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d066b6c6-1851-48de-a7bd-8a565f39e868 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '33b87170f0634b67bce60750847b5cbe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f1b38ebb804c41c69772c998aae72c41', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80378) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:07:24.092247 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d066b6c6-1851-48de-a7bd-8a565f39e868 service neutron] REQUEST_METHOD: `GET` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:07:24.092898 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d066b6c6-1851-48de-a7bd-8a565f39e868 service neutron] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:07:24.093384 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d066b6c6-1851-48de-a7bd-8a565f39e868 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:07:24.095291 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.fernet_utils [None req-d066b6c6-1851-48de-a7bd-8a565f39e868 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80378) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 10:07:24.158212 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG dbcounter [-] [80378] Writing DB stats keystone:SELECT=100 {{(pid=80378) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 10:07:24.220103 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d066b6c6-1851-48de-a7bd-8a565f39e868 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:07:24.222432 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d066b6c6-1851-48de-a7bd-8a565f39e868 service neutron] RBAC: Authorization granted {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:07:24.256485 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 647/1311] 10.222.0.82 () {60 vars in 1365 bytes} [Tue Aug 2 10:07:23 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 375 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 02 10:07:26.645154 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46f9c94d-3347-4a69-9da5-3b8ae916ba52 None None] REQUEST_METHOD: `GET` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:07:26.645608 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46f9c94d-3347-4a69-9da5-3b8ae916ba52 None None] SCRIPT_NAME: `/identity` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:07:26.645717 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46f9c94d-3347-4a69-9da5-3b8ae916ba52 None None] PATH_INFO: `/` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:07:26.647700 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: [pid: 80377|app: 0|req: 665/1312] 10.222.0.82 () {54 vars in 926 bytes} [Tue Aug 2 10:07:26 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 02 10:07:26.653044 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86dbc789-3f3b-49c2-85e5-0063572ad428 None None] REQUEST_METHOD: `POST` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:07:26.653540 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86dbc789-3f3b-49c2-85e5-0063572ad428 None None] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:07:26.653660 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86dbc789-3f3b-49c2-85e5-0063572ad428 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:07:26.735788 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.auth.core [None req-86dbc789-3f3b-49c2-85e5-0063572ad428 None None] MFA Rules not processed for user `891005fa5ff74041b75747d16cc66798`. Rule list: `[]` (Enabled: `True`). {{(pid=80378) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 02 10:07:26.737177 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.fernet_utils [None req-86dbc789-3f3b-49c2-85e5-0063572ad428 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80378) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 10:07:26.843868 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 648/1313] 10.222.0.82 () {58 vars in 1023 bytes} [Tue Aug 2 10:07:26 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 192 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 02 10:07:26.885139 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46f9c94d-3347-4a69-9da5-3b8ae916ba52 None None] Authenticating user token {{(pid=80377) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:07:26.886943 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.fernet_utils [None req-46f9c94d-3347-4a69-9da5-3b8ae916ba52 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80377) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 10:07:26.990252 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46f9c94d-3347-4a69-9da5-3b8ae916ba52 None None] Validating token access rules against request {{(pid=80377) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:07:27.059898 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG dbcounter [-] [80377] Writing DB stats keystone:SELECT=100 {{(pid=80377) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 10:07:27.062228 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-506b2f94-fb42-43ea-85c2-7cb973d7135b service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '33b87170f0634b67bce60750847b5cbe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f1b38ebb804c41c69772c998aae72c41', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80377) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:07:27.062863 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-506b2f94-fb42-43ea-85c2-7cb973d7135b service neutron] REQUEST_METHOD: `GET` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:07:27.063344 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-506b2f94-fb42-43ea-85c2-7cb973d7135b service neutron] SCRIPT_NAME: `/identity` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:07:27.063827 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-506b2f94-fb42-43ea-85c2-7cb973d7135b service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:07:27.065733 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.fernet_utils [None req-506b2f94-fb42-43ea-85c2-7cb973d7135b service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80377) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 10:07:27.172964 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-506b2f94-fb42-43ea-85c2-7cb973d7135b service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:07:27.174431 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-506b2f94-fb42-43ea-85c2-7cb973d7135b service neutron] RBAC: Authorization granted {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:07:27.192384 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: [pid: 80377|app: 0|req: 666/1314] 10.222.0.82 () {60 vars in 1365 bytes} [Tue Aug 2 10:07:26 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 308 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 02 10:07:29.518342 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cffc41cd-3741-4487-9d18-dc1ac58e5c15 None None] REQUEST_METHOD: `GET` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:07:29.518952 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cffc41cd-3741-4487-9d18-dc1ac58e5c15 None None] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:07:29.518952 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cffc41cd-3741-4487-9d18-dc1ac58e5c15 None None] PATH_INFO: `/` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:07:29.520299 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 649/1315] 10.222.0.82 () {54 vars in 926 bytes} [Tue Aug 2 10:07:29 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 02 10:07:29.525689 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34b7e55f-4444-467e-ac4d-ffaf4027cc1b None None] REQUEST_METHOD: `POST` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:07:29.526315 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34b7e55f-4444-467e-ac4d-ffaf4027cc1b None None] SCRIPT_NAME: `/identity` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:07:29.526315 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34b7e55f-4444-467e-ac4d-ffaf4027cc1b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:07:29.609767 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.auth.core [None req-34b7e55f-4444-467e-ac4d-ffaf4027cc1b None None] MFA Rules not processed for user `71a4a6ffabe74df39f0a7ca7ea7b33b8`. Rule list: `[]` (Enabled: `True`). {{(pid=80377) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 02 10:07:29.611032 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.fernet_utils [None req-34b7e55f-4444-467e-ac4d-ffaf4027cc1b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80377) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 10:07:29.666410 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG dbcounter [-] [80377] Writing DB stats keystone:SELECT=100 {{(pid=80377) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 10:07:29.713383 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: [pid: 80377|app: 0|req: 667/1316] 10.222.0.82 () {58 vars in 1023 bytes} [Tue Aug 2 10:07:29 2022] POST /identity/v3/auth/tokens => generated 3243 bytes in 189 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 02 10:07:29.753081 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cffc41cd-3741-4487-9d18-dc1ac58e5c15 None None] Authenticating user token {{(pid=80378) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:07:29.754740 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.fernet_utils [None req-cffc41cd-3741-4487-9d18-dc1ac58e5c15 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80378) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 10:07:29.764978 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG dbcounter [-] [80378] Writing DB stats keystone:SELECT=100 {{(pid=80378) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 10:07:29.858766 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cffc41cd-3741-4487-9d18-dc1ac58e5c15 None None] Validating token access rules against request {{(pid=80378) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:07:29.936429 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c832a51-d1e3-4afc-b257-a4f6c5442dd5 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '33b87170f0634b67bce60750847b5cbe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f1b38ebb804c41c69772c998aae72c41', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80378) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:07:29.937047 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c832a51-d1e3-4afc-b257-a4f6c5442dd5 service neutron] REQUEST_METHOD: `GET` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:07:29.937324 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c832a51-d1e3-4afc-b257-a4f6c5442dd5 service neutron] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:07:29.937596 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c832a51-d1e3-4afc-b257-a4f6c5442dd5 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:07:29.939271 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.fernet_utils [None req-3c832a51-d1e3-4afc-b257-a4f6c5442dd5 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80378) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 10:07:30.004728 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG dbcounter [-] [80378] Writing DB stats keystone:SELECT=100 {{(pid=80378) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 10:07:30.057098 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c832a51-d1e3-4afc-b257-a4f6c5442dd5 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:07:30.058569 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c832a51-d1e3-4afc-b257-a4f6c5442dd5 service neutron] RBAC: Authorization granted {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:07:30.076622 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 650/1317] 10.222.0.82 () {60 vars in 1365 bytes} [Tue Aug 2 10:07:29 2022] GET /identity/v3/auth/tokens => generated 3243 bytes in 324 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 02 10:07:32.676335 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e92f931e-f9f3-4d89-a5f3-73d42f34df28 None None] REQUEST_METHOD: `GET` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:07:32.676923 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e92f931e-f9f3-4d89-a5f3-73d42f34df28 None None] SCRIPT_NAME: `/identity` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:07:32.676923 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e92f931e-f9f3-4d89-a5f3-73d42f34df28 None None] PATH_INFO: `/` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:07:32.678926 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: [pid: 80377|app: 0|req: 668/1318] 10.222.0.82 () {54 vars in 926 bytes} [Tue Aug 2 10:07:32 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 02 10:07:32.684275 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82bb91e2-d13c-4069-bc39-a81b1bd457a6 None None] REQUEST_METHOD: `POST` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:07:32.684743 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82bb91e2-d13c-4069-bc39-a81b1bd457a6 None None] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:07:32.684840 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82bb91e2-d13c-4069-bc39-a81b1bd457a6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:07:32.758210 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.auth.core [None req-82bb91e2-d13c-4069-bc39-a81b1bd457a6 None None] MFA Rules not processed for user `891005fa5ff74041b75747d16cc66798`. Rule list: `[]` (Enabled: `True`). {{(pid=80378) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 02 10:07:32.759442 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.fernet_utils [None req-82bb91e2-d13c-4069-bc39-a81b1bd457a6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80378) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 10:07:32.859718 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 651/1319] 10.222.0.82 () {58 vars in 1023 bytes} [Tue Aug 2 10:07:32 2022] POST /identity/v3/auth/tokens => generated 3238 bytes in 176 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 02 10:07:32.898777 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e92f931e-f9f3-4d89-a5f3-73d42f34df28 None None] Authenticating user token {{(pid=80377) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:07:32.900319 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.fernet_utils [None req-e92f931e-f9f3-4d89-a5f3-73d42f34df28 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80377) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 10:07:33.010702 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e92f931e-f9f3-4d89-a5f3-73d42f34df28 None None] Validating token access rules against request {{(pid=80377) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:07:33.102677 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9cea3f48-5ea2-4d2c-98f3-d7e2676efd75 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '33b87170f0634b67bce60750847b5cbe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f1b38ebb804c41c69772c998aae72c41', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80377) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:07:33.103300 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cea3f48-5ea2-4d2c-98f3-d7e2676efd75 service neutron] REQUEST_METHOD: `GET` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:07:33.103582 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cea3f48-5ea2-4d2c-98f3-d7e2676efd75 service neutron] SCRIPT_NAME: `/identity` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:07:33.103859 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cea3f48-5ea2-4d2c-98f3-d7e2676efd75 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:07:33.105507 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.fernet_utils [None req-9cea3f48-5ea2-4d2c-98f3-d7e2676efd75 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80377) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 10:07:33.113289 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG dbcounter [-] [80377] Writing DB stats keystone:SELECT=100 {{(pid=80377) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 10:07:33.224936 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9cea3f48-5ea2-4d2c-98f3-d7e2676efd75 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:07:33.227340 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9cea3f48-5ea2-4d2c-98f3-d7e2676efd75 service neutron] RBAC: Authorization granted {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:07:33.249169 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: [pid: 80377|app: 0|req: 669/1320] 10.222.0.82 () {60 vars in 1365 bytes} [Tue Aug 2 10:07:32 2022] GET /identity/v3/auth/tokens => generated 3238 bytes in 351 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 02 10:07:35.968177 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b86b152-ee88-42c9-94e4-63774f472c87 None None] REQUEST_METHOD: `GET` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:07:35.968715 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b86b152-ee88-42c9-94e4-63774f472c87 None None] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:07:35.968715 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b86b152-ee88-42c9-94e4-63774f472c87 None None] PATH_INFO: `/` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:07:35.970138 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 652/1321] 10.222.0.82 () {54 vars in 926 bytes} [Tue Aug 2 10:07:35 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 02 10:07:35.975691 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c12de0c8-0ab5-4378-8a77-65a435d30848 None None] REQUEST_METHOD: `POST` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:07:35.976222 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c12de0c8-0ab5-4378-8a77-65a435d30848 None None] SCRIPT_NAME: `/identity` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:07:35.976313 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c12de0c8-0ab5-4378-8a77-65a435d30848 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:07:36.042706 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.auth.core [None req-c12de0c8-0ab5-4378-8a77-65a435d30848 None None] MFA Rules not processed for user `891005fa5ff74041b75747d16cc66798`. Rule list: `[]` (Enabled: `True`). {{(pid=80377) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 02 10:07:36.043917 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.fernet_utils [None req-c12de0c8-0ab5-4378-8a77-65a435d30848 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80377) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 10:07:36.106139 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG dbcounter [-] [80377] Writing DB stats keystone:SELECT=100 {{(pid=80377) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 10:07:36.150297 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: [pid: 80377|app: 0|req: 670/1322] 10.222.0.82 () {58 vars in 1023 bytes} [Tue Aug 2 10:07:35 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 176 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 02 10:07:36.204190 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bab93d5-62af-44f8-a9bc-1244a1183284 None None] REQUEST_METHOD: `GET` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:07:36.204487 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bab93d5-62af-44f8-a9bc-1244a1183284 None None] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:07:36.204770 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bab93d5-62af-44f8-a9bc-1244a1183284 None None] PATH_INFO: `/` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:07:36.206178 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 653/1323] 10.222.0.82 () {54 vars in 965 bytes} [Tue Aug 2 10:07:36 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 02 10:07:36.212808 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0f14c94-7f05-4d98-a2a9-4234e9b9008a None None] REQUEST_METHOD: `POST` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:07:36.213100 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0f14c94-7f05-4d98-a2a9-4234e9b9008a None None] SCRIPT_NAME: `/identity` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:07:36.213377 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0f14c94-7f05-4d98-a2a9-4234e9b9008a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:07:36.295700 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.auth.core [None req-a0f14c94-7f05-4d98-a2a9-4234e9b9008a None None] MFA Rules not processed for user `33b87170f0634b67bce60750847b5cbe`. Rule list: `[]` (Enabled: `True`). {{(pid=80377) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 02 10:07:36.297185 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.fernet_utils [None req-a0f14c94-7f05-4d98-a2a9-4234e9b9008a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80377) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 10:07:36.375716 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: [pid: 80377|app: 0|req: 671/1324] 10.222.0.82 () {58 vars in 1062 bytes} [Tue Aug 2 10:07:36 2022] POST /identity/v3/auth/tokens => generated 3121 bytes in 164 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 02 10:07:36.384784 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bab93d5-62af-44f8-a9bc-1244a1183284 None None] Authenticating user token {{(pid=80378) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:07:36.386444 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.fernet_utils [None req-8bab93d5-62af-44f8-a9bc-1244a1183284 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80378) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 10:07:36.406790 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG dbcounter [-] [80378] Writing DB stats keystone:SELECT=100 {{(pid=80378) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 10:07:36.493566 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bab93d5-62af-44f8-a9bc-1244a1183284 None None] Validating token access rules against request {{(pid=80378) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:07:36.571373 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-755ec429-e18d-49f4-abe6-5826fde490be service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '33b87170f0634b67bce60750847b5cbe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f1b38ebb804c41c69772c998aae72c41', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80378) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:07:36.572004 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-755ec429-e18d-49f4-abe6-5826fde490be service neutron] REQUEST_METHOD: `GET` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:07:36.572291 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-755ec429-e18d-49f4-abe6-5826fde490be service neutron] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:07:36.572564 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-755ec429-e18d-49f4-abe6-5826fde490be service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:07:36.574290 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.fernet_utils [None req-755ec429-e18d-49f4-abe6-5826fde490be service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80378) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 10:07:36.649807 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG dbcounter [-] [80378] Writing DB stats keystone:SELECT=100 {{(pid=80378) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 10:07:36.691187 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-755ec429-e18d-49f4-abe6-5826fde490be service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:07:36.693264 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-755ec429-e18d-49f4-abe6-5826fde490be service neutron] RBAC: Authorization granted {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:07:36.713819 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 654/1325] 10.222.0.82 () {60 vars in 1365 bytes} [Tue Aug 2 10:07:36 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 330 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 02 10:07:39.027810 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-715966da-949d-4dcb-a135-0b0cdac21079 None None] REQUEST_METHOD: `GET` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:07:39.028442 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-715966da-949d-4dcb-a135-0b0cdac21079 None None] SCRIPT_NAME: `/identity` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:07:39.028442 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-715966da-949d-4dcb-a135-0b0cdac21079 None None] PATH_INFO: `/` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:07:39.029868 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: [pid: 80377|app: 0|req: 672/1326] 10.222.0.82 () {54 vars in 926 bytes} [Tue Aug 2 10:07:39 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 02 10:07:39.035987 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-483de8d3-1fb0-4c1c-9db8-3dbd94806a40 None None] REQUEST_METHOD: `POST` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:07:39.036733 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-483de8d3-1fb0-4c1c-9db8-3dbd94806a40 None None] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:07:39.036733 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-483de8d3-1fb0-4c1c-9db8-3dbd94806a40 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:07:39.106033 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.auth.core [None req-483de8d3-1fb0-4c1c-9db8-3dbd94806a40 None None] MFA Rules not processed for user `891005fa5ff74041b75747d16cc66798`. Rule list: `[]` (Enabled: `True`). {{(pid=80378) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 02 10:07:39.107178 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.fernet_utils [None req-483de8d3-1fb0-4c1c-9db8-3dbd94806a40 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80378) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 10:07:39.206734 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 655/1327] 10.222.0.82 () {58 vars in 1023 bytes} [Tue Aug 2 10:07:39 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 172 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 02 10:07:39.245511 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-715966da-949d-4dcb-a135-0b0cdac21079 None None] Authenticating user token {{(pid=80377) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:07:39.247282 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.fernet_utils [None req-715966da-949d-4dcb-a135-0b0cdac21079 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80377) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 10:07:39.311464 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG dbcounter [-] [80377] Writing DB stats keystone:SELECT=100 {{(pid=80377) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 10:07:39.355258 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-715966da-949d-4dcb-a135-0b0cdac21079 None None] Validating token access rules against request {{(pid=80377) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:07:39.430010 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ca68c92-823d-4bc9-ad72-e4c4e249eb1e service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '33b87170f0634b67bce60750847b5cbe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f1b38ebb804c41c69772c998aae72c41', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80377) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:07:39.430553 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ca68c92-823d-4bc9-ad72-e4c4e249eb1e service neutron] REQUEST_METHOD: `GET` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:07:39.430828 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ca68c92-823d-4bc9-ad72-e4c4e249eb1e service neutron] SCRIPT_NAME: `/identity` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:07:39.431096 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ca68c92-823d-4bc9-ad72-e4c4e249eb1e service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:07:39.432719 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.fernet_utils [None req-2ca68c92-823d-4bc9-ad72-e4c4e249eb1e service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80377) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 10:07:39.547659 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG dbcounter [-] [80377] Writing DB stats keystone:SELECT=100 {{(pid=80377) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 10:07:39.550066 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ca68c92-823d-4bc9-ad72-e4c4e249eb1e service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:07:39.551806 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ca68c92-823d-4bc9-ad72-e4c4e249eb1e service neutron] RBAC: Authorization granted {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:07:39.571211 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: [pid: 80377|app: 0|req: 673/1328] 10.222.0.82 () {60 vars in 1365 bytes} [Tue Aug 2 10:07:39 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 326 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 02 10:07:41.663092 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffe1269f-0d7d-4e6d-862d-7bc19688698c None None] REQUEST_METHOD: `GET` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:07:41.663626 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffe1269f-0d7d-4e6d-862d-7bc19688698c None None] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:07:41.663626 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffe1269f-0d7d-4e6d-862d-7bc19688698c None None] PATH_INFO: `/` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:07:41.664960 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 656/1329] 10.222.0.82 () {54 vars in 926 bytes} [Tue Aug 2 10:07:41 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 02 10:07:41.670508 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f4f1ae7-abf4-4481-90e9-2e5bef9586f4 None None] REQUEST_METHOD: `POST` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:07:41.671192 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f4f1ae7-abf4-4481-90e9-2e5bef9586f4 None None] SCRIPT_NAME: `/identity` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:07:41.671339 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f4f1ae7-abf4-4481-90e9-2e5bef9586f4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:07:41.739660 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.auth.core [None req-2f4f1ae7-abf4-4481-90e9-2e5bef9586f4 None None] MFA Rules not processed for user `71a4a6ffabe74df39f0a7ca7ea7b33b8`. Rule list: `[]` (Enabled: `True`). {{(pid=80377) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 02 10:07:41.741029 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.fernet_utils [None req-2f4f1ae7-abf4-4481-90e9-2e5bef9586f4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80377) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 10:07:41.841225 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: [pid: 80377|app: 0|req: 674/1330] 10.222.0.82 () {58 vars in 1023 bytes} [Tue Aug 2 10:07:41 2022] POST /identity/v3/auth/tokens => generated 3243 bytes in 172 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 02 10:07:41.886058 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffe1269f-0d7d-4e6d-862d-7bc19688698c None None] Authenticating user token {{(pid=80378) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:07:41.887592 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.fernet_utils [None req-ffe1269f-0d7d-4e6d-862d-7bc19688698c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80378) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 10:07:41.928110 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG dbcounter [-] [80378] Writing DB stats keystone:SELECT=100 {{(pid=80378) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 10:07:42.006796 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffe1269f-0d7d-4e6d-862d-7bc19688698c None None] Validating token access rules against request {{(pid=80378) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:07:42.080724 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc1efc12-3f96-4adc-b482-2035d4ed3440 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '33b87170f0634b67bce60750847b5cbe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f1b38ebb804c41c69772c998aae72c41', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80378) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:07:42.081342 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc1efc12-3f96-4adc-b482-2035d4ed3440 service neutron] REQUEST_METHOD: `GET` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:07:42.081624 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc1efc12-3f96-4adc-b482-2035d4ed3440 service neutron] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:07:42.082068 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc1efc12-3f96-4adc-b482-2035d4ed3440 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:07:42.083703 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.fernet_utils [None req-fc1efc12-3f96-4adc-b482-2035d4ed3440 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80378) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 10:07:42.163212 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG dbcounter [-] [80378] Writing DB stats keystone:SELECT=100 {{(pid=80378) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 10:07:42.196656 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc1efc12-3f96-4adc-b482-2035d4ed3440 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:07:42.198066 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc1efc12-3f96-4adc-b482-2035d4ed3440 service neutron] RBAC: Authorization granted {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:07:42.216297 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 657/1331] 10.222.0.82 () {60 vars in 1365 bytes} [Tue Aug 2 10:07:41 2022] GET /identity/v3/auth/tokens => generated 3243 bytes in 331 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 02 10:07:44.737156 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-484abbd6-09ff-49e1-9e06-cbbcca6ff0ec None None] REQUEST_METHOD: `GET` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:07:44.737821 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-484abbd6-09ff-49e1-9e06-cbbcca6ff0ec None None] SCRIPT_NAME: `/identity` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:07:44.737821 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-484abbd6-09ff-49e1-9e06-cbbcca6ff0ec None None] PATH_INFO: `/` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:07:44.739745 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: [pid: 80377|app: 0|req: 675/1332] 10.222.0.82 () {54 vars in 926 bytes} [Tue Aug 2 10:07:44 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 02 10:07:44.745151 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-366a887d-c25b-4fd4-a1d1-558bb5c47c19 None None] REQUEST_METHOD: `POST` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:07:44.745574 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-366a887d-c25b-4fd4-a1d1-558bb5c47c19 None None] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:07:44.745728 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-366a887d-c25b-4fd4-a1d1-558bb5c47c19 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:07:44.817823 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.auth.core [None req-366a887d-c25b-4fd4-a1d1-558bb5c47c19 None None] MFA Rules not processed for user `891005fa5ff74041b75747d16cc66798`. Rule list: `[]` (Enabled: `True`). {{(pid=80378) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 02 10:07:44.819278 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.fernet_utils [None req-366a887d-c25b-4fd4-a1d1-558bb5c47c19 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80378) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 10:07:44.917227 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 658/1333] 10.222.0.82 () {58 vars in 1023 bytes} [Tue Aug 2 10:07:44 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 173 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 02 10:07:44.954106 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-484abbd6-09ff-49e1-9e06-cbbcca6ff0ec None None] Authenticating user token {{(pid=80377) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:07:44.955661 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.fernet_utils [None req-484abbd6-09ff-49e1-9e06-cbbcca6ff0ec None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80377) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 10:07:45.032733 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG dbcounter [-] [80377] Writing DB stats keystone:SELECT=100 {{(pid=80377) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 10:07:45.072765 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-484abbd6-09ff-49e1-9e06-cbbcca6ff0ec None None] Validating token access rules against request {{(pid=80377) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:07:45.144589 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0296b7f5-23f3-401d-9114-402e51404283 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '33b87170f0634b67bce60750847b5cbe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f1b38ebb804c41c69772c998aae72c41', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80377) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:07:45.145736 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0296b7f5-23f3-401d-9114-402e51404283 service neutron] REQUEST_METHOD: `GET` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:07:45.146051 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0296b7f5-23f3-401d-9114-402e51404283 service neutron] SCRIPT_NAME: `/identity` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:07:45.146328 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0296b7f5-23f3-401d-9114-402e51404283 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:07:45.148078 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.fernet_utils [None req-0296b7f5-23f3-401d-9114-402e51404283 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80377) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 10:07:45.258476 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0296b7f5-23f3-401d-9114-402e51404283 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:07:45.259825 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0296b7f5-23f3-401d-9114-402e51404283 service neutron] RBAC: Authorization granted {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:07:45.269494 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG dbcounter [-] [80377] Writing DB stats keystone:SELECT=100 {{(pid=80377) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 10:07:45.279669 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: [pid: 80377|app: 0|req: 676/1334] 10.222.0.82 () {60 vars in 1365 bytes} [Tue Aug 2 10:07:44 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 326 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 02 10:07:47.341039 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24875079-abf4-4d60-8f23-3370aa0252c0 None None] REQUEST_METHOD: `GET` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:07:47.341602 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24875079-abf4-4d60-8f23-3370aa0252c0 None None] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:07:47.341602 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24875079-abf4-4d60-8f23-3370aa0252c0 None None] PATH_INFO: `/` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:07:47.342989 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 659/1335] 10.222.0.82 () {54 vars in 926 bytes} [Tue Aug 2 10:07:47 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 02 10:07:47.347982 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1e35bf2-7f3c-452f-91f5-226826dfc988 None None] REQUEST_METHOD: `POST` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:07:47.348964 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1e35bf2-7f3c-452f-91f5-226826dfc988 None None] SCRIPT_NAME: `/identity` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:07:47.349218 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1e35bf2-7f3c-452f-91f5-226826dfc988 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:07:47.414267 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.auth.core [None req-f1e35bf2-7f3c-452f-91f5-226826dfc988 None None] MFA Rules not processed for user `891005fa5ff74041b75747d16cc66798`. Rule list: `[]` (Enabled: `True`). {{(pid=80377) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 02 10:07:47.415514 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.fernet_utils [None req-f1e35bf2-7f3c-452f-91f5-226826dfc988 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80377) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 10:07:47.514012 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: [pid: 80377|app: 0|req: 677/1336] 10.222.0.82 () {58 vars in 1023 bytes} [Tue Aug 2 10:07:47 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 167 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 02 10:07:47.555626 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24875079-abf4-4d60-8f23-3370aa0252c0 None None] Authenticating user token {{(pid=80378) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:07:47.557326 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.fernet_utils [None req-24875079-abf4-4d60-8f23-3370aa0252c0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80378) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 10:07:47.598603 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG dbcounter [-] [80378] Writing DB stats keystone:SELECT=100 {{(pid=80378) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 10:07:47.663697 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24875079-abf4-4d60-8f23-3370aa0252c0 None None] Validating token access rules against request {{(pid=80378) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:07:47.736699 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cad3519f-dd7b-4230-8e2d-fa6ffd8e7dde service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '33b87170f0634b67bce60750847b5cbe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f1b38ebb804c41c69772c998aae72c41', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80378) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:07:47.737316 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cad3519f-dd7b-4230-8e2d-fa6ffd8e7dde service neutron] REQUEST_METHOD: `GET` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:07:47.737600 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cad3519f-dd7b-4230-8e2d-fa6ffd8e7dde service neutron] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:07:47.737902 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cad3519f-dd7b-4230-8e2d-fa6ffd8e7dde service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:07:47.740075 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.fernet_utils [None req-cad3519f-dd7b-4230-8e2d-fa6ffd8e7dde service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80378) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 10:07:47.826039 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG dbcounter [-] [80378] Writing DB stats keystone:SELECT=100 {{(pid=80378) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 10:07:47.854362 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cad3519f-dd7b-4230-8e2d-fa6ffd8e7dde service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:07:47.855814 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cad3519f-dd7b-4230-8e2d-fa6ffd8e7dde service neutron] RBAC: Authorization granted {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:07:47.875170 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 660/1337] 10.222.0.82 () {60 vars in 1365 bytes} [Tue Aug 2 10:07:47 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 320 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 02 10:07:50.233768 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37f96aad-f0f7-47ea-8316-7e25c4608d47 None None] REQUEST_METHOD: `GET` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:07:50.234324 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37f96aad-f0f7-47ea-8316-7e25c4608d47 None None] SCRIPT_NAME: `/identity` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:07:50.234422 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37f96aad-f0f7-47ea-8316-7e25c4608d47 None None] PATH_INFO: `/` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:07:50.235733 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: [pid: 80377|app: 0|req: 678/1338] 10.222.0.82 () {54 vars in 926 bytes} [Tue Aug 2 10:07:50 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 02 10:07:50.241574 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30470816-118a-477b-8f09-48d836afb1e3 None None] REQUEST_METHOD: `POST` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:07:50.242021 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30470816-118a-477b-8f09-48d836afb1e3 None None] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:07:50.242166 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30470816-118a-477b-8f09-48d836afb1e3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:07:50.308494 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.auth.core [None req-30470816-118a-477b-8f09-48d836afb1e3 None None] MFA Rules not processed for user `891005fa5ff74041b75747d16cc66798`. Rule list: `[]` (Enabled: `True`). {{(pid=80378) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 02 10:07:50.310145 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.fernet_utils [None req-30470816-118a-477b-8f09-48d836afb1e3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80378) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 10:07:50.411784 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 661/1339] 10.222.0.82 () {58 vars in 1023 bytes} [Tue Aug 2 10:07:50 2022] POST /identity/v3/auth/tokens => generated 3238 bytes in 171 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 02 10:07:50.448683 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37f96aad-f0f7-47ea-8316-7e25c4608d47 None None] Authenticating user token {{(pid=80377) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:07:50.450252 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.fernet_utils [None req-37f96aad-f0f7-47ea-8316-7e25c4608d47 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80377) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 10:07:50.530537 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG dbcounter [-] [80377] Writing DB stats keystone:SELECT=100 {{(pid=80377) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 10:07:50.560621 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37f96aad-f0f7-47ea-8316-7e25c4608d47 None None] Validating token access rules against request {{(pid=80377) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:07:50.637406 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-956e7a32-1612-456e-8c16-55b6a09485de service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '33b87170f0634b67bce60750847b5cbe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f1b38ebb804c41c69772c998aae72c41', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80377) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:07:50.638119 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-956e7a32-1612-456e-8c16-55b6a09485de service neutron] REQUEST_METHOD: `GET` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:07:50.638456 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-956e7a32-1612-456e-8c16-55b6a09485de service neutron] SCRIPT_NAME: `/identity` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:07:50.638779 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-956e7a32-1612-456e-8c16-55b6a09485de service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:07:50.640466 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.fernet_utils [None req-956e7a32-1612-456e-8c16-55b6a09485de service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80377) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 10:07:50.750987 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-956e7a32-1612-456e-8c16-55b6a09485de service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:07:50.752438 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-956e7a32-1612-456e-8c16-55b6a09485de service neutron] RBAC: Authorization granted {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:07:50.771307 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: [pid: 80377|app: 0|req: 679/1340] 10.222.0.82 () {60 vars in 1365 bytes} [Tue Aug 2 10:07:50 2022] GET /identity/v3/auth/tokens => generated 3238 bytes in 324 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 02 10:07:54.947196 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fabbb03-a01d-43b8-ba97-044847e75c73 None None] REQUEST_METHOD: `GET` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:07:54.947831 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fabbb03-a01d-43b8-ba97-044847e75c73 None None] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:07:54.947831 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fabbb03-a01d-43b8-ba97-044847e75c73 None None] PATH_INFO: `/` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:07:54.949121 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 662/1341] 10.222.0.82 () {54 vars in 926 bytes} [Tue Aug 2 10:07:54 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 02 10:07:54.954565 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e69519d2-8113-43c8-b941-f4d886ffcd14 None None] REQUEST_METHOD: `POST` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:07:54.955019 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e69519d2-8113-43c8-b941-f4d886ffcd14 None None] SCRIPT_NAME: `/identity` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:07:54.955157 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e69519d2-8113-43c8-b941-f4d886ffcd14 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:07:54.960871 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG dbcounter [-] [80377] Writing DB stats keystone:SELECT=100 {{(pid=80377) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 10:07:55.030743 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.auth.core [None req-e69519d2-8113-43c8-b941-f4d886ffcd14 None None] MFA Rules not processed for user `891005fa5ff74041b75747d16cc66798`. Rule list: `[]` (Enabled: `True`). {{(pid=80377) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 02 10:07:55.032042 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.fernet_utils [None req-e69519d2-8113-43c8-b941-f4d886ffcd14 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80377) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 10:07:55.139991 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: [pid: 80377|app: 0|req: 680/1342] 10.222.0.82 () {58 vars in 1023 bytes} [Tue Aug 2 10:07:54 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 186 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 02 10:07:55.179606 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fabbb03-a01d-43b8-ba97-044847e75c73 None None] Authenticating user token {{(pid=80378) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:07:55.181191 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.fernet_utils [None req-3fabbb03-a01d-43b8-ba97-044847e75c73 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80378) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 10:07:55.249914 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG dbcounter [-] [80378] Writing DB stats keystone:SELECT=100 {{(pid=80378) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 10:07:55.308868 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fabbb03-a01d-43b8-ba97-044847e75c73 None None] Validating token access rules against request {{(pid=80378) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:07:55.386039 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23d3b963-2859-43a6-a185-4950672089ce service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '33b87170f0634b67bce60750847b5cbe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f1b38ebb804c41c69772c998aae72c41', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80378) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:07:55.386659 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23d3b963-2859-43a6-a185-4950672089ce service neutron] REQUEST_METHOD: `GET` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:07:55.386937 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23d3b963-2859-43a6-a185-4950672089ce service neutron] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:07:55.387203 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23d3b963-2859-43a6-a185-4950672089ce service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:07:55.388826 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.fernet_utils [None req-23d3b963-2859-43a6-a185-4950672089ce service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80378) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 10:07:55.487697 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG dbcounter [-] [80378] Writing DB stats keystone:SELECT=100 {{(pid=80378) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 10:07:55.505354 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23d3b963-2859-43a6-a185-4950672089ce service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:07:55.506847 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23d3b963-2859-43a6-a185-4950672089ce service neutron] RBAC: Authorization granted {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:07:55.525794 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 663/1343] 10.222.0.82 () {60 vars in 1365 bytes} [Tue Aug 2 10:07:55 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 347 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 02 10:07:58.299552 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f51c574-1893-4dc2-8b16-677e61410a61 None None] REQUEST_METHOD: `GET` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:07:58.300277 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f51c574-1893-4dc2-8b16-677e61410a61 None None] SCRIPT_NAME: `/identity` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:07:58.300277 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f51c574-1893-4dc2-8b16-677e61410a61 None None] PATH_INFO: `/` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:07:58.301738 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: [pid: 80377|app: 0|req: 681/1344] 10.222.0.82 () {54 vars in 926 bytes} [Tue Aug 2 10:07:58 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 02 10:07:58.308189 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a63d0b17-1681-4414-a709-d4b9f2d9f3dc None None] REQUEST_METHOD: `POST` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:07:58.308782 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a63d0b17-1681-4414-a709-d4b9f2d9f3dc None None] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:07:58.308782 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a63d0b17-1681-4414-a709-d4b9f2d9f3dc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:07:58.408178 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.auth.core [None req-a63d0b17-1681-4414-a709-d4b9f2d9f3dc None None] MFA Rules not processed for user `891005fa5ff74041b75747d16cc66798`. Rule list: `[]` (Enabled: `True`). {{(pid=80378) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 02 10:07:58.409434 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.fernet_utils [None req-a63d0b17-1681-4414-a709-d4b9f2d9f3dc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80378) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 10:07:58.537979 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 664/1345] 10.222.0.82 () {58 vars in 1023 bytes} [Tue Aug 2 10:07:58 2022] POST /identity/v3/auth/tokens => generated 3238 bytes in 231 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 02 10:07:58.582585 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f51c574-1893-4dc2-8b16-677e61410a61 None None] Authenticating user token {{(pid=80377) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:07:58.585014 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.fernet_utils [None req-2f51c574-1893-4dc2-8b16-677e61410a61 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80377) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 10:07:58.680754 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG dbcounter [-] [80377] Writing DB stats keystone:SELECT=100 {{(pid=80377) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 10:07:58.698501 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f51c574-1893-4dc2-8b16-677e61410a61 None None] Validating token access rules against request {{(pid=80377) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:07:58.785923 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e4a3cf9-0f25-4b70-a6ae-216042668993 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '33b87170f0634b67bce60750847b5cbe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f1b38ebb804c41c69772c998aae72c41', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80377) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:07:58.786554 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e4a3cf9-0f25-4b70-a6ae-216042668993 service neutron] REQUEST_METHOD: `GET` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:07:58.786831 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e4a3cf9-0f25-4b70-a6ae-216042668993 service neutron] SCRIPT_NAME: `/identity` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:07:58.787207 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e4a3cf9-0f25-4b70-a6ae-216042668993 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:07:58.788917 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.fernet_utils [None req-6e4a3cf9-0f25-4b70-a6ae-216042668993 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80377) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 10:07:58.913662 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e4a3cf9-0f25-4b70-a6ae-216042668993 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:07:58.915150 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e4a3cf9-0f25-4b70-a6ae-216042668993 service neutron] RBAC: Authorization granted {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:07:58.934213 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: [pid: 80377|app: 0|req: 682/1346] 10.222.0.82 () {60 vars in 1365 bytes} [Tue Aug 2 10:07:58 2022] GET /identity/v3/auth/tokens => generated 3238 bytes in 353 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 02 10:08:03.116606 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22424530-64b3-4a25-9391-434f9931eee2 None None] REQUEST_METHOD: `GET` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:08:03.117221 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22424530-64b3-4a25-9391-434f9931eee2 None None] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:08:03.117221 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22424530-64b3-4a25-9391-434f9931eee2 None None] PATH_INFO: `/` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:08:03.118576 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 665/1347] 10.222.0.82 () {54 vars in 926 bytes} [Tue Aug 2 10:08:03 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 02 10:08:03.123895 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ce1bd55-c1d5-4c19-a35a-af90b49e1a66 None None] REQUEST_METHOD: `POST` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:08:03.124356 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ce1bd55-c1d5-4c19-a35a-af90b49e1a66 None None] SCRIPT_NAME: `/identity` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:08:03.124649 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ce1bd55-c1d5-4c19-a35a-af90b49e1a66 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:08:03.140102 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG dbcounter [-] [80377] Writing DB stats keystone:SELECT=100 {{(pid=80377) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 10:08:03.197537 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.auth.core [None req-8ce1bd55-c1d5-4c19-a35a-af90b49e1a66 None None] MFA Rules not processed for user `891005fa5ff74041b75747d16cc66798`. Rule list: `[]` (Enabled: `True`). {{(pid=80377) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 02 10:08:03.198805 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.fernet_utils [None req-8ce1bd55-c1d5-4c19-a35a-af90b49e1a66 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80377) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 10:08:03.310888 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: [pid: 80377|app: 0|req: 683/1348] 10.222.0.82 () {58 vars in 1023 bytes} [Tue Aug 2 10:08:03 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 188 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 02 10:08:03.353933 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22424530-64b3-4a25-9391-434f9931eee2 None None] Authenticating user token {{(pid=80378) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:08:03.355482 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.fernet_utils [None req-22424530-64b3-4a25-9391-434f9931eee2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80378) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 10:08:03.438929 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG dbcounter [-] [80378] Writing DB stats keystone:SELECT=100 {{(pid=80378) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 10:08:03.491357 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22424530-64b3-4a25-9391-434f9931eee2 None None] Validating token access rules against request {{(pid=80378) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:08:03.579096 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d0e7a9c-2dab-453e-822e-c4417fe20a6d service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '33b87170f0634b67bce60750847b5cbe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f1b38ebb804c41c69772c998aae72c41', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80378) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:08:03.579730 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d0e7a9c-2dab-453e-822e-c4417fe20a6d service neutron] REQUEST_METHOD: `GET` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:08:03.580012 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d0e7a9c-2dab-453e-822e-c4417fe20a6d service neutron] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:08:03.580293 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d0e7a9c-2dab-453e-822e-c4417fe20a6d service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:08:03.582567 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.fernet_utils [None req-5d0e7a9c-2dab-453e-822e-c4417fe20a6d service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80378) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 10:08:03.706170 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG dbcounter [-] [80378] Writing DB stats keystone:SELECT=100 {{(pid=80378) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 10:08:03.711145 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d0e7a9c-2dab-453e-822e-c4417fe20a6d service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:08:03.712867 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d0e7a9c-2dab-453e-822e-c4417fe20a6d service neutron] RBAC: Authorization granted {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:08:03.733926 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 666/1349] 10.222.0.82 () {60 vars in 1365 bytes} [Tue Aug 2 10:08:03 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 381 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 02 10:08:08.806761 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c20f44b-05de-4acf-a8ee-4a871dd74a7a None None] REQUEST_METHOD: `GET` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:08:08.807255 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c20f44b-05de-4acf-a8ee-4a871dd74a7a None None] SCRIPT_NAME: `/identity` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:08:08.807346 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c20f44b-05de-4acf-a8ee-4a871dd74a7a None None] PATH_INFO: `/` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:08:08.808671 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: [pid: 80377|app: 0|req: 684/1350] 10.222.0.82 () {54 vars in 921 bytes} [Tue Aug 2 10:08:08 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 02 10:08:08.817331 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b661bd8-c6d9-45ea-a45f-8fb22f3cd8d6 None None] REQUEST_METHOD: `POST` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:08:08.818094 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b661bd8-c6d9-45ea-a45f-8fb22f3cd8d6 None None] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:08:08.818094 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b661bd8-c6d9-45ea-a45f-8fb22f3cd8d6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:08:08.922645 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.auth.core [None req-7b661bd8-c6d9-45ea-a45f-8fb22f3cd8d6 None None] MFA Rules not processed for user `ae6e910e50d74f29bfb89bf18d95a908`. Rule list: `[]` (Enabled: `True`). {{(pid=80378) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 02 10:08:08.924024 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.fernet_utils [None req-7b661bd8-c6d9-45ea-a45f-8fb22f3cd8d6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80378) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 10:08:09.160557 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 667/1351] 10.222.0.82 () {58 vars in 1018 bytes} [Tue Aug 2 10:08:08 2022] POST /identity/v3/auth/tokens => generated 3308 bytes in 347 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 02 10:08:13.308434 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG dbcounter [-] [80377] Writing DB stats keystone:SELECT=59 {{(pid=80377) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 10:08:13.605935 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bc0d0f2-3a05-43d3-9915-fce844876262 None None] REQUEST_METHOD: `GET` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:08:13.606545 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bc0d0f2-3a05-43d3-9915-fce844876262 None None] SCRIPT_NAME: `/identity` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:08:13.606845 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bc0d0f2-3a05-43d3-9915-fce844876262 None None] PATH_INFO: `/` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:08:13.610940 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: [pid: 80377|app: 0|req: 685/1352] 10.222.0.82 () {54 vars in 921 bytes} [Tue Aug 2 10:08:13 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 02 10:08:13.628626 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c1ea91b-9660-40e6-a430-57d22e4dc682 None None] REQUEST_METHOD: `POST` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:08:13.629290 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c1ea91b-9660-40e6-a430-57d22e4dc682 None None] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:08:13.629577 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c1ea91b-9660-40e6-a430-57d22e4dc682 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:08:13.731909 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG dbcounter [-] [80378] Writing DB stats keystone:SELECT=100 {{(pid=80378) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 10:08:13.767618 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.auth.core [None req-5c1ea91b-9660-40e6-a430-57d22e4dc682 None None] MFA Rules not processed for user `ae6e910e50d74f29bfb89bf18d95a908`. Rule list: `[]` (Enabled: `True`). {{(pid=80378) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 02 10:08:13.770223 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.fernet_utils [None req-5c1ea91b-9660-40e6-a430-57d22e4dc682 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80378) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 10:08:13.946578 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 668/1353] 10.222.0.82 () {58 vars in 1018 bytes} [Tue Aug 2 10:08:13 2022] POST /identity/v3/auth/tokens => generated 3308 bytes in 321 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 02 10:08:15.272303 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77471273-38ee-4c51-901f-dced5b93f08f None None] REQUEST_METHOD: `GET` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:08:15.272836 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77471273-38ee-4c51-901f-dced5b93f08f None None] SCRIPT_NAME: `/identity` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:08:15.272836 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77471273-38ee-4c51-901f-dced5b93f08f None None] PATH_INFO: `/` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:08:15.274249 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: [pid: 80377|app: 0|req: 686/1354] 10.222.0.82 () {54 vars in 921 bytes} [Tue Aug 2 10:08:15 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 02 10:08:15.279355 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19af6375-60c1-440c-80b0-8670f79527f6 None None] REQUEST_METHOD: `POST` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:08:15.280087 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19af6375-60c1-440c-80b0-8670f79527f6 None None] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:08:15.280087 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19af6375-60c1-440c-80b0-8670f79527f6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:08:15.363701 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.auth.core [None req-19af6375-60c1-440c-80b0-8670f79527f6 None None] MFA Rules not processed for user `ae6e910e50d74f29bfb89bf18d95a908`. Rule list: `[]` (Enabled: `True`). {{(pid=80378) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 02 10:08:15.364930 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.fernet_utils [None req-19af6375-60c1-440c-80b0-8670f79527f6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80378) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 10:08:15.410263 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG dbcounter [-] [80378] Writing DB stats keystone:SELECT=100 {{(pid=80378) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 10:08:15.495573 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 669/1355] 10.222.0.82 () {58 vars in 1018 bytes} [Tue Aug 2 10:08:15 2022] POST /identity/v3/auth/tokens => generated 3308 bytes in 217 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 02 10:08:19.195566 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f55b1872-f279-4b54-8306-bf0db6fa2c59 None None] REQUEST_METHOD: `GET` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:08:19.196553 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f55b1872-f279-4b54-8306-bf0db6fa2c59 None None] SCRIPT_NAME: `/identity` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:08:19.197033 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f55b1872-f279-4b54-8306-bf0db6fa2c59 None None] PATH_INFO: `/` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:08:19.200503 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: [pid: 80377|app: 0|req: 687/1356] 10.222.0.82 () {54 vars in 926 bytes} [Tue Aug 2 10:08:19 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 02 10:08:19.206379 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00bc51c7-514b-4f56-9534-bad381dace62 None None] REQUEST_METHOD: `POST` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:08:19.206849 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00bc51c7-514b-4f56-9534-bad381dace62 None None] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:08:19.206849 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00bc51c7-514b-4f56-9534-bad381dace62 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:08:19.292705 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.auth.core [None req-00bc51c7-514b-4f56-9534-bad381dace62 None None] MFA Rules not processed for user `891005fa5ff74041b75747d16cc66798`. Rule list: `[]` (Enabled: `True`). {{(pid=80378) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 02 10:08:19.294003 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.fernet_utils [None req-00bc51c7-514b-4f56-9534-bad381dace62 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80378) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 10:08:19.422024 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 670/1357] 10.222.0.82 () {58 vars in 1023 bytes} [Tue Aug 2 10:08:19 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 217 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 02 10:08:19.471344 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-892b326f-269e-4e25-bc05-b7542b148c75 None None] REQUEST_METHOD: `GET` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:08:19.471860 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-892b326f-269e-4e25-bc05-b7542b148c75 None None] SCRIPT_NAME: `/identity` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:08:19.472144 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-892b326f-269e-4e25-bc05-b7542b148c75 None None] PATH_INFO: `/` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:08:19.473737 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: [pid: 80377|app: 0|req: 688/1358] 10.222.0.82 () {54 vars in 962 bytes} [Tue Aug 2 10:08:19 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 02 10:08:19.498757 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dddd8b0c-2c75-4d0b-9306-77bbd656dde9 None None] REQUEST_METHOD: `POST` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:08:19.499388 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dddd8b0c-2c75-4d0b-9306-77bbd656dde9 None None] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:08:19.499881 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dddd8b0c-2c75-4d0b-9306-77bbd656dde9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:08:19.519199 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG dbcounter [-] [80378] Writing DB stats keystone:SELECT=100 {{(pid=80378) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 10:08:19.593692 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.auth.core [None req-dddd8b0c-2c75-4d0b-9306-77bbd656dde9 None None] MFA Rules not processed for user `88af518196674fe593ad88d30111fb7d`. Rule list: `[]` (Enabled: `True`). {{(pid=80378) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 02 10:08:19.595046 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.fernet_utils [None req-dddd8b0c-2c75-4d0b-9306-77bbd656dde9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80378) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 10:08:19.743898 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 671/1359] 10.222.0.82 () {58 vars in 1059 bytes} [Tue Aug 2 10:08:19 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 247 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 02 10:08:19.752130 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-892b326f-269e-4e25-bc05-b7542b148c75 None None] Authenticating user token {{(pid=80377) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:08:19.754278 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.fernet_utils [None req-892b326f-269e-4e25-bc05-b7542b148c75 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80377) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 10:08:19.923468 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-892b326f-269e-4e25-bc05-b7542b148c75 None None] Validating token access rules against request {{(pid=80377) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:08:20.017738 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG dbcounter [-] [80377] Writing DB stats keystone:SELECT=100 {{(pid=80377) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 10:08:20.071427 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90b988a8-90ba-4f14-a32f-50342f8c5616 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88af518196674fe593ad88d30111fb7d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f1b38ebb804c41c69772c998aae72c41', 'project_domain_id': 'default', 'roles': ['admin', 'ResellerAdmin', 'reader', 'service', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80377) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:08:20.072053 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90b988a8-90ba-4f14-a32f-50342f8c5616 service nova] REQUEST_METHOD: `GET` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:08:20.072347 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90b988a8-90ba-4f14-a32f-50342f8c5616 service nova] SCRIPT_NAME: `/identity` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:08:20.072609 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90b988a8-90ba-4f14-a32f-50342f8c5616 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:08:20.074366 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.fernet_utils [None req-90b988a8-90ba-4f14-a32f-50342f8c5616 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80377) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 10:08:20.184939 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90b988a8-90ba-4f14-a32f-50342f8c5616 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:08:20.186614 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90b988a8-90ba-4f14-a32f-50342f8c5616 service nova] RBAC: Authorization granted {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:08:20.207035 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: [pid: 80377|app: 0|req: 689/1360] 10.222.0.82 () {60 vars in 1365 bytes} [Tue Aug 2 10:08:19 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 455 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 02 10:08:22.513373 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3818594-f0fe-4bf3-b6d4-eea2a03b775f None None] REQUEST_METHOD: `GET` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:08:22.514148 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3818594-f0fe-4bf3-b6d4-eea2a03b775f None None] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:08:22.514148 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3818594-f0fe-4bf3-b6d4-eea2a03b775f None None] PATH_INFO: `/` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:08:22.517322 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 672/1361] 10.222.0.82 () {54 vars in 926 bytes} [Tue Aug 2 10:08:22 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 02 10:08:22.522140 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-222784b8-81f9-464e-961a-b03b40bda97c None None] REQUEST_METHOD: `POST` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:08:22.522681 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-222784b8-81f9-464e-961a-b03b40bda97c None None] SCRIPT_NAME: `/identity` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:08:22.523155 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-222784b8-81f9-464e-961a-b03b40bda97c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:08:22.597921 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.auth.core [None req-222784b8-81f9-464e-961a-b03b40bda97c None None] MFA Rules not processed for user `891005fa5ff74041b75747d16cc66798`. Rule list: `[]` (Enabled: `True`). {{(pid=80377) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 02 10:08:22.599123 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.fernet_utils [None req-222784b8-81f9-464e-961a-b03b40bda97c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80377) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 10:08:22.641074 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG dbcounter [-] [80377] Writing DB stats keystone:SELECT=100 {{(pid=80377) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 10:08:22.717824 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: [pid: 80377|app: 0|req: 690/1362] 10.222.0.82 () {58 vars in 1023 bytes} [Tue Aug 2 10:08:22 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 196 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 02 10:08:22.771314 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3818594-f0fe-4bf3-b6d4-eea2a03b775f None None] Authenticating user token {{(pid=80378) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:08:22.772936 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.fernet_utils [None req-f3818594-f0fe-4bf3-b6d4-eea2a03b775f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80378) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 10:08:22.857872 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG dbcounter [-] [80378] Writing DB stats keystone:SELECT=100 {{(pid=80378) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 10:08:22.953083 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3818594-f0fe-4bf3-b6d4-eea2a03b775f None None] Validating token access rules against request {{(pid=80378) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:08:23.118918 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1e9c09f-272d-490d-bf6c-e276d009d5da service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88af518196674fe593ad88d30111fb7d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f1b38ebb804c41c69772c998aae72c41', 'project_domain_id': 'default', 'roles': ['admin', 'ResellerAdmin', 'reader', 'service', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80378) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:08:23.119719 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1e9c09f-272d-490d-bf6c-e276d009d5da service nova] REQUEST_METHOD: `GET` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:08:23.120095 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1e9c09f-272d-490d-bf6c-e276d009d5da service nova] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:08:23.120435 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1e9c09f-272d-490d-bf6c-e276d009d5da service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:08:23.122278 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.fernet_utils [None req-c1e9c09f-272d-490d-bf6c-e276d009d5da service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80378) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 10:08:23.152260 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG dbcounter [-] [80378] Writing DB stats keystone:SELECT=100 {{(pid=80378) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 10:08:23.234402 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1e9c09f-272d-490d-bf6c-e276d009d5da service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:08:23.236209 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1e9c09f-272d-490d-bf6c-e276d009d5da service nova] RBAC: Authorization granted {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:08:23.257579 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 673/1363] 10.222.0.82 () {60 vars in 1365 bytes} [Tue Aug 2 10:08:22 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 487 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 02 10:08:25.713746 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25dcfa28-3319-4c0a-8f95-2384e248c27f None None] REQUEST_METHOD: `GET` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:08:25.714278 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25dcfa28-3319-4c0a-8f95-2384e248c27f None None] SCRIPT_NAME: `/identity` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:08:25.714278 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25dcfa28-3319-4c0a-8f95-2384e248c27f None None] PATH_INFO: `/` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:08:25.715781 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: [pid: 80377|app: 0|req: 691/1364] 10.222.0.82 () {54 vars in 926 bytes} [Tue Aug 2 10:08:25 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 02 10:08:25.722322 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-561c4f50-912f-4e26-9fa8-9e7e101ebe05 None None] REQUEST_METHOD: `POST` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:08:25.722810 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-561c4f50-912f-4e26-9fa8-9e7e101ebe05 None None] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:08:25.722927 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-561c4f50-912f-4e26-9fa8-9e7e101ebe05 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:08:25.813101 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.auth.core [None req-561c4f50-912f-4e26-9fa8-9e7e101ebe05 None None] MFA Rules not processed for user `891005fa5ff74041b75747d16cc66798`. Rule list: `[]` (Enabled: `True`). {{(pid=80378) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 02 10:08:25.815040 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.fernet_utils [None req-561c4f50-912f-4e26-9fa8-9e7e101ebe05 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80378) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 10:08:25.923581 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG dbcounter [-] [80378] Writing DB stats keystone:SELECT=100 {{(pid=80378) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 10:08:25.934118 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 674/1365] 10.222.0.82 () {58 vars in 1023 bytes} [Tue Aug 2 10:08:25 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 213 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 02 10:08:25.976497 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef2cb248-2b3e-430a-b1b1-c5a34d3086db None None] REQUEST_METHOD: `GET` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:08:25.976955 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef2cb248-2b3e-430a-b1b1-c5a34d3086db None None] SCRIPT_NAME: `/identity` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:08:25.977333 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef2cb248-2b3e-430a-b1b1-c5a34d3086db None None] PATH_INFO: `/` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:08:25.979670 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: [pid: 80377|app: 0|req: 692/1366] 10.222.0.82 () {54 vars in 962 bytes} [Tue Aug 2 10:08:25 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 02 10:08:25.986002 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c975661-458e-49e1-92a2-c3a522122f52 None None] REQUEST_METHOD: `POST` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:08:25.986835 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c975661-458e-49e1-92a2-c3a522122f52 None None] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:08:25.987215 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c975661-458e-49e1-92a2-c3a522122f52 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:08:26.084420 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.auth.core [None req-2c975661-458e-49e1-92a2-c3a522122f52 None None] MFA Rules not processed for user `88af518196674fe593ad88d30111fb7d`. Rule list: `[]` (Enabled: `True`). {{(pid=80378) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 02 10:08:26.085803 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.fernet_utils [None req-2c975661-458e-49e1-92a2-c3a522122f52 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80378) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 10:08:26.240307 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 675/1367] 10.222.0.82 () {58 vars in 1059 bytes} [Tue Aug 2 10:08:25 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 256 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 02 10:08:26.250155 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef2cb248-2b3e-430a-b1b1-c5a34d3086db None None] Authenticating user token {{(pid=80377) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:08:26.252780 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.fernet_utils [None req-ef2cb248-2b3e-430a-b1b1-c5a34d3086db None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80377) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 10:08:26.442459 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef2cb248-2b3e-430a-b1b1-c5a34d3086db None None] Validating token access rules against request {{(pid=80377) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:08:26.469889 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG dbcounter [-] [80377] Writing DB stats keystone:SELECT=100 {{(pid=80377) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 10:08:26.606042 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4efe6285-cc5c-4678-8394-8734de4659db service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88af518196674fe593ad88d30111fb7d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f1b38ebb804c41c69772c998aae72c41', 'project_domain_id': 'default', 'roles': ['admin', 'ResellerAdmin', 'reader', 'service', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80377) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:08:26.606589 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4efe6285-cc5c-4678-8394-8734de4659db service nova] REQUEST_METHOD: `GET` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:08:26.606874 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4efe6285-cc5c-4678-8394-8734de4659db service nova] SCRIPT_NAME: `/identity` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:08:26.607150 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4efe6285-cc5c-4678-8394-8734de4659db service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:08:26.608853 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.fernet_utils [None req-4efe6285-cc5c-4678-8394-8734de4659db service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80377) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 10:08:26.727357 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4efe6285-cc5c-4678-8394-8734de4659db service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:08:26.729063 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4efe6285-cc5c-4678-8394-8734de4659db service nova] RBAC: Authorization granted {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:08:26.752557 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: [pid: 80377|app: 0|req: 693/1368] 10.222.0.82 () {60 vars in 1365 bytes} [Tue Aug 2 10:08:26 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 503 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 02 10:08:27.679039 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1160e167-84f2-4d5d-9728-ecae083c9a3f None None] REQUEST_METHOD: `GET` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:08:27.679605 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1160e167-84f2-4d5d-9728-ecae083c9a3f None None] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:08:27.679605 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1160e167-84f2-4d5d-9728-ecae083c9a3f None None] PATH_INFO: `/` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:08:27.681070 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 676/1369] 10.222.0.82 () {54 vars in 926 bytes} [Tue Aug 2 10:08:27 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 02 10:08:27.686310 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f54c37ee-66c2-4614-8fa5-ba8407d4efb7 None None] REQUEST_METHOD: `POST` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:08:27.686608 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f54c37ee-66c2-4614-8fa5-ba8407d4efb7 None None] SCRIPT_NAME: `/identity` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:08:27.686887 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f54c37ee-66c2-4614-8fa5-ba8407d4efb7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:08:27.718794 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG dbcounter [-] [80377] Writing DB stats keystone:SELECT=100 {{(pid=80377) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 10:08:27.756110 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.auth.core [None req-f54c37ee-66c2-4614-8fa5-ba8407d4efb7 None None] MFA Rules not processed for user `891005fa5ff74041b75747d16cc66798`. Rule list: `[]` (Enabled: `True`). {{(pid=80377) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 02 10:08:27.758196 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.fernet_utils [None req-f54c37ee-66c2-4614-8fa5-ba8407d4efb7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80377) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 10:08:27.860658 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: [pid: 80377|app: 0|req: 694/1370] 10.222.0.82 () {58 vars in 1023 bytes} [Tue Aug 2 10:08:27 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 175 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 02 10:08:28.175989 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b2f6c42-5844-44dd-897d-2ef58c3859bd None None] REQUEST_METHOD: `GET` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:08:28.176276 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b2f6c42-5844-44dd-897d-2ef58c3859bd None None] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:08:28.176567 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b2f6c42-5844-44dd-897d-2ef58c3859bd None None] PATH_INFO: `/` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:08:28.178072 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 677/1371] 10.222.0.82 () {54 vars in 964 bytes} [Tue Aug 2 10:08:28 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 02 10:08:28.182869 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91d90df7-c49e-488d-8ced-79413806c977 None None] REQUEST_METHOD: `POST` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:08:28.183162 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91d90df7-c49e-488d-8ced-79413806c977 None None] SCRIPT_NAME: `/identity` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:08:28.183443 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91d90df7-c49e-488d-8ced-79413806c977 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:08:28.271190 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.auth.core [None req-91d90df7-c49e-488d-8ced-79413806c977 None None] MFA Rules not processed for user `1023de281ae74e9d8a6d6c312c1c97a7`. Rule list: `[]` (Enabled: `True`). {{(pid=80377) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 02 10:08:28.272755 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.fernet_utils [None req-91d90df7-c49e-488d-8ced-79413806c977 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80377) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 10:08:28.322814 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG dbcounter [-] [80377] Writing DB stats keystone:SELECT=100 {{(pid=80377) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 10:08:28.357420 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: [pid: 80377|app: 0|req: 695/1372] 10.222.0.82 () {58 vars in 1061 bytes} [Tue Aug 2 10:08:28 2022] POST /identity/v3/auth/tokens => generated 3120 bytes in 175 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 02 10:08:28.365943 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b2f6c42-5844-44dd-897d-2ef58c3859bd None None] Authenticating user token {{(pid=80378) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:08:28.367417 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.fernet_utils [None req-4b2f6c42-5844-44dd-897d-2ef58c3859bd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80378) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 10:08:28.423379 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG dbcounter [-] [80378] Writing DB stats keystone:SELECT=100 {{(pid=80378) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 10:08:28.483317 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b2f6c42-5844-44dd-897d-2ef58c3859bd None None] Validating token access rules against request {{(pid=80378) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:08:28.566202 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9beb7bbf-e2c5-49b6-b1a6-dec4820ca4f2 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1023de281ae74e9d8a6d6c312c1c97a7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f1b38ebb804c41c69772c998aae72c41', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80378) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:08:28.566834 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9beb7bbf-e2c5-49b6-b1a6-dec4820ca4f2 service cinder] REQUEST_METHOD: `GET` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:08:28.567116 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9beb7bbf-e2c5-49b6-b1a6-dec4820ca4f2 service cinder] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:08:28.567388 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9beb7bbf-e2c5-49b6-b1a6-dec4820ca4f2 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:08:28.569072 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.fernet_utils [None req-9beb7bbf-e2c5-49b6-b1a6-dec4820ca4f2 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80378) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 10:08:28.675889 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG dbcounter [-] [80378] Writing DB stats keystone:SELECT=100 {{(pid=80378) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 10:08:28.689860 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9beb7bbf-e2c5-49b6-b1a6-dec4820ca4f2 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:08:28.691324 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9beb7bbf-e2c5-49b6-b1a6-dec4820ca4f2 service cinder] RBAC: Authorization granted {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:08:28.711081 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 678/1373] 10.222.0.82 () {60 vars in 1365 bytes} [Tue Aug 2 10:08:28 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 346 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 02 10:08:28.910062 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b022f47-2115-4f64-808d-7a25a81fb16c None None] REQUEST_METHOD: `GET` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:08:28.910605 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b022f47-2115-4f64-808d-7a25a81fb16c None None] SCRIPT_NAME: `/identity` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:08:28.910605 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b022f47-2115-4f64-808d-7a25a81fb16c None None] PATH_INFO: `/` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:08:28.916591 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: [pid: 80377|app: 0|req: 696/1374] 10.222.0.82 () {54 vars in 926 bytes} [Tue Aug 2 10:08:28 2022] GET /identity => generated 270 bytes in 7 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 02 10:08:28.917002 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-212fd55f-3954-4e90-ae07-98df21fc7ea0 None None] REQUEST_METHOD: `POST` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:08:28.917298 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-212fd55f-3954-4e90-ae07-98df21fc7ea0 None None] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:08:28.917583 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-212fd55f-3954-4e90-ae07-98df21fc7ea0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:08:28.986910 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.auth.core [None req-212fd55f-3954-4e90-ae07-98df21fc7ea0 None None] MFA Rules not processed for user `891005fa5ff74041b75747d16cc66798`. Rule list: `[]` (Enabled: `True`). {{(pid=80378) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 02 10:08:28.987903 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.fernet_utils [None req-212fd55f-3954-4e90-ae07-98df21fc7ea0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80378) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 10:08:29.093963 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 679/1375] 10.222.0.82 () {58 vars in 1023 bytes} [Tue Aug 2 10:08:28 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 174 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 02 10:08:29.139487 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b022f47-2115-4f64-808d-7a25a81fb16c None None] Authenticating user token {{(pid=80377) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:08:29.141011 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.fernet_utils [None req-0b022f47-2115-4f64-808d-7a25a81fb16c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80377) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 10:08:29.289935 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b022f47-2115-4f64-808d-7a25a81fb16c None None] Validating token access rules against request {{(pid=80377) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:08:29.341072 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG dbcounter [-] [80377] Writing DB stats keystone:SELECT=100 {{(pid=80377) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 10:08:29.408789 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0681e000-8d6f-4655-9970-bd6e301ea7b8 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88af518196674fe593ad88d30111fb7d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f1b38ebb804c41c69772c998aae72c41', 'project_domain_id': 'default', 'roles': ['admin', 'ResellerAdmin', 'reader', 'service', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80377) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:08:29.409497 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0681e000-8d6f-4655-9970-bd6e301ea7b8 service nova] REQUEST_METHOD: `GET` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:08:29.409769 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0681e000-8d6f-4655-9970-bd6e301ea7b8 service nova] SCRIPT_NAME: `/identity` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:08:29.410234 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0681e000-8d6f-4655-9970-bd6e301ea7b8 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:08:29.411993 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.fernet_utils [None req-0681e000-8d6f-4655-9970-bd6e301ea7b8 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80377) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 10:08:29.523079 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0681e000-8d6f-4655-9970-bd6e301ea7b8 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:08:29.524911 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0681e000-8d6f-4655-9970-bd6e301ea7b8 service nova] RBAC: Authorization granted {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:08:29.543512 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: [pid: 80377|app: 0|req: 697/1376] 10.222.0.82 () {60 vars in 1365 bytes} [Tue Aug 2 10:08:29 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 404 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 02 10:08:31.636865 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-338bea4b-11ac-4c96-9726-46f52241591f None None] REQUEST_METHOD: `GET` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:08:31.637383 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-338bea4b-11ac-4c96-9726-46f52241591f None None] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:08:31.637488 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-338bea4b-11ac-4c96-9726-46f52241591f None None] PATH_INFO: `/` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:08:31.638890 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 680/1377] 10.222.0.82 () {54 vars in 926 bytes} [Tue Aug 2 10:08:31 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 02 10:08:31.644886 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74857030-d4e8-4112-ba65-76570f0774e0 None None] REQUEST_METHOD: `POST` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:08:31.645330 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74857030-d4e8-4112-ba65-76570f0774e0 None None] SCRIPT_NAME: `/identity` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:08:31.645502 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74857030-d4e8-4112-ba65-76570f0774e0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:08:31.718165 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.auth.core [None req-74857030-d4e8-4112-ba65-76570f0774e0 None None] MFA Rules not processed for user `891005fa5ff74041b75747d16cc66798`. Rule list: `[]` (Enabled: `True`). {{(pid=80377) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 02 10:08:31.719368 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.fernet_utils [None req-74857030-d4e8-4112-ba65-76570f0774e0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80377) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 10:08:31.726776 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG dbcounter [-] [80377] Writing DB stats keystone:SELECT=100 {{(pid=80377) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 10:08:31.832197 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: [pid: 80377|app: 0|req: 698/1378] 10.222.0.82 () {58 vars in 1023 bytes} [Tue Aug 2 10:08:31 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 188 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 02 10:08:31.898189 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-338bea4b-11ac-4c96-9726-46f52241591f None None] Authenticating user token {{(pid=80378) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:08:31.899740 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.fernet_utils [None req-338bea4b-11ac-4c96-9726-46f52241591f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80378) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 10:08:31.963671 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG dbcounter [-] [80378] Writing DB stats keystone:SELECT=100 {{(pid=80378) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 10:08:32.050840 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-338bea4b-11ac-4c96-9726-46f52241591f None None] Validating token access rules against request {{(pid=80378) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:08:32.172929 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71fac8a8-b498-4806-97d6-8c6cb3d8b64d service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88af518196674fe593ad88d30111fb7d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f1b38ebb804c41c69772c998aae72c41', 'project_domain_id': 'default', 'roles': ['admin', 'ResellerAdmin', 'reader', 'service', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80378) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:08:32.174113 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71fac8a8-b498-4806-97d6-8c6cb3d8b64d service nova] REQUEST_METHOD: `GET` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:08:32.174396 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71fac8a8-b498-4806-97d6-8c6cb3d8b64d service nova] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:08:32.174666 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71fac8a8-b498-4806-97d6-8c6cb3d8b64d service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:08:32.176420 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.fernet_utils [None req-71fac8a8-b498-4806-97d6-8c6cb3d8b64d service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80378) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 10:08:32.195013 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG dbcounter [-] [80378] Writing DB stats keystone:SELECT=100 {{(pid=80378) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 10:08:32.289745 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71fac8a8-b498-4806-97d6-8c6cb3d8b64d service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:08:32.291385 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71fac8a8-b498-4806-97d6-8c6cb3d8b64d service nova] RBAC: Authorization granted {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:08:32.326329 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 681/1379] 10.222.0.82 () {60 vars in 1365 bytes} [Tue Aug 2 10:08:31 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 428 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 02 10:08:34.514332 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd3e41a1-b9a5-4d3a-a958-e0189b53032f None None] REQUEST_METHOD: `GET` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:08:34.515033 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd3e41a1-b9a5-4d3a-a958-e0189b53032f None None] SCRIPT_NAME: `/identity` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:08:34.515033 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd3e41a1-b9a5-4d3a-a958-e0189b53032f None None] PATH_INFO: `/` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:08:34.516928 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: [pid: 80377|app: 0|req: 699/1380] 10.222.0.82 () {54 vars in 926 bytes} [Tue Aug 2 10:08:34 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 02 10:08:34.523849 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d6ea8d0-ac5a-4902-be48-899ce45a331b None None] REQUEST_METHOD: `POST` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:08:34.524446 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d6ea8d0-ac5a-4902-be48-899ce45a331b None None] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:08:34.524570 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d6ea8d0-ac5a-4902-be48-899ce45a331b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:08:34.607840 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.auth.core [None req-7d6ea8d0-ac5a-4902-be48-899ce45a331b None None] MFA Rules not processed for user `891005fa5ff74041b75747d16cc66798`. Rule list: `[]` (Enabled: `True`). {{(pid=80378) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 02 10:08:34.609055 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.fernet_utils [None req-7d6ea8d0-ac5a-4902-be48-899ce45a331b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80378) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 10:08:34.734116 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG dbcounter [-] [80378] Writing DB stats keystone:SELECT=100 {{(pid=80378) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 10:08:34.750032 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 682/1381] 10.222.0.82 () {58 vars in 1023 bytes} [Tue Aug 2 10:08:34 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 227 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 02 10:08:34.798319 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd3e41a1-b9a5-4d3a-a958-e0189b53032f None None] Authenticating user token {{(pid=80377) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:08:34.799960 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.fernet_utils [None req-bd3e41a1-b9a5-4d3a-a958-e0189b53032f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80377) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 10:08:34.993229 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG dbcounter [-] [80377] Writing DB stats keystone:SELECT=100 {{(pid=80377) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 10:08:35.002786 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd3e41a1-b9a5-4d3a-a958-e0189b53032f None None] Validating token access rules against request {{(pid=80377) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:08:35.151228 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7185fa99-af5b-4903-a216-68c7f9e359cd service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88af518196674fe593ad88d30111fb7d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f1b38ebb804c41c69772c998aae72c41', 'project_domain_id': 'default', 'roles': ['admin', 'ResellerAdmin', 'reader', 'service', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80377) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:08:35.151768 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7185fa99-af5b-4903-a216-68c7f9e359cd service nova] REQUEST_METHOD: `GET` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:08:35.152050 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7185fa99-af5b-4903-a216-68c7f9e359cd service nova] SCRIPT_NAME: `/identity` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:08:35.152323 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7185fa99-af5b-4903-a216-68c7f9e359cd service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:08:35.153985 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.fernet_utils [None req-7185fa99-af5b-4903-a216-68c7f9e359cd service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80377) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 10:08:35.257165 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7185fa99-af5b-4903-a216-68c7f9e359cd service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:08:35.258891 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7185fa99-af5b-4903-a216-68c7f9e359cd service nova] RBAC: Authorization granted {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:08:35.271207 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG dbcounter [-] [80377] Writing DB stats keystone:SELECT=100 {{(pid=80377) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 10:08:35.281187 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: [pid: 80377|app: 0|req: 700/1382] 10.222.0.82 () {60 vars in 1365 bytes} [Tue Aug 2 10:08:34 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 483 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 02 10:08:37.377144 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f86f155-50b1-490d-96be-e6ff4494b6ce None None] REQUEST_METHOD: `GET` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:08:37.378138 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f86f155-50b1-490d-96be-e6ff4494b6ce None None] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:08:37.378138 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f86f155-50b1-490d-96be-e6ff4494b6ce None None] PATH_INFO: `/` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:08:37.379183 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 683/1383] 10.222.0.82 () {54 vars in 926 bytes} [Tue Aug 2 10:08:37 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 02 10:08:37.384741 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9235123-dec2-4246-a2dd-7830bf1a6f19 None None] REQUEST_METHOD: `POST` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:08:37.385298 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9235123-dec2-4246-a2dd-7830bf1a6f19 None None] SCRIPT_NAME: `/identity` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:08:37.385298 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9235123-dec2-4246-a2dd-7830bf1a6f19 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:08:37.473490 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.auth.core [None req-c9235123-dec2-4246-a2dd-7830bf1a6f19 None None] MFA Rules not processed for user `891005fa5ff74041b75747d16cc66798`. Rule list: `[]` (Enabled: `True`). {{(pid=80377) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 02 10:08:37.474815 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.fernet_utils [None req-c9235123-dec2-4246-a2dd-7830bf1a6f19 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80377) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 10:08:37.584553 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: [pid: 80377|app: 0|req: 701/1384] 10.222.0.82 () {58 vars in 1023 bytes} [Tue Aug 2 10:08:37 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 201 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 02 10:08:37.630907 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f86f155-50b1-490d-96be-e6ff4494b6ce None None] Authenticating user token {{(pid=80378) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:08:37.632434 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.fernet_utils [None req-1f86f155-50b1-490d-96be-e6ff4494b6ce None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80378) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 10:08:37.802823 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f86f155-50b1-490d-96be-e6ff4494b6ce None None] Validating token access rules against request {{(pid=80378) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:08:37.877736 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG dbcounter [-] [80378] Writing DB stats keystone:SELECT=100 {{(pid=80378) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 10:08:37.928331 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2eb83a7a-61ba-435e-a58a-cfef4cb015e7 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88af518196674fe593ad88d30111fb7d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f1b38ebb804c41c69772c998aae72c41', 'project_domain_id': 'default', 'roles': ['admin', 'ResellerAdmin', 'reader', 'service', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80378) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:08:37.928975 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2eb83a7a-61ba-435e-a58a-cfef4cb015e7 service nova] REQUEST_METHOD: `GET` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:08:37.929255 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2eb83a7a-61ba-435e-a58a-cfef4cb015e7 service nova] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:08:37.929524 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2eb83a7a-61ba-435e-a58a-cfef4cb015e7 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:08:37.931189 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.fernet_utils [None req-2eb83a7a-61ba-435e-a58a-cfef4cb015e7 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80378) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 10:08:38.037170 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2eb83a7a-61ba-435e-a58a-cfef4cb015e7 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:08:38.039318 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2eb83a7a-61ba-435e-a58a-cfef4cb015e7 service nova] RBAC: Authorization granted {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:08:38.058668 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 684/1385] 10.222.0.82 () {60 vars in 1365 bytes} [Tue Aug 2 10:08:37 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 428 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 02 10:08:40.242454 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94c67049-5964-452e-94b4-3c4165df541a None None] REQUEST_METHOD: `GET` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:08:40.243328 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94c67049-5964-452e-94b4-3c4165df541a None None] SCRIPT_NAME: `/identity` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:08:40.243328 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94c67049-5964-452e-94b4-3c4165df541a None None] PATH_INFO: `/` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:08:40.244682 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: [pid: 80377|app: 0|req: 702/1386] 10.222.0.82 () {54 vars in 926 bytes} [Tue Aug 2 10:08:40 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 02 10:08:40.250912 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af67cfee-cce0-4e39-8942-3f02638b49c2 None None] REQUEST_METHOD: `POST` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:08:40.252103 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af67cfee-cce0-4e39-8942-3f02638b49c2 None None] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:08:40.252685 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af67cfee-cce0-4e39-8942-3f02638b49c2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:08:40.349124 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.auth.core [None req-af67cfee-cce0-4e39-8942-3f02638b49c2 None None] MFA Rules not processed for user `891005fa5ff74041b75747d16cc66798`. Rule list: `[]` (Enabled: `True`). {{(pid=80378) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 02 10:08:40.350836 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.fernet_utils [None req-af67cfee-cce0-4e39-8942-3f02638b49c2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80378) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 10:08:40.392957 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG dbcounter [-] [80378] Writing DB stats keystone:SELECT=100 {{(pid=80378) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 10:08:40.475311 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 685/1387] 10.222.0.82 () {58 vars in 1023 bytes} [Tue Aug 2 10:08:40 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 226 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 02 10:08:40.521875 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94c67049-5964-452e-94b4-3c4165df541a None None] Authenticating user token {{(pid=80377) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:08:40.523553 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.fernet_utils [None req-94c67049-5964-452e-94b4-3c4165df541a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80377) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 10:08:40.623913 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG dbcounter [-] [80377] Writing DB stats keystone:SELECT=100 {{(pid=80377) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 10:08:40.717640 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94c67049-5964-452e-94b4-3c4165df541a None None] Validating token access rules against request {{(pid=80377) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:08:40.884366 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66d43d3f-490e-41e3-b18f-27f03e6e3585 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88af518196674fe593ad88d30111fb7d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f1b38ebb804c41c69772c998aae72c41', 'project_domain_id': 'default', 'roles': ['admin', 'ResellerAdmin', 'reader', 'service', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80377) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:08:40.885268 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66d43d3f-490e-41e3-b18f-27f03e6e3585 service nova] REQUEST_METHOD: `GET` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:08:40.885633 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66d43d3f-490e-41e3-b18f-27f03e6e3585 service nova] SCRIPT_NAME: `/identity` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:08:40.886181 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66d43d3f-490e-41e3-b18f-27f03e6e3585 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:08:40.888674 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.fernet_utils [None req-66d43d3f-490e-41e3-b18f-27f03e6e3585 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80377) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 10:08:40.951191 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG dbcounter [-] [80377] Writing DB stats keystone:SELECT=100 {{(pid=80377) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 10:08:41.017806 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66d43d3f-490e-41e3-b18f-27f03e6e3585 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:08:41.019536 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66d43d3f-490e-41e3-b18f-27f03e6e3585 service nova] RBAC: Authorization granted {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:08:41.041253 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: [pid: 80377|app: 0|req: 703/1388] 10.222.0.82 () {60 vars in 1365 bytes} [Tue Aug 2 10:08:40 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 520 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 02 10:08:43.306275 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d0c4ae2-d954-4ff4-9f0f-1442b32e1d5c None None] REQUEST_METHOD: `GET` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:08:43.306838 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d0c4ae2-d954-4ff4-9f0f-1442b32e1d5c None None] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:08:43.306838 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d0c4ae2-d954-4ff4-9f0f-1442b32e1d5c None None] PATH_INFO: `/` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:08:43.308201 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 686/1389] 10.222.0.82 () {54 vars in 926 bytes} [Tue Aug 2 10:08:43 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 02 10:08:43.313369 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-314ab01c-0fe5-4428-a3b8-372fb29a7c4f None None] REQUEST_METHOD: `POST` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:08:43.313864 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-314ab01c-0fe5-4428-a3b8-372fb29a7c4f None None] SCRIPT_NAME: `/identity` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:08:43.313962 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-314ab01c-0fe5-4428-a3b8-372fb29a7c4f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:08:43.381270 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.auth.core [None req-314ab01c-0fe5-4428-a3b8-372fb29a7c4f None None] MFA Rules not processed for user `891005fa5ff74041b75747d16cc66798`. Rule list: `[]` (Enabled: `True`). {{(pid=80377) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 02 10:08:43.382591 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.fernet_utils [None req-314ab01c-0fe5-4428-a3b8-372fb29a7c4f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80377) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 10:08:43.486230 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: [pid: 80377|app: 0|req: 704/1390] 10.222.0.82 () {58 vars in 1023 bytes} [Tue Aug 2 10:08:43 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 174 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 02 10:08:43.530542 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d0c4ae2-d954-4ff4-9f0f-1442b32e1d5c None None] Authenticating user token {{(pid=80378) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:08:43.532803 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.fernet_utils [None req-9d0c4ae2-d954-4ff4-9f0f-1442b32e1d5c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80378) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 10:08:43.684348 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d0c4ae2-d954-4ff4-9f0f-1442b32e1d5c None None] Validating token access rules against request {{(pid=80378) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:08:43.701798 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG dbcounter [-] [80378] Writing DB stats keystone:SELECT=100 {{(pid=80378) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 10:08:43.801530 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-526412be-160c-4166-ae9a-5d2de8799ff6 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88af518196674fe593ad88d30111fb7d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f1b38ebb804c41c69772c998aae72c41', 'project_domain_id': 'default', 'roles': ['admin', 'ResellerAdmin', 'reader', 'service', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80378) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:08:43.802193 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-526412be-160c-4166-ae9a-5d2de8799ff6 service nova] REQUEST_METHOD: `GET` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:08:43.802478 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-526412be-160c-4166-ae9a-5d2de8799ff6 service nova] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:08:43.802815 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-526412be-160c-4166-ae9a-5d2de8799ff6 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:08:43.804636 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.fernet_utils [None req-526412be-160c-4166-ae9a-5d2de8799ff6 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80378) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 10:08:43.911921 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-526412be-160c-4166-ae9a-5d2de8799ff6 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:08:43.913540 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-526412be-160c-4166-ae9a-5d2de8799ff6 service nova] RBAC: Authorization granted {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:08:43.933042 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 687/1391] 10.222.0.82 () {60 vars in 1365 bytes} [Tue Aug 2 10:08:43 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 403 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 02 10:08:44.774986 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4df6505f-29b8-4ac3-8a39-c5c9b20e2687 None None] REQUEST_METHOD: `GET` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:08:44.775967 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4df6505f-29b8-4ac3-8a39-c5c9b20e2687 None None] SCRIPT_NAME: `/identity` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:08:44.775967 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4df6505f-29b8-4ac3-8a39-c5c9b20e2687 None None] PATH_INFO: `/` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:08:44.776903 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: [pid: 80377|app: 0|req: 705/1392] 10.222.0.82 () {54 vars in 926 bytes} [Tue Aug 2 10:08:44 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 02 10:08:44.783762 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6856e59b-6cee-453c-88b1-8761282638dd None None] REQUEST_METHOD: `POST` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:08:44.784273 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6856e59b-6cee-453c-88b1-8761282638dd None None] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:08:44.784802 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6856e59b-6cee-453c-88b1-8761282638dd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:08:44.813346 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG dbcounter [-] [80378] Writing DB stats keystone:SELECT=100 {{(pid=80378) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 10:08:44.871022 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.auth.core [None req-6856e59b-6cee-453c-88b1-8761282638dd None None] MFA Rules not processed for user `891005fa5ff74041b75747d16cc66798`. Rule list: `[]` (Enabled: `True`). {{(pid=80378) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 02 10:08:44.873517 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.fernet_utils [None req-6856e59b-6cee-453c-88b1-8761282638dd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80378) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 10:08:45.000195 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 688/1393] 10.222.0.82 () {58 vars in 1023 bytes} [Tue Aug 2 10:08:44 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 219 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 02 10:08:45.062886 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c00bcd7d-c43c-4072-b612-0a68c5ffa133 None None] REQUEST_METHOD: `GET` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:08:45.063193 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c00bcd7d-c43c-4072-b612-0a68c5ffa133 None None] SCRIPT_NAME: `/identity` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:08:45.064124 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c00bcd7d-c43c-4072-b612-0a68c5ffa133 None None] PATH_INFO: `/` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:08:45.065528 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: [pid: 80377|app: 0|req: 706/1394] 10.222.0.82 () {54 vars in 966 bytes} [Tue Aug 2 10:08:45 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 02 10:08:45.071014 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-857d46a1-9fa3-4132-a3e9-f749ca8b48a1 None None] REQUEST_METHOD: `POST` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:08:45.071306 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-857d46a1-9fa3-4132-a3e9-f749ca8b48a1 None None] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:08:45.071584 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-857d46a1-9fa3-4132-a3e9-f749ca8b48a1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:08:45.157462 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.auth.core [None req-857d46a1-9fa3-4132-a3e9-f749ca8b48a1 None None] MFA Rules not processed for user `fa964af53a2b4fb9ace6afb544d54d40`. Rule list: `[]` (Enabled: `True`). {{(pid=80378) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 02 10:08:45.158776 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.fernet_utils [None req-857d46a1-9fa3-4132-a3e9-f749ca8b48a1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80378) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 10:08:45.204924 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG dbcounter [-] [80378] Writing DB stats keystone:SELECT=100 {{(pid=80378) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 10:08:45.251489 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 689/1395] 10.222.0.82 () {58 vars in 1063 bytes} [Tue Aug 2 10:08:45 2022] POST /identity/v3/auth/tokens => generated 3120 bytes in 181 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 02 10:08:45.262571 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c00bcd7d-c43c-4072-b612-0a68c5ffa133 None None] Authenticating user token {{(pid=80377) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:08:45.264935 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.fernet_utils [None req-c00bcd7d-c43c-4072-b612-0a68c5ffa133 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80377) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 10:08:45.281666 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG dbcounter [-] [80377] Writing DB stats keystone:SELECT=100 {{(pid=80377) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 10:08:45.384814 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c00bcd7d-c43c-4072-b612-0a68c5ffa133 None None] Validating token access rules against request {{(pid=80377) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:08:45.460290 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7efa2ea-2c44-41fa-87d9-c434928312f1 service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa964af53a2b4fb9ace6afb544d54d40', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f1b38ebb804c41c69772c998aae72c41', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80377) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:08:45.460911 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7efa2ea-2c44-41fa-87d9-c434928312f1 service glance] REQUEST_METHOD: `GET` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:08:45.461195 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7efa2ea-2c44-41fa-87d9-c434928312f1 service glance] SCRIPT_NAME: `/identity` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:08:45.461468 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7efa2ea-2c44-41fa-87d9-c434928312f1 service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:08:45.463116 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.fernet_utils [None req-b7efa2ea-2c44-41fa-87d9-c434928312f1 service glance] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80377) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 10:08:45.539816 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG dbcounter [-] [80377] Writing DB stats keystone:SELECT=100 {{(pid=80377) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 10:08:45.585267 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7efa2ea-2c44-41fa-87d9-c434928312f1 service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:08:45.586746 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7efa2ea-2c44-41fa-87d9-c434928312f1 service glance] RBAC: Authorization granted {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:08:45.606879 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: [pid: 80377|app: 0|req: 707/1396] 10.222.0.82 () {60 vars in 1365 bytes} [Tue Aug 2 10:08:45 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 346 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 02 10:08:46.048233 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-899cb88b-98e7-4db9-a831-44774577676b None None] REQUEST_METHOD: `GET` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:08:46.049161 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-899cb88b-98e7-4db9-a831-44774577676b None None] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:08:46.049161 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-899cb88b-98e7-4db9-a831-44774577676b None None] PATH_INFO: `/` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:08:46.050315 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 690/1397] 10.222.0.82 () {54 vars in 926 bytes} [Tue Aug 2 10:08:46 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 02 10:08:46.056044 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d90a35b6-3fec-47db-8877-dbead675f505 None None] REQUEST_METHOD: `POST` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:08:46.056546 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d90a35b6-3fec-47db-8877-dbead675f505 None None] SCRIPT_NAME: `/identity` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:08:46.056650 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d90a35b6-3fec-47db-8877-dbead675f505 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:08:46.126595 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.auth.core [None req-d90a35b6-3fec-47db-8877-dbead675f505 None None] MFA Rules not processed for user `891005fa5ff74041b75747d16cc66798`. Rule list: `[]` (Enabled: `True`). {{(pid=80377) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 02 10:08:46.127847 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.fernet_utils [None req-d90a35b6-3fec-47db-8877-dbead675f505 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80377) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 10:08:46.247502 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: [pid: 80377|app: 0|req: 708/1398] 10.222.0.82 () {58 vars in 1023 bytes} [Tue Aug 2 10:08:46 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 192 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 02 10:08:46.295864 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-899cb88b-98e7-4db9-a831-44774577676b None None] Authenticating user token {{(pid=80378) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:08:46.297430 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.fernet_utils [None req-899cb88b-98e7-4db9-a831-44774577676b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80378) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 10:08:46.371155 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f36d0b9-2f46-43aa-8492-e249c66aa28f None None] REQUEST_METHOD: `POST` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:08:46.371523 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f36d0b9-2f46-43aa-8492-e249c66aa28f None None] SCRIPT_NAME: `/identity` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:08:46.371928 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f36d0b9-2f46-43aa-8492-e249c66aa28f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:08:46.401661 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG dbcounter [-] [80377] Writing DB stats keystone:SELECT=100 {{(pid=80377) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 10:08:46.453666 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.auth.core [None req-2f36d0b9-2f46-43aa-8492-e249c66aa28f None None] MFA Rules not processed for user `bd8dd72086db48b8aaac2ed0058872df`. Rule list: `[]` (Enabled: `True`). {{(pid=80377) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 02 10:08:46.455125 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.fernet_utils [None req-2f36d0b9-2f46-43aa-8492-e249c66aa28f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80377) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 10:08:46.463213 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-899cb88b-98e7-4db9-a831-44774577676b None None] Validating token access rules against request {{(pid=80378) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:08:46.514882 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG dbcounter [-] [80378] Writing DB stats keystone:SELECT=100 {{(pid=80378) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 10:08:46.562523 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: [pid: 80377|app: 0|req: 709/1399] 10.222.0.82 () {58 vars in 1009 bytes} [Tue Aug 2 10:08:46 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 192 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 02 10:08:46.591405 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a42377f7-6290-433c-a8e2-ad76fe40088a service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88af518196674fe593ad88d30111fb7d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f1b38ebb804c41c69772c998aae72c41', 'project_domain_id': 'default', 'roles': ['admin', 'ResellerAdmin', 'reader', 'service', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80378) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:08:46.592043 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a42377f7-6290-433c-a8e2-ad76fe40088a service nova] REQUEST_METHOD: `GET` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:08:46.592328 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a42377f7-6290-433c-a8e2-ad76fe40088a service nova] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:08:46.592602 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a42377f7-6290-433c-a8e2-ad76fe40088a service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:08:46.595263 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.fernet_utils [None req-a42377f7-6290-433c-a8e2-ad76fe40088a service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80378) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 10:08:46.613889 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3124c70-929a-401b-acb6-a9887668e76f None None] REQUEST_METHOD: `GET` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:08:46.614150 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3124c70-929a-401b-acb6-a9887668e76f None None] SCRIPT_NAME: `/identity` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:08:46.614459 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3124c70-929a-401b-acb6-a9887668e76f None None] PATH_INFO: `/` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:08:46.616186 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: [pid: 80377|app: 0|req: 710/1400] 10.222.0.82 () {54 vars in 943 bytes} [Tue Aug 2 10:08:46 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 02 10:08:46.624179 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8dfc24b6-2bc1-468c-86e1-7eba657efd5a None None] REQUEST_METHOD: `POST` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:08:46.624179 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8dfc24b6-2bc1-468c-86e1-7eba657efd5a None None] SCRIPT_NAME: `/identity` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:08:46.624179 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8dfc24b6-2bc1-468c-86e1-7eba657efd5a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:08:46.695329 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a42377f7-6290-433c-a8e2-ad76fe40088a service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:08:46.697352 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a42377f7-6290-433c-a8e2-ad76fe40088a service nova] RBAC: Authorization granted {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:08:46.704257 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.auth.core [None req-8dfc24b6-2bc1-468c-86e1-7eba657efd5a None None] MFA Rules not processed for user `bef1b2e714cb45c4a51eb6d3e15aad6d`. Rule list: `[]` (Enabled: `True`). {{(pid=80377) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 02 10:08:46.705489 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.fernet_utils [None req-8dfc24b6-2bc1-468c-86e1-7eba657efd5a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80377) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 10:08:46.716713 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 691/1401] 10.222.0.82 () {60 vars in 1365 bytes} [Tue Aug 2 10:08:46 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 421 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 02 10:08:46.762583 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG dbcounter [-] [80377] Writing DB stats keystone:SELECT=100 {{(pid=80377) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 10:08:46.826820 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: [pid: 80377|app: 0|req: 711/1402] 10.222.0.82 () {58 vars in 1040 bytes} [Tue Aug 2 10:08:46 2022] POST /identity/v3/auth/tokens => generated 3304 bytes in 205 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 02 10:08:46.837478 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a42377f7-6290-433c-a8e2-ad76fe40088a service nova] Authenticating user token {{(pid=80378) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:08:46.838755 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.fernet_utils [None req-a42377f7-6290-433c-a8e2-ad76fe40088a service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80378) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 10:08:46.905192 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG dbcounter [-] [80378] Writing DB stats keystone:SELECT=100 {{(pid=80378) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 10:08:46.956278 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a42377f7-6290-433c-a8e2-ad76fe40088a service nova] Validating token access rules against request {{(pid=80378) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:08:47.071347 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07deb596-5dde-4f8e-a828-259f45263419 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bef1b2e714cb45c4a51eb6d3e15aad6d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f1b38ebb804c41c69772c998aae72c41', 'project_domain_id': 'default', 'roles': ['admin', 'service', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80378) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:08:47.072161 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07deb596-5dde-4f8e-a828-259f45263419 service swift] REQUEST_METHOD: `GET` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:08:47.072316 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07deb596-5dde-4f8e-a828-259f45263419 service swift] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:08:47.072586 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07deb596-5dde-4f8e-a828-259f45263419 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:08:47.074308 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.fernet_utils [None req-07deb596-5dde-4f8e-a828-259f45263419 service swift] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80378) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 10:08:47.164980 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07deb596-5dde-4f8e-a828-259f45263419 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:08:47.166558 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07deb596-5dde-4f8e-a828-259f45263419 service swift] RBAC: Authorization granted {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:08:47.171417 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG dbcounter [-] [80378] Writing DB stats keystone:SELECT=100 {{(pid=80378) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 10:08:47.178980 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 692/1403] 10.222.0.82 () {60 vars in 1384 bytes} [Tue Aug 2 10:08:46 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 342 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Aug 02 10:08:48.994248 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ee36c6d-ac29-4c2e-aefb-eef9bafbd544 None None] REQUEST_METHOD: `GET` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:08:48.994812 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ee36c6d-ac29-4c2e-aefb-eef9bafbd544 None None] SCRIPT_NAME: `/identity` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:08:48.994915 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ee36c6d-ac29-4c2e-aefb-eef9bafbd544 None None] PATH_INFO: `/` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:08:49.000382 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: [pid: 80377|app: 0|req: 712/1404] 10.222.0.82 () {54 vars in 926 bytes} [Tue Aug 2 10:08:48 2022] GET /identity => generated 270 bytes in 7 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 02 10:08:49.002796 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61de0a87-f94f-4737-810f-7d5ec2100e78 None None] REQUEST_METHOD: `POST` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:08:49.003511 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61de0a87-f94f-4737-810f-7d5ec2100e78 None None] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:08:49.003511 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61de0a87-f94f-4737-810f-7d5ec2100e78 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:08:49.104462 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.auth.core [None req-61de0a87-f94f-4737-810f-7d5ec2100e78 None None] MFA Rules not processed for user `891005fa5ff74041b75747d16cc66798`. Rule list: `[]` (Enabled: `True`). {{(pid=80378) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 02 10:08:49.106547 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.fernet_utils [None req-61de0a87-f94f-4737-810f-7d5ec2100e78 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80378) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 10:08:49.275687 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 693/1405] 10.222.0.82 () {58 vars in 1023 bytes} [Tue Aug 2 10:08:49 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 275 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 02 10:08:49.345304 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ee36c6d-ac29-4c2e-aefb-eef9bafbd544 None None] Authenticating user token {{(pid=80377) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:08:49.346974 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.fernet_utils [None req-6ee36c6d-ac29-4c2e-aefb-eef9bafbd544 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80377) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 10:08:49.573688 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ee36c6d-ac29-4c2e-aefb-eef9bafbd544 None None] Validating token access rules against request {{(pid=80377) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:08:49.607471 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG dbcounter [-] [80377] Writing DB stats keystone:SELECT=100 {{(pid=80377) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 10:08:49.726604 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-166c0ce8-cead-4508-a3a2-d84375e2f01d service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88af518196674fe593ad88d30111fb7d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f1b38ebb804c41c69772c998aae72c41', 'project_domain_id': 'default', 'roles': ['admin', 'ResellerAdmin', 'reader', 'service', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80377) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:08:49.727381 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-166c0ce8-cead-4508-a3a2-d84375e2f01d service nova] REQUEST_METHOD: `GET` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:08:49.727667 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-166c0ce8-cead-4508-a3a2-d84375e2f01d service nova] SCRIPT_NAME: `/identity` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:08:49.727933 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-166c0ce8-cead-4508-a3a2-d84375e2f01d service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:08:49.729606 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.fernet_utils [None req-166c0ce8-cead-4508-a3a2-d84375e2f01d service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80377) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 10:08:49.855870 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-166c0ce8-cead-4508-a3a2-d84375e2f01d service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:08:49.857559 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-166c0ce8-cead-4508-a3a2-d84375e2f01d service nova] RBAC: Authorization granted {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:08:49.885973 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: [pid: 80377|app: 0|req: 713/1406] 10.222.0.82 () {60 vars in 1365 bytes} [Tue Aug 2 10:08:49 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 542 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 02 10:08:52.138525 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f71f7566-9c60-4f6e-9887-5f975288d131 None None] REQUEST_METHOD: `GET` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:08:52.139051 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f71f7566-9c60-4f6e-9887-5f975288d131 None None] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:08:52.139051 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f71f7566-9c60-4f6e-9887-5f975288d131 None None] PATH_INFO: `/` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:08:52.140382 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 694/1407] 10.222.0.82 () {54 vars in 926 bytes} [Tue Aug 2 10:08:52 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 02 10:08:52.146561 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5078898e-e194-444a-a365-9fb7181954f7 None None] REQUEST_METHOD: `POST` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:08:52.147087 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5078898e-e194-444a-a365-9fb7181954f7 None None] SCRIPT_NAME: `/identity` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:08:52.147216 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5078898e-e194-444a-a365-9fb7181954f7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:08:52.201558 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG dbcounter [-] [80377] Writing DB stats keystone:SELECT=100 {{(pid=80377) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 10:08:52.255905 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.auth.core [None req-5078898e-e194-444a-a365-9fb7181954f7 None None] MFA Rules not processed for user `891005fa5ff74041b75747d16cc66798`. Rule list: `[]` (Enabled: `True`). {{(pid=80377) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 02 10:08:52.258154 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.fernet_utils [None req-5078898e-e194-444a-a365-9fb7181954f7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80377) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 10:08:52.495182 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: [pid: 80377|app: 0|req: 714/1408] 10.222.0.82 () {58 vars in 1023 bytes} [Tue Aug 2 10:08:52 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 350 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 02 10:08:52.572156 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f71f7566-9c60-4f6e-9887-5f975288d131 None None] Authenticating user token {{(pid=80378) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:08:52.574135 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.fernet_utils [None req-f71f7566-9c60-4f6e-9887-5f975288d131 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80378) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 10:08:52.665311 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG dbcounter [-] [80378] Writing DB stats keystone:SELECT=100 {{(pid=80378) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 10:08:52.751323 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f71f7566-9c60-4f6e-9887-5f975288d131 None None] Validating token access rules against request {{(pid=80378) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:08:52.897401 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d47a47e7-481a-4cab-a88b-fdd42e734491 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88af518196674fe593ad88d30111fb7d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f1b38ebb804c41c69772c998aae72c41', 'project_domain_id': 'default', 'roles': ['admin', 'ResellerAdmin', 'reader', 'service', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80378) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:08:52.898065 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d47a47e7-481a-4cab-a88b-fdd42e734491 service nova] REQUEST_METHOD: `GET` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:08:52.898348 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d47a47e7-481a-4cab-a88b-fdd42e734491 service nova] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:08:52.898624 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d47a47e7-481a-4cab-a88b-fdd42e734491 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:08:52.900259 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.fernet_utils [None req-d47a47e7-481a-4cab-a88b-fdd42e734491 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80378) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 10:08:52.949507 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG dbcounter [-] [80378] Writing DB stats keystone:SELECT=100 {{(pid=80378) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 10:08:53.005929 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d47a47e7-481a-4cab-a88b-fdd42e734491 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:08:53.007460 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d47a47e7-481a-4cab-a88b-fdd42e734491 service nova] RBAC: Authorization granted {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:08:53.027467 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 695/1409] 10.222.0.82 () {60 vars in 1365 bytes} [Tue Aug 2 10:08:52 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 456 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 02 10:09:02.488995 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG dbcounter [-] [80377] Writing DB stats keystone:SELECT=51 {{(pid=80377) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 10:09:03.025215 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG dbcounter [-] [80378] Writing DB stats keystone:SELECT=27 {{(pid=80378) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 10:09:10.368262 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f22569f-b43e-41d3-8083-2d32a633f08a None None] REQUEST_METHOD: `GET` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:09:10.368855 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f22569f-b43e-41d3-8083-2d32a633f08a None None] SCRIPT_NAME: `/identity` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:09:10.368855 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f22569f-b43e-41d3-8083-2d32a633f08a None None] PATH_INFO: `/` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:09:10.370280 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: [pid: 80377|app: 0|req: 715/1410] 10.222.0.82 () {54 vars in 918 bytes} [Tue Aug 2 10:09:10 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 02 10:09:10.375789 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0c2fa1e-7a05-4d3a-9e1a-c9289b502dd6 None None] REQUEST_METHOD: `POST` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:09:10.376703 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0c2fa1e-7a05-4d3a-9e1a-c9289b502dd6 None None] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:09:10.376703 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0c2fa1e-7a05-4d3a-9e1a-c9289b502dd6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:09:10.479013 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.auth.core [None req-b0c2fa1e-7a05-4d3a-9e1a-c9289b502dd6 None None] MFA Rules not processed for user `ae6e910e50d74f29bfb89bf18d95a908`. Rule list: `[]` (Enabled: `True`). {{(pid=80378) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 02 10:09:10.480238 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.fernet_utils [None req-b0c2fa1e-7a05-4d3a-9e1a-c9289b502dd6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80378) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 10:09:10.611845 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 696/1411] 10.222.0.82 () {58 vars in 1015 bytes} [Tue Aug 2 10:09:10 2022] POST /identity/v3/auth/tokens => generated 3308 bytes in 237 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 02 10:09:10.627879 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62473d97-8fc7-4bd9-b744-19207eaa574c None None] REQUEST_METHOD: `GET` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:09:10.628157 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62473d97-8fc7-4bd9-b744-19207eaa574c None None] SCRIPT_NAME: `/identity` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:09:10.628434 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62473d97-8fc7-4bd9-b744-19207eaa574c None None] PATH_INFO: `/` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:09:10.629810 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: [pid: 80377|app: 0|req: 716/1412] 10.222.0.82 () {54 vars in 918 bytes} [Tue Aug 2 10:09:10 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 02 10:09:10.634758 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-181e6ae2-60a9-4292-9674-b45ab95ad5d5 None None] REQUEST_METHOD: `POST` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:09:10.635048 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-181e6ae2-60a9-4292-9674-b45ab95ad5d5 None None] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:09:10.635321 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-181e6ae2-60a9-4292-9674-b45ab95ad5d5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:09:10.710932 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG dbcounter [-] [80378] Writing DB stats keystone:SELECT=100 {{(pid=80378) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 10:09:10.714049 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.auth.core [None req-181e6ae2-60a9-4292-9674-b45ab95ad5d5 None None] MFA Rules not processed for user `88af518196674fe593ad88d30111fb7d`. Rule list: `[]` (Enabled: `True`). {{(pid=80378) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 02 10:09:10.716010 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.fernet_utils [None req-181e6ae2-60a9-4292-9674-b45ab95ad5d5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80378) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 10:09:10.836309 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 697/1413] 10.222.0.82 () {58 vars in 1015 bytes} [Tue Aug 2 10:09:10 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 203 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 02 10:09:13.347423 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0f3a582-f60c-4800-af2d-a4a7e4f2f09a None None] REQUEST_METHOD: `GET` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:09:13.348009 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0f3a582-f60c-4800-af2d-a4a7e4f2f09a None None] SCRIPT_NAME: `/identity` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:09:13.348009 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0f3a582-f60c-4800-af2d-a4a7e4f2f09a None None] PATH_INFO: `/` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:09:13.349313 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: [pid: 80377|app: 0|req: 717/1414] 127.0.0.1 () {54 vars in 916 bytes} [Tue Aug 2 10:09:13 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 02 10:09:13.354844 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e18183f2-f718-41c1-af9f-902e01b6f655 None None] REQUEST_METHOD: `POST` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:09:13.355361 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e18183f2-f718-41c1-af9f-902e01b6f655 None None] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:09:13.355458 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e18183f2-f718-41c1-af9f-902e01b6f655 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:09:13.427930 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.auth.core [None req-e18183f2-f718-41c1-af9f-902e01b6f655 None None] MFA Rules not processed for user `891005fa5ff74041b75747d16cc66798`. Rule list: `[]` (Enabled: `True`). {{(pid=80378) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 02 10:09:13.429165 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.fernet_utils [None req-e18183f2-f718-41c1-af9f-902e01b6f655 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80378) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 10:09:13.490616 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG dbcounter [-] [80378] Writing DB stats keystone:SELECT=100 {{(pid=80378) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 10:09:13.536945 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 698/1415] 127.0.0.1 () {58 vars in 1013 bytes} [Tue Aug 2 10:09:13 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 183 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 02 10:09:13.587722 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0f3a582-f60c-4800-af2d-a4a7e4f2f09a None None] Authenticating user token {{(pid=80377) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:09:13.589382 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.fernet_utils [None req-b0f3a582-f60c-4800-af2d-a4a7e4f2f09a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80377) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 10:09:13.755250 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0f3a582-f60c-4800-af2d-a4a7e4f2f09a None None] Validating token access rules against request {{(pid=80377) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:09:13.840012 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG dbcounter [-] [80377] Writing DB stats keystone:SELECT=100 {{(pid=80377) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 10:09:13.884327 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de3663d9-c4bb-4a92-b4db-7c902c8ad126 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88af518196674fe593ad88d30111fb7d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f1b38ebb804c41c69772c998aae72c41', 'project_domain_id': 'default', 'roles': ['admin', 'ResellerAdmin', 'reader', 'service', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80377) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:09:13.884953 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de3663d9-c4bb-4a92-b4db-7c902c8ad126 service nova] REQUEST_METHOD: `GET` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:09:13.885235 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de3663d9-c4bb-4a92-b4db-7c902c8ad126 service nova] SCRIPT_NAME: `/identity` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:09:13.885514 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de3663d9-c4bb-4a92-b4db-7c902c8ad126 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:09:13.887184 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.fernet_utils [None req-de3663d9-c4bb-4a92-b4db-7c902c8ad126 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80377) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 10:09:13.985507 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de3663d9-c4bb-4a92-b4db-7c902c8ad126 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:09:13.987201 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de3663d9-c4bb-4a92-b4db-7c902c8ad126 service nova] RBAC: Authorization granted {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:09:14.008897 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: [pid: 80377|app: 0|req: 718/1416] 10.222.0.82 () {60 vars in 1365 bytes} [Tue Aug 2 10:09:13 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 422 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 02 10:09:14.942942 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-985f073f-ab6f-4572-a003-4cf405f7d204 None None] REQUEST_METHOD: `GET` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:09:14.943493 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-985f073f-ab6f-4572-a003-4cf405f7d204 None None] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:09:14.943493 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-985f073f-ab6f-4572-a003-4cf405f7d204 None None] PATH_INFO: `/` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:09:14.945218 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 699/1417] 127.0.0.1 () {54 vars in 901 bytes} [Tue Aug 2 10:09:14 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 02 10:09:14.950889 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b2bfda2-4d42-4f80-b8f5-7fb8b26b7ee0 None None] REQUEST_METHOD: `POST` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:09:14.951447 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b2bfda2-4d42-4f80-b8f5-7fb8b26b7ee0 None None] SCRIPT_NAME: `/identity` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:09:14.951842 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b2bfda2-4d42-4f80-b8f5-7fb8b26b7ee0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:09:15.040201 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.auth.core [None req-5b2bfda2-4d42-4f80-b8f5-7fb8b26b7ee0 None None] MFA Rules not processed for user `891005fa5ff74041b75747d16cc66798`. Rule list: `[]` (Enabled: `True`). {{(pid=80377) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 02 10:09:15.041458 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.fernet_utils [None req-5b2bfda2-4d42-4f80-b8f5-7fb8b26b7ee0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80377) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 10:09:15.074153 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG dbcounter [-] [80377] Writing DB stats keystone:SELECT=100 {{(pid=80377) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 10:09:15.154962 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: [pid: 80377|app: 0|req: 719/1418] 127.0.0.1 () {58 vars in 998 bytes} [Tue Aug 2 10:09:14 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 206 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 02 10:09:15.313004 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-985f073f-ab6f-4572-a003-4cf405f7d204 None None] Authenticating user token {{(pid=80378) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:09:15.314812 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.fernet_utils [None req-985f073f-ab6f-4572-a003-4cf405f7d204 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80378) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 10:09:15.481082 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-985f073f-ab6f-4572-a003-4cf405f7d204 None None] Validating token access rules against request {{(pid=80378) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:09:15.522984 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG dbcounter [-] [80378] Writing DB stats keystone:SELECT=100 {{(pid=80378) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 10:09:15.607642 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-568fb1bc-db38-4409-9a71-9af16414abca service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88af518196674fe593ad88d30111fb7d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f1b38ebb804c41c69772c998aae72c41', 'project_domain_id': 'default', 'roles': ['admin', 'ResellerAdmin', 'reader', 'service', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80378) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:09:15.608160 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-568fb1bc-db38-4409-9a71-9af16414abca service nova] REQUEST_METHOD: `GET` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:09:15.608442 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-568fb1bc-db38-4409-9a71-9af16414abca service nova] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:09:15.609245 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-568fb1bc-db38-4409-9a71-9af16414abca service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:09:15.610951 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.fernet_utils [None req-568fb1bc-db38-4409-9a71-9af16414abca service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80378) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 10:09:15.712118 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-568fb1bc-db38-4409-9a71-9af16414abca service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:09:15.713761 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-568fb1bc-db38-4409-9a71-9af16414abca service nova] RBAC: Authorization granted {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:09:15.732135 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 700/1419] 10.222.0.82 () {60 vars in 1365 bytes} [Tue Aug 2 10:09:15 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 420 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 02 10:09:16.657723 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2517c33-a881-46d9-970f-cbd00252bcd7 None None] REQUEST_METHOD: `GET` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:09:16.658399 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2517c33-a881-46d9-970f-cbd00252bcd7 None None] SCRIPT_NAME: `/identity` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:09:16.658399 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2517c33-a881-46d9-970f-cbd00252bcd7 None None] PATH_INFO: `/` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:09:16.659760 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: [pid: 80377|app: 0|req: 720/1420] 127.0.0.1 () {54 vars in 901 bytes} [Tue Aug 2 10:09:16 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 02 10:09:16.665329 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9321c0cd-9e7c-4ba7-a143-064690d9b385 None None] REQUEST_METHOD: `POST` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:09:16.665864 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9321c0cd-9e7c-4ba7-a143-064690d9b385 None None] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:09:16.665971 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9321c0cd-9e7c-4ba7-a143-064690d9b385 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:09:16.735702 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG dbcounter [-] [80378] Writing DB stats keystone:SELECT=100 {{(pid=80378) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 10:09:16.759297 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.auth.core [None req-9321c0cd-9e7c-4ba7-a143-064690d9b385 None None] MFA Rules not processed for user `891005fa5ff74041b75747d16cc66798`. Rule list: `[]` (Enabled: `True`). {{(pid=80378) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 02 10:09:16.760506 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.fernet_utils [None req-9321c0cd-9e7c-4ba7-a143-064690d9b385 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80378) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 10:09:16.870187 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 701/1421] 127.0.0.1 () {58 vars in 998 bytes} [Tue Aug 2 10:09:16 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 206 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 02 10:09:17.027180 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2517c33-a881-46d9-970f-cbd00252bcd7 None None] Authenticating user token {{(pid=80377) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:09:17.031594 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.fernet_utils [None req-e2517c33-a881-46d9-970f-cbd00252bcd7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80377) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 10:09:17.191834 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2517c33-a881-46d9-970f-cbd00252bcd7 None None] Validating token access rules against request {{(pid=80377) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:09:17.205557 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG dbcounter [-] [80377] Writing DB stats keystone:SELECT=100 {{(pid=80377) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 10:09:17.310278 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2a0e8d0-ce49-4025-9120-5c09640cce26 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88af518196674fe593ad88d30111fb7d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f1b38ebb804c41c69772c998aae72c41', 'project_domain_id': 'default', 'roles': ['admin', 'ResellerAdmin', 'reader', 'service', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80377) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:09:17.310990 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2a0e8d0-ce49-4025-9120-5c09640cce26 service nova] REQUEST_METHOD: `GET` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:09:17.311278 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2a0e8d0-ce49-4025-9120-5c09640cce26 service nova] SCRIPT_NAME: `/identity` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:09:17.311550 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2a0e8d0-ce49-4025-9120-5c09640cce26 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:09:17.313739 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.fernet_utils [None req-b2a0e8d0-ce49-4025-9120-5c09640cce26 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80377) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 10:09:17.415852 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2a0e8d0-ce49-4025-9120-5c09640cce26 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:09:17.417497 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2a0e8d0-ce49-4025-9120-5c09640cce26 service nova] RBAC: Authorization granted {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:09:17.438397 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: [pid: 80377|app: 0|req: 721/1422] 10.222.0.82 () {60 vars in 1365 bytes} [Tue Aug 2 10:09:17 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 413 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 02 10:09:18.335887 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c8b5bb4-5e7d-4ea8-8298-c97c5f712d06 None None] REQUEST_METHOD: `GET` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:09:18.336842 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c8b5bb4-5e7d-4ea8-8298-c97c5f712d06 None None] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:09:18.336842 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c8b5bb4-5e7d-4ea8-8298-c97c5f712d06 None None] PATH_INFO: `/` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:09:18.337842 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 702/1423] 127.0.0.1 () {54 vars in 901 bytes} [Tue Aug 2 10:09:18 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 02 10:09:18.343936 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4022ad1c-0324-4f51-8891-1291da91c512 None None] REQUEST_METHOD: `POST` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:09:18.344672 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4022ad1c-0324-4f51-8891-1291da91c512 None None] SCRIPT_NAME: `/identity` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:09:18.344672 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4022ad1c-0324-4f51-8891-1291da91c512 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:09:18.365022 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG dbcounter [-] [80377] Writing DB stats keystone:SELECT=100 {{(pid=80377) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 10:09:18.426966 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.auth.core [None req-4022ad1c-0324-4f51-8891-1291da91c512 None None] MFA Rules not processed for user `891005fa5ff74041b75747d16cc66798`. Rule list: `[]` (Enabled: `True`). {{(pid=80377) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 02 10:09:18.428227 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.fernet_utils [None req-4022ad1c-0324-4f51-8891-1291da91c512 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80377) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 10:09:18.545265 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: [pid: 80377|app: 0|req: 722/1424] 127.0.0.1 () {58 vars in 998 bytes} [Tue Aug 2 10:09:18 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 203 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 02 10:09:18.700423 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c8b5bb4-5e7d-4ea8-8298-c97c5f712d06 None None] Authenticating user token {{(pid=80378) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:09:18.701970 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.fernet_utils [None req-6c8b5bb4-5e7d-4ea8-8298-c97c5f712d06 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80378) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 10:09:18.825573 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG dbcounter [-] [80378] Writing DB stats keystone:SELECT=100 {{(pid=80378) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 10:09:18.867148 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c8b5bb4-5e7d-4ea8-8298-c97c5f712d06 None None] Validating token access rules against request {{(pid=80378) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:09:19.009078 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9bc6d84-3ab9-4917-9451-17dcd75fe1a5 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88af518196674fe593ad88d30111fb7d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f1b38ebb804c41c69772c998aae72c41', 'project_domain_id': 'default', 'roles': ['admin', 'ResellerAdmin', 'reader', 'service', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80378) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:09:19.009710 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9bc6d84-3ab9-4917-9451-17dcd75fe1a5 service nova] REQUEST_METHOD: `GET` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:09:19.010026 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9bc6d84-3ab9-4917-9451-17dcd75fe1a5 service nova] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:09:19.010305 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9bc6d84-3ab9-4917-9451-17dcd75fe1a5 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:09:19.011949 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.fernet_utils [None req-e9bc6d84-3ab9-4917-9451-17dcd75fe1a5 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80378) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 10:09:19.094998 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG dbcounter [-] [80378] Writing DB stats keystone:SELECT=100 {{(pid=80378) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 10:09:19.109761 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9bc6d84-3ab9-4917-9451-17dcd75fe1a5 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:09:19.111405 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9bc6d84-3ab9-4917-9451-17dcd75fe1a5 service nova] RBAC: Authorization granted {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:09:19.130068 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 703/1425] 10.222.0.82 () {60 vars in 1365 bytes} [Tue Aug 2 10:09:18 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 430 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 02 10:09:20.026756 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-707e5da7-4448-4ffe-8617-5a79c5bc0d21 None None] REQUEST_METHOD: `GET` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:09:20.027514 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-707e5da7-4448-4ffe-8617-5a79c5bc0d21 None None] SCRIPT_NAME: `/identity` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:09:20.027514 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-707e5da7-4448-4ffe-8617-5a79c5bc0d21 None None] PATH_INFO: `/` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:09:20.028668 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: [pid: 80377|app: 0|req: 723/1426] 127.0.0.1 () {54 vars in 901 bytes} [Tue Aug 2 10:09:20 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 02 10:09:20.033946 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d9711db-2831-4dd2-a0b1-b67509398b64 None None] REQUEST_METHOD: `POST` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:09:20.034461 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d9711db-2831-4dd2-a0b1-b67509398b64 None None] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:09:20.034591 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d9711db-2831-4dd2-a0b1-b67509398b64 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:09:20.117074 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.auth.core [None req-7d9711db-2831-4dd2-a0b1-b67509398b64 None None] MFA Rules not processed for user `891005fa5ff74041b75747d16cc66798`. Rule list: `[]` (Enabled: `True`). {{(pid=80378) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 02 10:09:20.118263 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.fernet_utils [None req-7d9711db-2831-4dd2-a0b1-b67509398b64 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80378) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 10:09:20.222262 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 704/1427] 127.0.0.1 () {58 vars in 998 bytes} [Tue Aug 2 10:09:20 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 189 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 02 10:09:20.375294 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-707e5da7-4448-4ffe-8617-5a79c5bc0d21 None None] Authenticating user token {{(pid=80377) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:09:20.376886 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.fernet_utils [None req-707e5da7-4448-4ffe-8617-5a79c5bc0d21 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80377) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 10:09:20.464723 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG dbcounter [-] [80377] Writing DB stats keystone:SELECT=100 {{(pid=80377) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 10:09:20.528438 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-707e5da7-4448-4ffe-8617-5a79c5bc0d21 None None] Validating token access rules against request {{(pid=80377) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:09:20.664228 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fcde5531-25af-4923-b602-56ee47603b6b service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88af518196674fe593ad88d30111fb7d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f1b38ebb804c41c69772c998aae72c41', 'project_domain_id': 'default', 'roles': ['admin', 'ResellerAdmin', 'reader', 'service', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80377) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:09:20.664768 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcde5531-25af-4923-b602-56ee47603b6b service nova] REQUEST_METHOD: `GET` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:09:20.665047 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcde5531-25af-4923-b602-56ee47603b6b service nova] SCRIPT_NAME: `/identity` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:09:20.665321 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcde5531-25af-4923-b602-56ee47603b6b service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:09:20.667016 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.fernet_utils [None req-fcde5531-25af-4923-b602-56ee47603b6b service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80377) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 10:09:20.721694 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG dbcounter [-] [80377] Writing DB stats keystone:SELECT=100 {{(pid=80377) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 10:09:20.770448 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fcde5531-25af-4923-b602-56ee47603b6b service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:09:20.772155 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fcde5531-25af-4923-b602-56ee47603b6b service nova] RBAC: Authorization granted {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:09:20.792006 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: [pid: 80377|app: 0|req: 724/1428] 10.222.0.82 () {60 vars in 1365 bytes} [Tue Aug 2 10:09:20 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 417 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 02 10:09:21.637673 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8eafcb5-2895-4a6f-b9c4-5bd1be95e9e3 None None] REQUEST_METHOD: `GET` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:09:21.637673 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8eafcb5-2895-4a6f-b9c4-5bd1be95e9e3 None None] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:09:21.639081 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8eafcb5-2895-4a6f-b9c4-5bd1be95e9e3 None None] PATH_INFO: `/` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:09:21.640449 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 705/1429] 127.0.0.1 () {54 vars in 901 bytes} [Tue Aug 2 10:09:21 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 02 10:09:21.646365 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acbdac1a-3389-4641-81f2-40622da0f8e8 None None] REQUEST_METHOD: `POST` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:09:21.647113 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acbdac1a-3389-4641-81f2-40622da0f8e8 None None] SCRIPT_NAME: `/identity` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:09:21.647314 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acbdac1a-3389-4641-81f2-40622da0f8e8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:09:21.727563 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.auth.core [None req-acbdac1a-3389-4641-81f2-40622da0f8e8 None None] MFA Rules not processed for user `891005fa5ff74041b75747d16cc66798`. Rule list: `[]` (Enabled: `True`). {{(pid=80377) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 02 10:09:21.728820 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.fernet_utils [None req-acbdac1a-3389-4641-81f2-40622da0f8e8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80377) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 10:09:21.833451 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: [pid: 80377|app: 0|req: 725/1430] 127.0.0.1 () {58 vars in 998 bytes} [Tue Aug 2 10:09:21 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 188 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 02 10:09:22.004606 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8eafcb5-2895-4a6f-b9c4-5bd1be95e9e3 None None] Authenticating user token {{(pid=80378) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:09:22.006769 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.fernet_utils [None req-f8eafcb5-2895-4a6f-b9c4-5bd1be95e9e3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80378) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 10:09:22.077869 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG dbcounter [-] [80378] Writing DB stats keystone:SELECT=100 {{(pid=80378) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 10:09:22.178215 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8eafcb5-2895-4a6f-b9c4-5bd1be95e9e3 None None] Validating token access rules against request {{(pid=80378) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:09:22.294100 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-661957d7-8344-4b9a-b7d0-f28ac1610336 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88af518196674fe593ad88d30111fb7d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f1b38ebb804c41c69772c998aae72c41', 'project_domain_id': 'default', 'roles': ['admin', 'ResellerAdmin', 'reader', 'service', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80378) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:09:22.294762 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-661957d7-8344-4b9a-b7d0-f28ac1610336 service nova] REQUEST_METHOD: `GET` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:09:22.295046 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-661957d7-8344-4b9a-b7d0-f28ac1610336 service nova] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:09:22.295331 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-661957d7-8344-4b9a-b7d0-f28ac1610336 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:09:22.297004 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.fernet_utils [None req-661957d7-8344-4b9a-b7d0-f28ac1610336 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80378) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 10:09:22.304285 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG dbcounter [-] [80378] Writing DB stats keystone:SELECT=100 {{(pid=80378) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 10:09:22.401689 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-661957d7-8344-4b9a-b7d0-f28ac1610336 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:09:22.403446 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-661957d7-8344-4b9a-b7d0-f28ac1610336 service nova] RBAC: Authorization granted {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:09:22.423489 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 706/1431] 10.222.0.82 () {60 vars in 1365 bytes} [Tue Aug 2 10:09:22 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 420 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 02 10:09:23.248597 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cfd83a6-0799-4765-ba20-b463ebb67ac4 None None] REQUEST_METHOD: `GET` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:09:23.249185 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cfd83a6-0799-4765-ba20-b463ebb67ac4 None None] SCRIPT_NAME: `/identity` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:09:23.249185 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cfd83a6-0799-4765-ba20-b463ebb67ac4 None None] PATH_INFO: `/` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:09:23.250836 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: [pid: 80377|app: 0|req: 726/1432] 127.0.0.1 () {54 vars in 901 bytes} [Tue Aug 2 10:09:23 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 02 10:09:23.255951 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4feeb18-9dbf-4b9f-a49e-a1a1386d70c4 None None] REQUEST_METHOD: `POST` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:09:23.256519 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4feeb18-9dbf-4b9f-a49e-a1a1386d70c4 None None] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:09:23.256630 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4feeb18-9dbf-4b9f-a49e-a1a1386d70c4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:09:23.339587 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.auth.core [None req-d4feeb18-9dbf-4b9f-a49e-a1a1386d70c4 None None] MFA Rules not processed for user `891005fa5ff74041b75747d16cc66798`. Rule list: `[]` (Enabled: `True`). {{(pid=80378) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 02 10:09:23.340854 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.fernet_utils [None req-d4feeb18-9dbf-4b9f-a49e-a1a1386d70c4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80378) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 10:09:23.414751 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG dbcounter [-] [80378] Writing DB stats keystone:SELECT=100 {{(pid=80378) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 10:09:23.446723 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 707/1433] 127.0.0.1 () {58 vars in 998 bytes} [Tue Aug 2 10:09:23 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 192 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 02 10:09:23.602269 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1cfd83a6-0799-4765-ba20-b463ebb67ac4 None None] Authenticating user token {{(pid=80377) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:09:23.603975 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.fernet_utils [None req-1cfd83a6-0799-4765-ba20-b463ebb67ac4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80377) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 10:09:23.623375 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG dbcounter [-] [80377] Writing DB stats keystone:SELECT=100 {{(pid=80377) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 10:09:23.779020 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1cfd83a6-0799-4765-ba20-b463ebb67ac4 None None] Validating token access rules against request {{(pid=80377) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:09:23.875558 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG dbcounter [-] [80377] Writing DB stats keystone:SELECT=100 {{(pid=80377) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 10:09:23.902327 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7a4c2f1-3c33-44ce-a6e1-42e76cfefb6e service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88af518196674fe593ad88d30111fb7d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f1b38ebb804c41c69772c998aae72c41', 'project_domain_id': 'default', 'roles': ['admin', 'ResellerAdmin', 'reader', 'service', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80377) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:09:23.902955 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7a4c2f1-3c33-44ce-a6e1-42e76cfefb6e service nova] REQUEST_METHOD: `GET` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:09:23.903238 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7a4c2f1-3c33-44ce-a6e1-42e76cfefb6e service nova] SCRIPT_NAME: `/identity` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:09:23.903515 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7a4c2f1-3c33-44ce-a6e1-42e76cfefb6e service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:09:23.905172 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.fernet_utils [None req-b7a4c2f1-3c33-44ce-a6e1-42e76cfefb6e service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80377) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 10:09:23.997074 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7a4c2f1-3c33-44ce-a6e1-42e76cfefb6e service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:09:23.998717 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7a4c2f1-3c33-44ce-a6e1-42e76cfefb6e service nova] RBAC: Authorization granted {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:09:24.019605 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: [pid: 80377|app: 0|req: 727/1434] 10.222.0.82 () {60 vars in 1365 bytes} [Tue Aug 2 10:09:23 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 418 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 02 10:09:24.841251 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5696e4a4-6763-464c-805f-9aef1753d859 None None] REQUEST_METHOD: `GET` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:09:24.841952 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5696e4a4-6763-464c-805f-9aef1753d859 None None] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:09:24.841952 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5696e4a4-6763-464c-805f-9aef1753d859 None None] PATH_INFO: `/` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:09:24.843845 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 708/1435] 127.0.0.1 () {54 vars in 901 bytes} [Tue Aug 2 10:09:24 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 02 10:09:24.849149 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45f32b8a-a804-46eb-bd8d-a4d39e8522eb None None] REQUEST_METHOD: `POST` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:09:24.849822 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45f32b8a-a804-46eb-bd8d-a4d39e8522eb None None] SCRIPT_NAME: `/identity` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:09:24.849822 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45f32b8a-a804-46eb-bd8d-a4d39e8522eb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:09:24.932210 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.auth.core [None req-45f32b8a-a804-46eb-bd8d-a4d39e8522eb None None] MFA Rules not processed for user `891005fa5ff74041b75747d16cc66798`. Rule list: `[]` (Enabled: `True`). {{(pid=80377) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 02 10:09:24.933464 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.fernet_utils [None req-45f32b8a-a804-46eb-bd8d-a4d39e8522eb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80377) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 10:09:24.981942 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG dbcounter [-] [80377] Writing DB stats keystone:SELECT=100 {{(pid=80377) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 10:09:25.040642 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: [pid: 80377|app: 0|req: 728/1436] 127.0.0.1 () {58 vars in 998 bytes} [Tue Aug 2 10:09:24 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 192 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 02 10:09:25.196740 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5696e4a4-6763-464c-805f-9aef1753d859 None None] Authenticating user token {{(pid=80378) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:09:25.198202 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.fernet_utils [None req-5696e4a4-6763-464c-805f-9aef1753d859 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80378) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 10:09:25.357940 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5696e4a4-6763-464c-805f-9aef1753d859 None None] Validating token access rules against request {{(pid=80378) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:09:25.414679 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG dbcounter [-] [80378] Writing DB stats keystone:SELECT=100 {{(pid=80378) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 10:09:25.475250 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3c17d68-680c-4205-bf61-06e6394b892f service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88af518196674fe593ad88d30111fb7d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f1b38ebb804c41c69772c998aae72c41', 'project_domain_id': 'default', 'roles': ['admin', 'ResellerAdmin', 'reader', 'service', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80378) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:09:25.475883 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3c17d68-680c-4205-bf61-06e6394b892f service nova] REQUEST_METHOD: `GET` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:09:25.476169 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3c17d68-680c-4205-bf61-06e6394b892f service nova] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:09:25.476448 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3c17d68-680c-4205-bf61-06e6394b892f service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:09:25.478148 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.fernet_utils [None req-f3c17d68-680c-4205-bf61-06e6394b892f service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80378) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 10:09:25.569913 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3c17d68-680c-4205-bf61-06e6394b892f service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:09:25.571517 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3c17d68-680c-4205-bf61-06e6394b892f service nova] RBAC: Authorization granted {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:09:25.590394 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 709/1437] 10.222.0.82 () {60 vars in 1365 bytes} [Tue Aug 2 10:09:25 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 394 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 02 10:09:26.443607 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6821687b-cfc3-4ea0-8084-7a7e329672ca None None] REQUEST_METHOD: `GET` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:09:26.444186 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6821687b-cfc3-4ea0-8084-7a7e329672ca None None] SCRIPT_NAME: `/identity` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:09:26.444186 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6821687b-cfc3-4ea0-8084-7a7e329672ca None None] PATH_INFO: `/` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:09:26.445534 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: [pid: 80377|app: 0|req: 729/1438] 127.0.0.1 () {54 vars in 901 bytes} [Tue Aug 2 10:09:26 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 02 10:09:26.451411 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83cead7e-925b-41fc-ba2a-ccd0f176e3c9 None None] REQUEST_METHOD: `POST` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:09:26.451909 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83cead7e-925b-41fc-ba2a-ccd0f176e3c9 None None] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:09:26.452024 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83cead7e-925b-41fc-ba2a-ccd0f176e3c9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:09:26.545112 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.auth.core [None req-83cead7e-925b-41fc-ba2a-ccd0f176e3c9 None None] MFA Rules not processed for user `891005fa5ff74041b75747d16cc66798`. Rule list: `[]` (Enabled: `True`). {{(pid=80378) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 02 10:09:26.546968 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.fernet_utils [None req-83cead7e-925b-41fc-ba2a-ccd0f176e3c9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80378) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 10:09:26.549170 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG dbcounter [-] [80378] Writing DB stats keystone:SELECT=100 {{(pid=80378) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 10:09:26.657517 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 710/1439] 127.0.0.1 () {58 vars in 998 bytes} [Tue Aug 2 10:09:26 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 207 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 02 10:09:26.812064 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6821687b-cfc3-4ea0-8084-7a7e329672ca None None] Authenticating user token {{(pid=80377) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:09:26.814019 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.fernet_utils [None req-6821687b-cfc3-4ea0-8084-7a7e329672ca None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80377) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 10:09:26.976057 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6821687b-cfc3-4ea0-8084-7a7e329672ca None None] Validating token access rules against request {{(pid=80377) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:09:27.008385 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG dbcounter [-] [80377] Writing DB stats keystone:SELECT=100 {{(pid=80377) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 10:09:27.096614 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-615e9929-3716-4fbe-9397-82d5d6eacd25 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88af518196674fe593ad88d30111fb7d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f1b38ebb804c41c69772c998aae72c41', 'project_domain_id': 'default', 'roles': ['admin', 'ResellerAdmin', 'reader', 'service', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80377) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:09:27.097144 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-615e9929-3716-4fbe-9397-82d5d6eacd25 service nova] REQUEST_METHOD: `GET` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:09:27.097427 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-615e9929-3716-4fbe-9397-82d5d6eacd25 service nova] SCRIPT_NAME: `/identity` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:09:27.097699 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-615e9929-3716-4fbe-9397-82d5d6eacd25 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:09:27.099353 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.fernet_utils [None req-615e9929-3716-4fbe-9397-82d5d6eacd25 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80377) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 10:09:27.193938 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-615e9929-3716-4fbe-9397-82d5d6eacd25 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:09:27.195595 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-615e9929-3716-4fbe-9397-82d5d6eacd25 service nova] RBAC: Authorization granted {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:09:27.214086 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: [pid: 80377|app: 0|req: 730/1440] 10.222.0.82 () {60 vars in 1365 bytes} [Tue Aug 2 10:09:26 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 403 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 02 10:09:29.158384 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e07f7698-64b7-4cf5-a4e1-7d20d5789cb9 None None] REQUEST_METHOD: `GET` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:09:29.158949 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e07f7698-64b7-4cf5-a4e1-7d20d5789cb9 None None] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:09:29.158949 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e07f7698-64b7-4cf5-a4e1-7d20d5789cb9 None None] PATH_INFO: `/` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:09:29.160224 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 711/1441] 127.0.0.1 () {54 vars in 916 bytes} [Tue Aug 2 10:09:29 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 02 10:09:29.166924 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55582340-d8f0-4beb-a62d-311f61601d9a None None] REQUEST_METHOD: `POST` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:09:29.167417 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55582340-d8f0-4beb-a62d-311f61601d9a None None] SCRIPT_NAME: `/identity` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:09:29.167534 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55582340-d8f0-4beb-a62d-311f61601d9a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:09:29.209227 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG dbcounter [-] [80377] Writing DB stats keystone:SELECT=100 {{(pid=80377) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 10:09:29.243433 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.auth.core [None req-55582340-d8f0-4beb-a62d-311f61601d9a None None] MFA Rules not processed for user `891005fa5ff74041b75747d16cc66798`. Rule list: `[]` (Enabled: `True`). {{(pid=80377) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 02 10:09:29.244645 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.fernet_utils [None req-55582340-d8f0-4beb-a62d-311f61601d9a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80377) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 10:09:29.351360 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: [pid: 80377|app: 0|req: 731/1442] 127.0.0.1 () {58 vars in 1013 bytes} [Tue Aug 2 10:09:29 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 186 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 02 10:09:29.391043 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e07f7698-64b7-4cf5-a4e1-7d20d5789cb9 None None] Authenticating user token {{(pid=80378) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:09:29.392607 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.fernet_utils [None req-e07f7698-64b7-4cf5-a4e1-7d20d5789cb9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80378) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 10:09:29.496851 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e07f7698-64b7-4cf5-a4e1-7d20d5789cb9 None None] Validating token access rules against request {{(pid=80378) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:09:29.530075 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG dbcounter [-] [80378] Writing DB stats keystone:SELECT=100 {{(pid=80378) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 10:09:29.578814 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc1cf282-7329-4014-af04-084498611cf6 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '33b87170f0634b67bce60750847b5cbe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f1b38ebb804c41c69772c998aae72c41', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80378) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:09:29.579468 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc1cf282-7329-4014-af04-084498611cf6 service neutron] REQUEST_METHOD: `GET` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:09:29.579748 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc1cf282-7329-4014-af04-084498611cf6 service neutron] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:09:29.580025 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc1cf282-7329-4014-af04-084498611cf6 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:09:29.581718 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.fernet_utils [None req-fc1cf282-7329-4014-af04-084498611cf6 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80378) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 10:09:29.693085 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc1cf282-7329-4014-af04-084498611cf6 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:09:29.694566 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc1cf282-7329-4014-af04-084498611cf6 service neutron] RBAC: Authorization granted {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:09:29.715136 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 712/1443] 10.222.0.82 () {60 vars in 1365 bytes} [Tue Aug 2 10:09:29 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 325 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 02 10:09:32.025451 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b98498c-4a0f-46b4-bc9a-6ca0fce94de8 None None] REQUEST_METHOD: `GET` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:09:32.026071 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b98498c-4a0f-46b4-bc9a-6ca0fce94de8 None None] SCRIPT_NAME: `/identity` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:09:32.026071 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b98498c-4a0f-46b4-bc9a-6ca0fce94de8 None None] PATH_INFO: `/` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:09:32.027366 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: [pid: 80377|app: 0|req: 732/1444] 127.0.0.1 () {54 vars in 916 bytes} [Tue Aug 2 10:09:32 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 02 10:09:32.033241 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3691bd2f-20c3-45e8-abd4-14f0282eacc7 None None] REQUEST_METHOD: `POST` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:09:32.033702 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3691bd2f-20c3-45e8-abd4-14f0282eacc7 None None] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:09:32.033840 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3691bd2f-20c3-45e8-abd4-14f0282eacc7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:09:32.114025 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.auth.core [None req-3691bd2f-20c3-45e8-abd4-14f0282eacc7 None None] MFA Rules not processed for user `891005fa5ff74041b75747d16cc66798`. Rule list: `[]` (Enabled: `True`). {{(pid=80378) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 02 10:09:32.115141 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.fernet_utils [None req-3691bd2f-20c3-45e8-abd4-14f0282eacc7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80378) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 10:09:32.128439 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG dbcounter [-] [80378] Writing DB stats keystone:SELECT=100 {{(pid=80378) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 10:09:32.215737 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 713/1445] 127.0.0.1 () {58 vars in 1013 bytes} [Tue Aug 2 10:09:32 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 183 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 02 10:09:32.255185 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b98498c-4a0f-46b4-bc9a-6ca0fce94de8 None None] Authenticating user token {{(pid=80377) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:09:32.256753 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.fernet_utils [None req-4b98498c-4a0f-46b4-bc9a-6ca0fce94de8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80377) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 10:09:32.388834 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b98498c-4a0f-46b4-bc9a-6ca0fce94de8 None None] Validating token access rules against request {{(pid=80377) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:09:32.400987 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG dbcounter [-] [80377] Writing DB stats keystone:SELECT=100 {{(pid=80377) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 10:09:32.465061 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46d2d949-d41a-4a4e-b1fc-88402fc1cd97 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '33b87170f0634b67bce60750847b5cbe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f1b38ebb804c41c69772c998aae72c41', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80377) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:09:32.465679 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46d2d949-d41a-4a4e-b1fc-88402fc1cd97 service neutron] REQUEST_METHOD: `GET` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:09:32.466075 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46d2d949-d41a-4a4e-b1fc-88402fc1cd97 service neutron] SCRIPT_NAME: `/identity` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:09:32.466353 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46d2d949-d41a-4a4e-b1fc-88402fc1cd97 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:09:32.468099 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.fernet_utils [None req-46d2d949-d41a-4a4e-b1fc-88402fc1cd97 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80377) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 10:09:32.580968 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46d2d949-d41a-4a4e-b1fc-88402fc1cd97 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:09:32.582463 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46d2d949-d41a-4a4e-b1fc-88402fc1cd97 service neutron] RBAC: Authorization granted {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:09:32.600787 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: [pid: 80377|app: 0|req: 733/1446] 10.222.0.82 () {60 vars in 1365 bytes} [Tue Aug 2 10:09:32 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 346 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 02 10:09:34.979806 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1932e6fb-dee3-4b59-ae5a-7fa9d85dfeae None None] REQUEST_METHOD: `GET` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:09:34.980335 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1932e6fb-dee3-4b59-ae5a-7fa9d85dfeae None None] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:09:34.980335 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1932e6fb-dee3-4b59-ae5a-7fa9d85dfeae None None] PATH_INFO: `/` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:09:34.981840 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 714/1447] 127.0.0.1 () {54 vars in 916 bytes} [Tue Aug 2 10:09:34 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 02 10:09:34.987273 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99048e25-908a-4634-9d7b-1b9da9a35fcd None None] REQUEST_METHOD: `POST` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:09:34.987734 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99048e25-908a-4634-9d7b-1b9da9a35fcd None None] SCRIPT_NAME: `/identity` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:09:34.987830 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99048e25-908a-4634-9d7b-1b9da9a35fcd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:09:35.042668 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG dbcounter [-] [80377] Writing DB stats keystone:SELECT=100 {{(pid=80377) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 10:09:35.062719 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.auth.core [None req-99048e25-908a-4634-9d7b-1b9da9a35fcd None None] MFA Rules not processed for user `891005fa5ff74041b75747d16cc66798`. Rule list: `[]` (Enabled: `True`). {{(pid=80377) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 02 10:09:35.063945 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.fernet_utils [None req-99048e25-908a-4634-9d7b-1b9da9a35fcd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80377) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 10:09:35.172332 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: [pid: 80377|app: 0|req: 734/1448] 127.0.0.1 () {58 vars in 1013 bytes} [Tue Aug 2 10:09:34 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 186 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 02 10:09:35.213357 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1932e6fb-dee3-4b59-ae5a-7fa9d85dfeae None None] Authenticating user token {{(pid=80378) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:09:35.215167 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.fernet_utils [None req-1932e6fb-dee3-4b59-ae5a-7fa9d85dfeae None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80378) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 10:09:35.446960 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1932e6fb-dee3-4b59-ae5a-7fa9d85dfeae None None] Validating token access rules against request {{(pid=80378) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:09:35.488548 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG dbcounter [-] [80378] Writing DB stats keystone:SELECT=100 {{(pid=80378) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 10:09:35.525354 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aff23ff6-9bcb-4c5d-afcc-8668a6090ae7 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '33b87170f0634b67bce60750847b5cbe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f1b38ebb804c41c69772c998aae72c41', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80378) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:09:35.525999 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aff23ff6-9bcb-4c5d-afcc-8668a6090ae7 service neutron] REQUEST_METHOD: `GET` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:09:35.526278 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aff23ff6-9bcb-4c5d-afcc-8668a6090ae7 service neutron] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:09:35.526547 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aff23ff6-9bcb-4c5d-afcc-8668a6090ae7 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:09:35.528159 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.fernet_utils [None req-aff23ff6-9bcb-4c5d-afcc-8668a6090ae7 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80378) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 10:09:35.662175 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aff23ff6-9bcb-4c5d-afcc-8668a6090ae7 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:09:35.663617 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aff23ff6-9bcb-4c5d-afcc-8668a6090ae7 service neutron] RBAC: Authorization granted {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:09:35.683824 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 715/1449] 10.222.0.82 () {60 vars in 1365 bytes} [Tue Aug 2 10:09:35 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 471 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 02 10:09:37.907999 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cd6e16d-df93-4307-8984-1fd7c831c162 None None] REQUEST_METHOD: `GET` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:09:37.908741 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cd6e16d-df93-4307-8984-1fd7c831c162 None None] SCRIPT_NAME: `/identity` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:09:37.908741 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cd6e16d-df93-4307-8984-1fd7c831c162 None None] PATH_INFO: `/` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:09:37.911004 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: [pid: 80377|app: 0|req: 735/1450] 127.0.0.1 () {54 vars in 916 bytes} [Tue Aug 2 10:09:37 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 02 10:09:37.915513 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c4626e9-7ae4-48f0-aad4-1246f0264615 None None] REQUEST_METHOD: `POST` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:09:37.915997 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c4626e9-7ae4-48f0-aad4-1246f0264615 None None] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:09:37.916111 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c4626e9-7ae4-48f0-aad4-1246f0264615 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:09:37.988201 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.auth.core [None req-2c4626e9-7ae4-48f0-aad4-1246f0264615 None None] MFA Rules not processed for user `891005fa5ff74041b75747d16cc66798`. Rule list: `[]` (Enabled: `True`). {{(pid=80378) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 02 10:09:37.989426 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.fernet_utils [None req-2c4626e9-7ae4-48f0-aad4-1246f0264615 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80378) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 10:09:38.010163 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG dbcounter [-] [80378] Writing DB stats keystone:SELECT=100 {{(pid=80378) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 10:09:38.088535 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 716/1451] 127.0.0.1 () {58 vars in 1013 bytes} [Tue Aug 2 10:09:37 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 174 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 02 10:09:38.127351 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1cd6e16d-df93-4307-8984-1fd7c831c162 None None] Authenticating user token {{(pid=80377) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:09:38.128879 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.fernet_utils [None req-1cd6e16d-df93-4307-8984-1fd7c831c162 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80377) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 10:09:38.240423 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1cd6e16d-df93-4307-8984-1fd7c831c162 None None] Validating token access rules against request {{(pid=80377) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:09:38.261514 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG dbcounter [-] [80377] Writing DB stats keystone:SELECT=100 {{(pid=80377) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 10:09:38.323898 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c11bf772-8b9b-41b8-8822-d2502d903cc2 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '33b87170f0634b67bce60750847b5cbe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f1b38ebb804c41c69772c998aae72c41', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80377) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:09:38.324788 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c11bf772-8b9b-41b8-8822-d2502d903cc2 service neutron] REQUEST_METHOD: `GET` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:09:38.325084 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c11bf772-8b9b-41b8-8822-d2502d903cc2 service neutron] SCRIPT_NAME: `/identity` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:09:38.325510 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c11bf772-8b9b-41b8-8822-d2502d903cc2 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:09:38.327929 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.fernet_utils [None req-c11bf772-8b9b-41b8-8822-d2502d903cc2 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80377) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 10:09:38.445848 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c11bf772-8b9b-41b8-8822-d2502d903cc2 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:09:38.447215 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c11bf772-8b9b-41b8-8822-d2502d903cc2 service neutron] RBAC: Authorization granted {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:09:38.465412 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: [pid: 80377|app: 0|req: 736/1452] 10.222.0.82 () {60 vars in 1365 bytes} [Tue Aug 2 10:09:38 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 339 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 02 10:09:40.579104 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdf8aed7-8ac0-46fd-8cfd-39d2f2cb6367 None None] REQUEST_METHOD: `GET` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:09:40.579676 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdf8aed7-8ac0-46fd-8cfd-39d2f2cb6367 None None] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:09:40.579676 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdf8aed7-8ac0-46fd-8cfd-39d2f2cb6367 None None] PATH_INFO: `/` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:09:40.581028 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 717/1453] 127.0.0.1 () {54 vars in 916 bytes} [Tue Aug 2 10:09:40 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 02 10:09:40.586214 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce78fcfe-a162-4be4-8158-49899ed5ce32 None None] REQUEST_METHOD: `POST` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:09:40.586654 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce78fcfe-a162-4be4-8158-49899ed5ce32 None None] SCRIPT_NAME: `/identity` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:09:40.586772 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce78fcfe-a162-4be4-8158-49899ed5ce32 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:09:40.651732 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG dbcounter [-] [80377] Writing DB stats keystone:SELECT=100 {{(pid=80377) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 10:09:40.655489 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.auth.core [None req-ce78fcfe-a162-4be4-8158-49899ed5ce32 None None] MFA Rules not processed for user `891005fa5ff74041b75747d16cc66798`. Rule list: `[]` (Enabled: `True`). {{(pid=80377) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 02 10:09:40.657995 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.fernet_utils [None req-ce78fcfe-a162-4be4-8158-49899ed5ce32 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80377) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 10:09:40.758496 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: [pid: 80377|app: 0|req: 737/1454] 127.0.0.1 () {58 vars in 1013 bytes} [Tue Aug 2 10:09:40 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 173 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 02 10:09:40.797907 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdf8aed7-8ac0-46fd-8cfd-39d2f2cb6367 None None] Authenticating user token {{(pid=80378) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:09:40.799659 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.fernet_utils [None req-cdf8aed7-8ac0-46fd-8cfd-39d2f2cb6367 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80378) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 10:09:40.913914 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdf8aed7-8ac0-46fd-8cfd-39d2f2cb6367 None None] Validating token access rules against request {{(pid=80378) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:09:40.967701 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG dbcounter [-] [80378] Writing DB stats keystone:SELECT=100 {{(pid=80378) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 10:09:40.994563 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a1bc378-5453-4810-8acb-bd762a271c14 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '33b87170f0634b67bce60750847b5cbe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f1b38ebb804c41c69772c998aae72c41', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80378) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:09:40.995194 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a1bc378-5453-4810-8acb-bd762a271c14 service neutron] REQUEST_METHOD: `GET` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:09:40.995472 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a1bc378-5453-4810-8acb-bd762a271c14 service neutron] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:09:40.995770 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a1bc378-5453-4810-8acb-bd762a271c14 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:09:40.997436 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.fernet_utils [None req-9a1bc378-5453-4810-8acb-bd762a271c14 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80378) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 10:09:41.119388 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a1bc378-5453-4810-8acb-bd762a271c14 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:09:41.120887 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a1bc378-5453-4810-8acb-bd762a271c14 service neutron] RBAC: Authorization granted {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:09:41.140110 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 718/1455] 10.222.0.82 () {60 vars in 1365 bytes} [Tue Aug 2 10:09:40 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 343 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 02 10:09:43.288687 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96c1f922-f5ec-4c0d-a92a-b141f4f231d0 None None] REQUEST_METHOD: `GET` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:09:43.289227 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96c1f922-f5ec-4c0d-a92a-b141f4f231d0 None None] SCRIPT_NAME: `/identity` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:09:43.289227 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96c1f922-f5ec-4c0d-a92a-b141f4f231d0 None None] PATH_INFO: `/` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:09:43.290719 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: [pid: 80377|app: 0|req: 738/1456] 127.0.0.1 () {54 vars in 916 bytes} [Tue Aug 2 10:09:43 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 02 10:09:43.296222 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-618da5dc-bdda-44b9-ab26-a35d7648ea1b None None] REQUEST_METHOD: `POST` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:09:43.296611 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-618da5dc-bdda-44b9-ab26-a35d7648ea1b None None] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:09:43.296776 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-618da5dc-bdda-44b9-ab26-a35d7648ea1b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:09:43.362828 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.auth.core [None req-618da5dc-bdda-44b9-ab26-a35d7648ea1b None None] MFA Rules not processed for user `891005fa5ff74041b75747d16cc66798`. Rule list: `[]` (Enabled: `True`). {{(pid=80378) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 02 10:09:43.364047 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.fernet_utils [None req-618da5dc-bdda-44b9-ab26-a35d7648ea1b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80378) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 10:09:43.393046 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG dbcounter [-] [80378] Writing DB stats keystone:SELECT=100 {{(pid=80378) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 10:09:43.461956 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 719/1457] 127.0.0.1 () {58 vars in 1013 bytes} [Tue Aug 2 10:09:43 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 167 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 02 10:09:43.501278 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96c1f922-f5ec-4c0d-a92a-b141f4f231d0 None None] Authenticating user token {{(pid=80377) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:09:43.502865 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.fernet_utils [None req-96c1f922-f5ec-4c0d-a92a-b141f4f231d0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80377) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 10:09:43.610075 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96c1f922-f5ec-4c0d-a92a-b141f4f231d0 None None] Validating token access rules against request {{(pid=80377) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:09:43.639649 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG dbcounter [-] [80377] Writing DB stats keystone:SELECT=100 {{(pid=80377) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 10:09:43.695260 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d343092c-782a-47c6-b12a-385f69483664 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '33b87170f0634b67bce60750847b5cbe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f1b38ebb804c41c69772c998aae72c41', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80377) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:09:43.695956 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d343092c-782a-47c6-b12a-385f69483664 service neutron] REQUEST_METHOD: `GET` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:09:43.696237 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d343092c-782a-47c6-b12a-385f69483664 service neutron] SCRIPT_NAME: `/identity` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:09:43.696513 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d343092c-782a-47c6-b12a-385f69483664 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:09:43.698204 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.fernet_utils [None req-d343092c-782a-47c6-b12a-385f69483664 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80377) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 10:09:43.809449 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d343092c-782a-47c6-b12a-385f69483664 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:09:43.811481 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d343092c-782a-47c6-b12a-385f69483664 service neutron] RBAC: Authorization granted {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:09:43.831750 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: [pid: 80377|app: 0|req: 739/1458] 10.222.0.82 () {60 vars in 1365 bytes} [Tue Aug 2 10:09:43 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 331 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 02 10:09:45.971276 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb8c09c4-dbd3-4696-bb3d-2c16d5e6f95f None None] REQUEST_METHOD: `GET` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:09:45.971869 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb8c09c4-dbd3-4696-bb3d-2c16d5e6f95f None None] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:09:45.971980 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb8c09c4-dbd3-4696-bb3d-2c16d5e6f95f None None] PATH_INFO: `/` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:09:45.973367 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 720/1459] 127.0.0.1 () {54 vars in 916 bytes} [Tue Aug 2 10:09:45 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 02 10:09:45.980338 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f6a62c3-1741-4ce3-9909-6a14d6c363e6 None None] REQUEST_METHOD: `POST` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:09:45.980848 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f6a62c3-1741-4ce3-9909-6a14d6c363e6 None None] SCRIPT_NAME: `/identity` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:09:45.981216 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f6a62c3-1741-4ce3-9909-6a14d6c363e6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:09:46.050444 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.auth.core [None req-4f6a62c3-1741-4ce3-9909-6a14d6c363e6 None None] MFA Rules not processed for user `891005fa5ff74041b75747d16cc66798`. Rule list: `[]` (Enabled: `True`). {{(pid=80377) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 02 10:09:46.051653 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.fernet_utils [None req-4f6a62c3-1741-4ce3-9909-6a14d6c363e6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80377) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 10:09:46.060170 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG dbcounter [-] [80377] Writing DB stats keystone:SELECT=100 {{(pid=80377) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 10:09:46.152215 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: [pid: 80377|app: 0|req: 740/1460] 127.0.0.1 () {58 vars in 1013 bytes} [Tue Aug 2 10:09:45 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 173 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 02 10:09:46.190129 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb8c09c4-dbd3-4696-bb3d-2c16d5e6f95f None None] Authenticating user token {{(pid=80378) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:09:46.191700 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.fernet_utils [None req-cb8c09c4-dbd3-4696-bb3d-2c16d5e6f95f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80378) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 10:09:46.296693 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb8c09c4-dbd3-4696-bb3d-2c16d5e6f95f None None] Validating token access rules against request {{(pid=80378) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:09:46.352314 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG dbcounter [-] [80378] Writing DB stats keystone:SELECT=100 {{(pid=80378) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 10:09:46.372569 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b0513e2-5b1a-42d0-92f1-b0f7ca902687 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '33b87170f0634b67bce60750847b5cbe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f1b38ebb804c41c69772c998aae72c41', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80378) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:09:46.373891 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b0513e2-5b1a-42d0-92f1-b0f7ca902687 service neutron] REQUEST_METHOD: `GET` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:09:46.374171 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b0513e2-5b1a-42d0-92f1-b0f7ca902687 service neutron] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:09:46.374447 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b0513e2-5b1a-42d0-92f1-b0f7ca902687 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:09:46.376123 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.fernet_utils [None req-3b0513e2-5b1a-42d0-92f1-b0f7ca902687 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80378) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 10:09:46.489031 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b0513e2-5b1a-42d0-92f1-b0f7ca902687 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:09:46.490514 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b0513e2-5b1a-42d0-92f1-b0f7ca902687 service neutron] RBAC: Authorization granted {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:09:46.508993 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 721/1461] 10.222.0.82 () {60 vars in 1365 bytes} [Tue Aug 2 10:09:46 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 319 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 02 10:09:48.606276 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-709cb992-3e67-49ff-8cdd-c035d005a812 None None] REQUEST_METHOD: `GET` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:09:48.606922 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-709cb992-3e67-49ff-8cdd-c035d005a812 None None] SCRIPT_NAME: `/identity` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:09:48.606922 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-709cb992-3e67-49ff-8cdd-c035d005a812 None None] PATH_INFO: `/` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:09:48.608769 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: [pid: 80377|app: 0|req: 741/1462] 127.0.0.1 () {54 vars in 916 bytes} [Tue Aug 2 10:09:48 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 02 10:09:48.614432 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33bdd80f-6e8a-46f2-b6f0-570d3ce0cdac None None] REQUEST_METHOD: `POST` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:09:48.614964 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33bdd80f-6e8a-46f2-b6f0-570d3ce0cdac None None] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:09:48.615070 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33bdd80f-6e8a-46f2-b6f0-570d3ce0cdac None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:09:48.683907 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.auth.core [None req-33bdd80f-6e8a-46f2-b6f0-570d3ce0cdac None None] MFA Rules not processed for user `891005fa5ff74041b75747d16cc66798`. Rule list: `[]` (Enabled: `True`). {{(pid=80378) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 02 10:09:48.685134 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.fernet_utils [None req-33bdd80f-6e8a-46f2-b6f0-570d3ce0cdac None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80378) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 10:09:48.730169 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG dbcounter [-] [80378] Writing DB stats keystone:SELECT=100 {{(pid=80378) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 10:09:48.791571 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 722/1463] 127.0.0.1 () {58 vars in 1013 bytes} [Tue Aug 2 10:09:48 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 178 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 02 10:09:48.830995 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-709cb992-3e67-49ff-8cdd-c035d005a812 None None] Authenticating user token {{(pid=80377) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:09:48.832549 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.fernet_utils [None req-709cb992-3e67-49ff-8cdd-c035d005a812 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80377) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 10:09:49.073633 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-709cb992-3e67-49ff-8cdd-c035d005a812 None None] Validating token access rules against request {{(pid=80377) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:09:49.113543 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG dbcounter [-] [80377] Writing DB stats keystone:SELECT=100 {{(pid=80377) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 10:09:49.150513 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0428c777-0d9d-40f3-8bb1-5c51cc251054 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '33b87170f0634b67bce60750847b5cbe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f1b38ebb804c41c69772c998aae72c41', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80377) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:09:49.151143 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0428c777-0d9d-40f3-8bb1-5c51cc251054 service neutron] REQUEST_METHOD: `GET` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:09:49.151426 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0428c777-0d9d-40f3-8bb1-5c51cc251054 service neutron] SCRIPT_NAME: `/identity` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:09:49.151703 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0428c777-0d9d-40f3-8bb1-5c51cc251054 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:09:49.153341 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.fernet_utils [None req-0428c777-0d9d-40f3-8bb1-5c51cc251054 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80377) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 10:09:49.267460 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0428c777-0d9d-40f3-8bb1-5c51cc251054 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:09:49.268896 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0428c777-0d9d-40f3-8bb1-5c51cc251054 service neutron] RBAC: Authorization granted {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:09:49.287639 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: [pid: 80377|app: 0|req: 742/1464] 10.222.0.82 () {60 vars in 1365 bytes} [Tue Aug 2 10:09:48 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 457 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 02 10:09:51.351625 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57a11e51-ebff-4205-9d1a-ae1c9796acb5 None None] REQUEST_METHOD: `GET` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:09:51.352212 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57a11e51-ebff-4205-9d1a-ae1c9796acb5 None None] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:09:51.352212 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57a11e51-ebff-4205-9d1a-ae1c9796acb5 None None] PATH_INFO: `/` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:09:51.353538 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 723/1465] 127.0.0.1 () {54 vars in 916 bytes} [Tue Aug 2 10:09:51 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 02 10:09:51.359114 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04278ea5-e8a0-47c4-9298-571f48c6c92a None None] REQUEST_METHOD: `POST` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:09:51.359530 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04278ea5-e8a0-47c4-9298-571f48c6c92a None None] SCRIPT_NAME: `/identity` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:09:51.359742 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04278ea5-e8a0-47c4-9298-571f48c6c92a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:09:51.431921 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.auth.core [None req-04278ea5-e8a0-47c4-9298-571f48c6c92a None None] MFA Rules not processed for user `891005fa5ff74041b75747d16cc66798`. Rule list: `[]` (Enabled: `True`). {{(pid=80377) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 02 10:09:51.433157 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.fernet_utils [None req-04278ea5-e8a0-47c4-9298-571f48c6c92a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80377) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 10:09:51.453252 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG dbcounter [-] [80377] Writing DB stats keystone:SELECT=100 {{(pid=80377) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 10:09:51.554636 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: [pid: 80377|app: 0|req: 743/1466] 127.0.0.1 () {58 vars in 1013 bytes} [Tue Aug 2 10:09:51 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 197 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 02 10:09:51.592893 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57a11e51-ebff-4205-9d1a-ae1c9796acb5 None None] Authenticating user token {{(pid=80378) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:09:51.594527 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.fernet_utils [None req-57a11e51-ebff-4205-9d1a-ae1c9796acb5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80378) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 10:09:51.710635 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57a11e51-ebff-4205-9d1a-ae1c9796acb5 None None] Validating token access rules against request {{(pid=80378) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:09:51.781673 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG dbcounter [-] [80378] Writing DB stats keystone:SELECT=100 {{(pid=80378) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 10:09:51.788889 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52962404-4e54-4242-88bf-f136073052cd service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '33b87170f0634b67bce60750847b5cbe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f1b38ebb804c41c69772c998aae72c41', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80378) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:09:51.789509 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52962404-4e54-4242-88bf-f136073052cd service neutron] REQUEST_METHOD: `GET` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:09:51.789806 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52962404-4e54-4242-88bf-f136073052cd service neutron] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:09:51.790086 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52962404-4e54-4242-88bf-f136073052cd service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:09:51.791715 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.fernet_utils [None req-52962404-4e54-4242-88bf-f136073052cd service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80378) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 10:09:51.925039 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52962404-4e54-4242-88bf-f136073052cd service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:09:51.926522 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52962404-4e54-4242-88bf-f136073052cd service neutron] RBAC: Authorization granted {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:09:51.944400 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 724/1467] 10.222.0.82 () {60 vars in 1365 bytes} [Tue Aug 2 10:09:51 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 352 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 02 10:09:54.026274 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85dfc728-8c74-4436-8d45-4391ce85ae67 None None] REQUEST_METHOD: `GET` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:09:54.026859 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85dfc728-8c74-4436-8d45-4391ce85ae67 None None] SCRIPT_NAME: `/identity` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:09:54.026859 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85dfc728-8c74-4436-8d45-4391ce85ae67 None None] PATH_INFO: `/` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:09:54.028157 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: [pid: 80377|app: 0|req: 744/1468] 127.0.0.1 () {54 vars in 916 bytes} [Tue Aug 2 10:09:54 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 02 10:09:54.034273 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a5ee574-2cec-49df-b28b-b950b6747eca None None] REQUEST_METHOD: `POST` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:09:54.034767 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a5ee574-2cec-49df-b28b-b950b6747eca None None] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:09:54.034864 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a5ee574-2cec-49df-b28b-b950b6747eca None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:09:54.102595 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.auth.core [None req-6a5ee574-2cec-49df-b28b-b950b6747eca None None] MFA Rules not processed for user `891005fa5ff74041b75747d16cc66798`. Rule list: `[]` (Enabled: `True`). {{(pid=80378) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 02 10:09:54.103732 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.fernet_utils [None req-6a5ee574-2cec-49df-b28b-b950b6747eca None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80378) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 10:09:54.156840 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG dbcounter [-] [80378] Writing DB stats keystone:SELECT=100 {{(pid=80378) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 10:09:54.221535 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 725/1469] 127.0.0.1 () {58 vars in 1013 bytes} [Tue Aug 2 10:09:54 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 188 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 02 10:09:54.260868 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85dfc728-8c74-4436-8d45-4391ce85ae67 None None] Authenticating user token {{(pid=80377) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:09:54.263037 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.fernet_utils [None req-85dfc728-8c74-4436-8d45-4391ce85ae67 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80377) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 10:09:54.381469 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85dfc728-8c74-4436-8d45-4391ce85ae67 None None] Validating token access rules against request {{(pid=80377) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:09:54.430604 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG dbcounter [-] [80377] Writing DB stats keystone:SELECT=100 {{(pid=80377) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 10:09:54.457981 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7972e3e3-2f83-41fe-8360-6d10397a173d service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '33b87170f0634b67bce60750847b5cbe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f1b38ebb804c41c69772c998aae72c41', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80377) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:09:54.458605 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7972e3e3-2f83-41fe-8360-6d10397a173d service neutron] REQUEST_METHOD: `GET` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:09:54.458881 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7972e3e3-2f83-41fe-8360-6d10397a173d service neutron] SCRIPT_NAME: `/identity` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:09:54.459151 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7972e3e3-2f83-41fe-8360-6d10397a173d service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:09:54.460944 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.fernet_utils [None req-7972e3e3-2f83-41fe-8360-6d10397a173d service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80377) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 10:09:54.579183 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7972e3e3-2f83-41fe-8360-6d10397a173d service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:09:54.580637 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7972e3e3-2f83-41fe-8360-6d10397a173d service neutron] RBAC: Authorization granted {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:09:54.600347 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: [pid: 80377|app: 0|req: 745/1470] 10.222.0.82 () {60 vars in 1365 bytes} [Tue Aug 2 10:09:54 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 340 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 02 10:10:04.218967 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG dbcounter [-] [80378] Writing DB stats keystone:SELECT=22 {{(pid=80378) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 10:10:04.597553 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG dbcounter [-] [80377] Writing DB stats keystone:SELECT=66 {{(pid=80377) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 10:10:14.524246 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-994e7524-c0b1-484c-a51f-3603534b6eaa None None] REQUEST_METHOD: `GET` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:10:14.524857 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-994e7524-c0b1-484c-a51f-3603534b6eaa None None] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:10:14.524857 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-994e7524-c0b1-484c-a51f-3603534b6eaa None None] PATH_INFO: `/` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:10:14.526178 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 726/1471] 10.222.0.82 () {54 vars in 926 bytes} [Tue Aug 2 10:10:14 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 02 10:10:14.532765 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af2d9d6e-e1e2-4d0a-841d-8795e3509b2f None None] REQUEST_METHOD: `POST` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:10:14.533215 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af2d9d6e-e1e2-4d0a-841d-8795e3509b2f None None] SCRIPT_NAME: `/identity` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:10:14.533327 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af2d9d6e-e1e2-4d0a-841d-8795e3509b2f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:10:14.601116 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.auth.core [None req-af2d9d6e-e1e2-4d0a-841d-8795e3509b2f None None] MFA Rules not processed for user `891005fa5ff74041b75747d16cc66798`. Rule list: `[]` (Enabled: `True`). {{(pid=80377) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 02 10:10:14.602382 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.fernet_utils [None req-af2d9d6e-e1e2-4d0a-841d-8795e3509b2f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80377) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 10:10:14.699238 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: [pid: 80377|app: 0|req: 746/1472] 10.222.0.82 () {58 vars in 1023 bytes} [Tue Aug 2 10:10:14 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 167 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 02 10:10:14.785924 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2420f99f-cb00-4dd6-a24c-58283af00ddd None None] REQUEST_METHOD: `GET` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:10:14.786216 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2420f99f-cb00-4dd6-a24c-58283af00ddd None None] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:10:14.786495 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2420f99f-cb00-4dd6-a24c-58283af00ddd None None] PATH_INFO: `/` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:10:14.787869 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 727/1473] 10.222.0.82 () {54 vars in 966 bytes} [Tue Aug 2 10:10:14 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 02 10:10:14.792517 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1659b493-df3b-44a0-a0a8-0711cb72a7e5 None None] REQUEST_METHOD: `POST` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:10:14.793074 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1659b493-df3b-44a0-a0a8-0711cb72a7e5 None None] SCRIPT_NAME: `/identity` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:10:14.793346 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1659b493-df3b-44a0-a0a8-0711cb72a7e5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:10:14.869750 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.auth.core [None req-1659b493-df3b-44a0-a0a8-0711cb72a7e5 None None] MFA Rules not processed for user `fa964af53a2b4fb9ace6afb544d54d40`. Rule list: `[]` (Enabled: `True`). {{(pid=80377) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 02 10:10:14.871011 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.fernet_utils [None req-1659b493-df3b-44a0-a0a8-0711cb72a7e5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80377) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 10:10:14.892209 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG dbcounter [-] [80377] Writing DB stats keystone:SELECT=100 {{(pid=80377) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 10:10:14.948216 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: [pid: 80377|app: 0|req: 747/1474] 10.222.0.82 () {58 vars in 1063 bytes} [Tue Aug 2 10:10:14 2022] POST /identity/v3/auth/tokens => generated 3120 bytes in 157 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 02 10:10:14.956443 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2420f99f-cb00-4dd6-a24c-58283af00ddd None None] Authenticating user token {{(pid=80378) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:10:14.957997 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.fernet_utils [None req-2420f99f-cb00-4dd6-a24c-58283af00ddd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80378) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 10:10:15.081147 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2420f99f-cb00-4dd6-a24c-58283af00ddd None None] Validating token access rules against request {{(pid=80378) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:10:15.174586 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd98dc83-ea9a-4cf4-9843-dad1fa75ff6b service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa964af53a2b4fb9ace6afb544d54d40', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f1b38ebb804c41c69772c998aae72c41', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80378) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:10:15.175213 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd98dc83-ea9a-4cf4-9843-dad1fa75ff6b service glance] REQUEST_METHOD: `GET` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:10:15.175494 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd98dc83-ea9a-4cf4-9843-dad1fa75ff6b service glance] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:10:15.175766 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd98dc83-ea9a-4cf4-9843-dad1fa75ff6b service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:10:15.177423 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.fernet_utils [None req-bd98dc83-ea9a-4cf4-9843-dad1fa75ff6b service glance] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80378) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 10:10:15.235154 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG dbcounter [-] [80378] Writing DB stats keystone:SELECT=100 {{(pid=80378) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 10:10:15.299294 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd98dc83-ea9a-4cf4-9843-dad1fa75ff6b service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:10:15.300739 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd98dc83-ea9a-4cf4-9843-dad1fa75ff6b service glance] RBAC: Authorization granted {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:10:15.320573 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 728/1475] 10.222.0.82 () {60 vars in 1365 bytes} [Tue Aug 2 10:10:14 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 365 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 02 10:10:17.434064 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-427e47f9-ea9f-4bb7-b6ad-790463dc2415 None None] REQUEST_METHOD: `GET` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:10:17.435291 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-427e47f9-ea9f-4bb7-b6ad-790463dc2415 None None] SCRIPT_NAME: `/identity` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:10:17.435552 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-427e47f9-ea9f-4bb7-b6ad-790463dc2415 None None] PATH_INFO: `/` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:10:17.438029 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: [pid: 80377|app: 0|req: 748/1476] 10.222.0.82 () {54 vars in 926 bytes} [Tue Aug 2 10:10:17 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 02 10:10:17.445760 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79889bc6-c0fa-4ec4-a5a0-b6fca09306c7 None None] REQUEST_METHOD: `POST` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:10:17.446225 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79889bc6-c0fa-4ec4-a5a0-b6fca09306c7 None None] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:10:17.446337 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79889bc6-c0fa-4ec4-a5a0-b6fca09306c7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:10:17.512315 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.auth.core [None req-79889bc6-c0fa-4ec4-a5a0-b6fca09306c7 None None] MFA Rules not processed for user `891005fa5ff74041b75747d16cc66798`. Rule list: `[]` (Enabled: `True`). {{(pid=80378) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 02 10:10:17.514177 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.fernet_utils [None req-79889bc6-c0fa-4ec4-a5a0-b6fca09306c7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80378) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 10:10:17.617507 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 729/1477] 10.222.0.82 () {58 vars in 1023 bytes} [Tue Aug 2 10:10:17 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 173 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 02 10:10:17.641954 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-427e47f9-ea9f-4bb7-b6ad-790463dc2415 None None] Authenticating user token {{(pid=80377) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:10:17.643477 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.fernet_utils [None req-427e47f9-ea9f-4bb7-b6ad-790463dc2415 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80377) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 10:10:17.768973 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-427e47f9-ea9f-4bb7-b6ad-790463dc2415 None None] Validating token access rules against request {{(pid=80377) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:10:17.817850 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG dbcounter [-] [80377] Writing DB stats keystone:SELECT=100 {{(pid=80377) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 10:10:17.863925 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16d74437-8778-40b3-ae7a-fdb73aa14a3f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '891005fa5ff74041b75747d16cc66798', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7047912c5acd4adeba5d87c3bf895e39', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80377) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:10:17.864552 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16d74437-8778-40b3-ae7a-fdb73aa14a3f admin admin] REQUEST_METHOD: `GET` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:10:17.864915 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16d74437-8778-40b3-ae7a-fdb73aa14a3f admin admin] SCRIPT_NAME: `/identity` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:10:17.865108 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16d74437-8778-40b3-ae7a-fdb73aa14a3f admin admin] PATH_INFO: `/v3/projects` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:10:17.865827 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16d74437-8778-40b3-ae7a-fdb73aa14a3f admin admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:10:17.867538 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: WARNING py.warnings [None req-16d74437-8778-40b3-ae7a-fdb73aa14a3f admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:list_projects": "(role:reader and system_scope:all) or (role:reader and domain_id:%(target.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Aug 02 10:10:17.867538 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: warnings.warn(msg) Aug 02 10:10:17.867538 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: Aug 02 10:10:17.868670 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16d74437-8778-40b3-ae7a-fdb73aa14a3f admin admin] RBAC: Authorization granted {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:10:17.894919 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: [pid: 80377|app: 0|req: 749/1478] 10.222.0.82 () {56 vars in 1108 bytes} [Tue Aug 2 10:10:17 2022] GET /identity/v3/projects => generated 2553 bytes in 253 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 02 10:10:19.956327 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be9b1eb1-2d03-4f6e-ad05-4449004b9077 None None] REQUEST_METHOD: `GET` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:10:19.956903 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be9b1eb1-2d03-4f6e-ad05-4449004b9077 None None] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:10:19.956903 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be9b1eb1-2d03-4f6e-ad05-4449004b9077 None None] PATH_INFO: `/` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:10:19.958517 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 730/1479] 10.222.0.82 () {54 vars in 926 bytes} [Tue Aug 2 10:10:19 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 02 10:10:19.964186 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01bccc58-b485-4d27-9342-55d50a7688da None None] REQUEST_METHOD: `POST` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:10:19.964732 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01bccc58-b485-4d27-9342-55d50a7688da None None] SCRIPT_NAME: `/identity` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:10:19.964732 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01bccc58-b485-4d27-9342-55d50a7688da None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:10:20.031261 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.auth.core [None req-01bccc58-b485-4d27-9342-55d50a7688da None None] MFA Rules not processed for user `891005fa5ff74041b75747d16cc66798`. Rule list: `[]` (Enabled: `True`). {{(pid=80377) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 02 10:10:20.032489 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.fernet_utils [None req-01bccc58-b485-4d27-9342-55d50a7688da None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80377) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 10:10:20.138313 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: [pid: 80377|app: 0|req: 750/1480] 10.222.0.82 () {58 vars in 1023 bytes} [Tue Aug 2 10:10:19 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 175 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 02 10:10:20.182470 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be9b1eb1-2d03-4f6e-ad05-4449004b9077 None None] Authenticating user token {{(pid=80378) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:10:20.184373 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.fernet_utils [None req-be9b1eb1-2d03-4f6e-ad05-4449004b9077 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80378) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 10:10:20.195041 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG dbcounter [-] [80378] Writing DB stats keystone:SELECT=100 {{(pid=80378) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 10:10:20.358795 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be9b1eb1-2d03-4f6e-ad05-4449004b9077 None None] Validating token access rules against request {{(pid=80378) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:10:20.440208 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG dbcounter [-] [80378] Writing DB stats keystone:SELECT=100 {{(pid=80378) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 10:10:20.474526 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24064087-ce9f-4ee4-83db-841f87e1f82d service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88af518196674fe593ad88d30111fb7d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f1b38ebb804c41c69772c998aae72c41', 'project_domain_id': 'default', 'roles': ['admin', 'ResellerAdmin', 'reader', 'service', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80378) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:10:20.475156 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24064087-ce9f-4ee4-83db-841f87e1f82d service nova] REQUEST_METHOD: `GET` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:10:20.475428 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24064087-ce9f-4ee4-83db-841f87e1f82d service nova] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:10:20.475698 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24064087-ce9f-4ee4-83db-841f87e1f82d service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:10:20.477325 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.fernet_utils [None req-24064087-ce9f-4ee4-83db-841f87e1f82d service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80378) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 10:10:20.569978 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24064087-ce9f-4ee4-83db-841f87e1f82d service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:10:20.571586 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24064087-ce9f-4ee4-83db-841f87e1f82d service nova] RBAC: Authorization granted {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:10:20.592409 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 731/1481] 10.222.0.82 () {60 vars in 1365 bytes} [Tue Aug 2 10:10:20 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 411 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 02 10:10:22.825708 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db515533-686f-4db0-9425-06813e7ccf84 None None] REQUEST_METHOD: `GET` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:10:22.826325 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db515533-686f-4db0-9425-06813e7ccf84 None None] SCRIPT_NAME: `/identity` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:10:22.826325 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db515533-686f-4db0-9425-06813e7ccf84 None None] PATH_INFO: `/` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:10:22.827705 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: [pid: 80377|app: 0|req: 751/1482] 10.222.0.82 () {54 vars in 926 bytes} [Tue Aug 2 10:10:22 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 02 10:10:22.833202 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-798c1c13-4d80-45d9-b6cf-0d836acf8021 None None] REQUEST_METHOD: `POST` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:10:22.833695 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-798c1c13-4d80-45d9-b6cf-0d836acf8021 None None] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:10:22.833821 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-798c1c13-4d80-45d9-b6cf-0d836acf8021 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:10:22.906205 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.auth.core [None req-798c1c13-4d80-45d9-b6cf-0d836acf8021 None None] MFA Rules not processed for user `891005fa5ff74041b75747d16cc66798`. Rule list: `[]` (Enabled: `True`). {{(pid=80378) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 02 10:10:22.907531 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.fernet_utils [None req-798c1c13-4d80-45d9-b6cf-0d836acf8021 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80378) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 10:10:22.956994 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG dbcounter [-] [80378] Writing DB stats keystone:SELECT=100 {{(pid=80378) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 10:10:23.017552 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 732/1483] 10.222.0.82 () {58 vars in 1023 bytes} [Tue Aug 2 10:10:22 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 185 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 02 10:10:23.054780 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db515533-686f-4db0-9425-06813e7ccf84 None None] Authenticating user token {{(pid=80377) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:10:23.056328 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.fernet_utils [None req-db515533-686f-4db0-9425-06813e7ccf84 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80377) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 10:10:23.081866 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG dbcounter [-] [80377] Writing DB stats keystone:SELECT=100 {{(pid=80377) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 10:10:23.166219 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db515533-686f-4db0-9425-06813e7ccf84 None None] Validating token access rules against request {{(pid=80377) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:10:23.246304 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbe4cfa1-4b19-4ced-bdd5-b01d211d56e1 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '33b87170f0634b67bce60750847b5cbe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f1b38ebb804c41c69772c998aae72c41', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80377) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:10:23.247507 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbe4cfa1-4b19-4ced-bdd5-b01d211d56e1 service neutron] REQUEST_METHOD: `GET` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:10:23.247859 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbe4cfa1-4b19-4ced-bdd5-b01d211d56e1 service neutron] SCRIPT_NAME: `/identity` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:10:23.248190 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbe4cfa1-4b19-4ced-bdd5-b01d211d56e1 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:10:23.249710 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.fernet_utils [None req-bbe4cfa1-4b19-4ced-bdd5-b01d211d56e1 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80377) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 10:10:23.326280 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG dbcounter [-] [80377] Writing DB stats keystone:SELECT=100 {{(pid=80377) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 10:10:23.364523 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bbe4cfa1-4b19-4ced-bdd5-b01d211d56e1 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:10:23.365999 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bbe4cfa1-4b19-4ced-bdd5-b01d211d56e1 service neutron] RBAC: Authorization granted {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:10:23.384456 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: [pid: 80377|app: 0|req: 752/1484] 10.222.0.82 () {60 vars in 1365 bytes} [Tue Aug 2 10:10:23 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 330 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 02 10:10:25.488177 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df09cf44-c635-469b-b953-77514e796b11 None None] REQUEST_METHOD: `GET` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:10:25.488743 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df09cf44-c635-469b-b953-77514e796b11 None None] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:10:25.488743 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df09cf44-c635-469b-b953-77514e796b11 None None] PATH_INFO: `/` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:10:25.490173 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 733/1485] 10.222.0.82 () {54 vars in 926 bytes} [Tue Aug 2 10:10:25 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 02 10:10:25.495976 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22f3bbb1-8304-4f17-aa16-ee659452516c None None] REQUEST_METHOD: `POST` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:10:25.496424 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22f3bbb1-8304-4f17-aa16-ee659452516c None None] SCRIPT_NAME: `/identity` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:10:25.496683 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22f3bbb1-8304-4f17-aa16-ee659452516c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:10:25.566528 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.auth.core [None req-22f3bbb1-8304-4f17-aa16-ee659452516c None None] MFA Rules not processed for user `891005fa5ff74041b75747d16cc66798`. Rule list: `[]` (Enabled: `True`). {{(pid=80377) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 02 10:10:25.567765 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.fernet_utils [None req-22f3bbb1-8304-4f17-aa16-ee659452516c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80377) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 10:10:25.670239 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: [pid: 80377|app: 0|req: 753/1486] 10.222.0.82 () {58 vars in 1023 bytes} [Tue Aug 2 10:10:25 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 175 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 02 10:10:25.708182 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df09cf44-c635-469b-b953-77514e796b11 None None] Authenticating user token {{(pid=80378) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:10:25.709717 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.fernet_utils [None req-df09cf44-c635-469b-b953-77514e796b11 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80378) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 10:10:25.817379 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df09cf44-c635-469b-b953-77514e796b11 None None] Validating token access rules against request {{(pid=80378) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:10:25.893684 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG dbcounter [-] [80378] Writing DB stats keystone:SELECT=100 {{(pid=80378) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 10:10:25.895892 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7901721-c62f-4a05-b659-fbfde64e4138 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '33b87170f0634b67bce60750847b5cbe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f1b38ebb804c41c69772c998aae72c41', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80378) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:10:25.896813 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7901721-c62f-4a05-b659-fbfde64e4138 service neutron] REQUEST_METHOD: `GET` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:10:25.897139 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7901721-c62f-4a05-b659-fbfde64e4138 service neutron] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:10:25.897634 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7901721-c62f-4a05-b659-fbfde64e4138 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:10:25.899520 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.fernet_utils [None req-d7901721-c62f-4a05-b659-fbfde64e4138 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80378) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 10:10:26.012976 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7901721-c62f-4a05-b659-fbfde64e4138 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:10:26.014442 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7901721-c62f-4a05-b659-fbfde64e4138 service neutron] RBAC: Authorization granted {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:10:26.032584 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 734/1487] 10.222.0.82 () {60 vars in 1365 bytes} [Tue Aug 2 10:10:25 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 325 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 02 10:10:28.226694 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ec0bf8f-401f-4fc0-b893-c29796894ab2 None None] REQUEST_METHOD: `GET` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:10:28.227515 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ec0bf8f-401f-4fc0-b893-c29796894ab2 None None] SCRIPT_NAME: `/identity` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:10:28.227715 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ec0bf8f-401f-4fc0-b893-c29796894ab2 None None] PATH_INFO: `/` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:10:28.229236 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: [pid: 80377|app: 0|req: 754/1488] 10.222.0.82 () {54 vars in 926 bytes} [Tue Aug 2 10:10:28 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 02 10:10:28.235306 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15c6dc0e-7451-4108-be44-cc99df066f09 None None] REQUEST_METHOD: `POST` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:10:28.235772 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15c6dc0e-7451-4108-be44-cc99df066f09 None None] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:10:28.235891 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15c6dc0e-7451-4108-be44-cc99df066f09 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:10:28.306774 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.auth.core [None req-15c6dc0e-7451-4108-be44-cc99df066f09 None None] MFA Rules not processed for user `891005fa5ff74041b75747d16cc66798`. Rule list: `[]` (Enabled: `True`). {{(pid=80378) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 02 10:10:28.307976 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.fernet_utils [None req-15c6dc0e-7451-4108-be44-cc99df066f09 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80378) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 10:10:28.363032 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG dbcounter [-] [80378] Writing DB stats keystone:SELECT=100 {{(pid=80378) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 10:10:28.413888 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 735/1489] 10.222.0.82 () {58 vars in 1023 bytes} [Tue Aug 2 10:10:28 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 180 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 02 10:10:28.462569 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ec0bf8f-401f-4fc0-b893-c29796894ab2 None None] Authenticating user token {{(pid=80377) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:10:28.464221 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.fernet_utils [None req-3ec0bf8f-401f-4fc0-b893-c29796894ab2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80377) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 10:10:28.506336 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG dbcounter [-] [80377] Writing DB stats keystone:SELECT=100 {{(pid=80377) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 10:10:28.580909 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ec0bf8f-401f-4fc0-b893-c29796894ab2 None None] Validating token access rules against request {{(pid=80377) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:10:28.659706 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7cd0127f-9c52-488e-ba4f-180fd70710ae service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '33b87170f0634b67bce60750847b5cbe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f1b38ebb804c41c69772c998aae72c41', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80377) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:10:28.660341 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cd0127f-9c52-488e-ba4f-180fd70710ae service neutron] REQUEST_METHOD: `GET` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:10:28.660623 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cd0127f-9c52-488e-ba4f-180fd70710ae service neutron] SCRIPT_NAME: `/identity` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:10:28.660896 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cd0127f-9c52-488e-ba4f-180fd70710ae service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:10:28.662696 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.fernet_utils [None req-7cd0127f-9c52-488e-ba4f-180fd70710ae service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80377) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 10:10:28.747545 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG dbcounter [-] [80377] Writing DB stats keystone:SELECT=100 {{(pid=80377) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 10:10:28.779023 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7cd0127f-9c52-488e-ba4f-180fd70710ae service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:10:28.780437 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7cd0127f-9c52-488e-ba4f-180fd70710ae service neutron] RBAC: Authorization granted {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:10:28.800094 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: [pid: 80377|app: 0|req: 755/1490] 10.222.0.82 () {60 vars in 1365 bytes} [Tue Aug 2 10:10:28 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 338 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 02 10:10:31.094173 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8ba0714-1c32-402c-bbc4-5a3f026d2e48 None None] REQUEST_METHOD: `GET` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:10:31.094704 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8ba0714-1c32-402c-bbc4-5a3f026d2e48 None None] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:10:31.094704 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8ba0714-1c32-402c-bbc4-5a3f026d2e48 None None] PATH_INFO: `/` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:10:31.096075 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 736/1491] 10.222.0.82 () {54 vars in 926 bytes} [Tue Aug 2 10:10:31 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 02 10:10:31.101328 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1cbba0b-f03a-4553-9ee9-5042906173ae None None] REQUEST_METHOD: `POST` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:10:31.101799 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1cbba0b-f03a-4553-9ee9-5042906173ae None None] SCRIPT_NAME: `/identity` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:10:31.101917 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1cbba0b-f03a-4553-9ee9-5042906173ae None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:10:31.173851 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.auth.core [None req-b1cbba0b-f03a-4553-9ee9-5042906173ae None None] MFA Rules not processed for user `891005fa5ff74041b75747d16cc66798`. Rule list: `[]` (Enabled: `True`). {{(pid=80377) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 02 10:10:31.175071 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.fernet_utils [None req-b1cbba0b-f03a-4553-9ee9-5042906173ae None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80377) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 10:10:31.271994 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: [pid: 80377|app: 0|req: 756/1492] 10.222.0.82 () {58 vars in 1023 bytes} [Tue Aug 2 10:10:31 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 172 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 02 10:10:31.314032 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8ba0714-1c32-402c-bbc4-5a3f026d2e48 None None] Authenticating user token {{(pid=80378) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:10:31.315842 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.fernet_utils [None req-d8ba0714-1c32-402c-bbc4-5a3f026d2e48 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80378) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 10:10:31.430181 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8ba0714-1c32-402c-bbc4-5a3f026d2e48 None None] Validating token access rules against request {{(pid=80378) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:10:31.508006 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e584ad04-703b-47cc-a5f2-57aab2a4c58c service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '33b87170f0634b67bce60750847b5cbe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f1b38ebb804c41c69772c998aae72c41', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80378) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:10:31.508625 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e584ad04-703b-47cc-a5f2-57aab2a4c58c service neutron] REQUEST_METHOD: `GET` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:10:31.508907 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e584ad04-703b-47cc-a5f2-57aab2a4c58c service neutron] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:10:31.509183 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e584ad04-703b-47cc-a5f2-57aab2a4c58c service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:10:31.510870 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.fernet_utils [None req-e584ad04-703b-47cc-a5f2-57aab2a4c58c service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80378) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 10:10:31.521392 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG dbcounter [-] [80378] Writing DB stats keystone:SELECT=100 {{(pid=80378) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 10:10:31.630543 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e584ad04-703b-47cc-a5f2-57aab2a4c58c service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:10:31.631982 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e584ad04-703b-47cc-a5f2-57aab2a4c58c service neutron] RBAC: Authorization granted {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:10:31.649738 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 737/1493] 10.222.0.82 () {60 vars in 1365 bytes} [Tue Aug 2 10:10:31 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 336 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 02 10:10:41.270135 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG dbcounter [-] [80377] Writing DB stats keystone:SELECT=82 {{(pid=80377) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 10:10:41.647543 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG dbcounter [-] [80378] Writing DB stats keystone:SELECT=50 {{(pid=80378) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 10:11:59.767698 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bb5380d-846e-456d-8169-dd9693e2622f None None] REQUEST_METHOD: `POST` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:11:59.768573 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bb5380d-846e-456d-8169-dd9693e2622f None None] SCRIPT_NAME: `/identity` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:11:59.768573 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bb5380d-846e-456d-8169-dd9693e2622f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:11:59.846675 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.auth.core [None req-5bb5380d-846e-456d-8169-dd9693e2622f None None] MFA Rules not processed for user `891005fa5ff74041b75747d16cc66798`. Rule list: `[]` (Enabled: `True`). {{(pid=80377) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 02 10:11:59.847929 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.fernet_utils [None req-5bb5380d-846e-456d-8169-dd9693e2622f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80377) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 10:11:59.945239 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: [pid: 80377|app: 0|req: 757/1494] 10.222.0.82 () {58 vars in 958 bytes} [Tue Aug 2 10:11:59 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 179 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 02 10:11:59.953437 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-976853e2-6130-4813-8ff0-ccdc0eaa1d61 None None] REQUEST_METHOD: `POST` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:11:59.953947 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-976853e2-6130-4813-8ff0-ccdc0eaa1d61 None None] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:11:59.954057 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-976853e2-6130-4813-8ff0-ccdc0eaa1d61 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:12:00.029346 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.auth.core [None req-976853e2-6130-4813-8ff0-ccdc0eaa1d61 None None] MFA Rules not processed for user `891005fa5ff74041b75747d16cc66798`. Rule list: `[]` (Enabled: `True`). {{(pid=80378) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 02 10:12:00.030486 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.fernet_utils [None req-976853e2-6130-4813-8ff0-ccdc0eaa1d61 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80378) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 10:12:00.128439 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 738/1495] 10.222.0.82 () {58 vars in 958 bytes} [Tue Aug 2 10:11:59 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 176 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 02 10:12:00.133102 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bb5380d-846e-456d-8169-dd9693e2622f None None] Authenticating user token {{(pid=80377) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:12:00.134347 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.fernet_utils [None req-5bb5380d-846e-456d-8169-dd9693e2622f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80377) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 10:12:00.145606 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bb5380d-846e-456d-8169-dd9693e2622f None None] Validating token access rules against request {{(pid=80377) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:12:00.205873 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG dbcounter [-] [80377] Writing DB stats keystone:SELECT=100 {{(pid=80377) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 10:12:00.239365 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ced6dd7-182e-4632-81ed-423aee51c9e6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '891005fa5ff74041b75747d16cc66798', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7047912c5acd4adeba5d87c3bf895e39', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80377) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:12:00.239982 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ced6dd7-182e-4632-81ed-423aee51c9e6 admin admin] REQUEST_METHOD: `GET` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:12:00.240261 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ced6dd7-182e-4632-81ed-423aee51c9e6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:12:00.240533 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ced6dd7-182e-4632-81ed-423aee51c9e6 admin admin] PATH_INFO: `/v3/domains` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:12:00.241226 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ced6dd7-182e-4632-81ed-423aee51c9e6 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80377) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 02 10:12:00.241529 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ced6dd7-182e-4632-81ed-423aee51c9e6 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:12:00.242573 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: WARNING py.warnings [None req-1ced6dd7-182e-4632-81ed-423aee51c9e6 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:list_domains": "role:reader and system_scope:all" failed scope check. The token used to make the request was project scoped but the policy requires ['system'] scope. This behavior may change in the future where using the intended scope is required Aug 02 10:12:00.242573 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: warnings.warn(msg) Aug 02 10:12:00.242573 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: Aug 02 10:12:00.243467 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ced6dd7-182e-4632-81ed-423aee51c9e6 admin admin] RBAC: Authorization granted {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:12:00.255071 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: [pid: 80377|app: 0|req: 758/1496] 10.222.0.82 () {58 vars in 1153 bytes} [Tue Aug 2 10:12:00 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 122 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 02 10:12:00.264107 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-976853e2-6130-4813-8ff0-ccdc0eaa1d61 None None] Authenticating user token {{(pid=80378) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:12:00.273220 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-976853e2-6130-4813-8ff0-ccdc0eaa1d61 None None] Validating token access rules against request {{(pid=80378) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:12:00.335059 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG dbcounter [-] [80378] Writing DB stats keystone:SELECT=100 {{(pid=80378) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 10:12:00.369130 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9503960a-705c-4104-97e7-f82a6b91e3bc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '891005fa5ff74041b75747d16cc66798', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7047912c5acd4adeba5d87c3bf895e39', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80378) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:12:00.369831 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9503960a-705c-4104-97e7-f82a6b91e3bc admin admin] REQUEST_METHOD: `POST` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:12:00.370113 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9503960a-705c-4104-97e7-f82a6b91e3bc admin admin] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:12:00.370384 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9503960a-705c-4104-97e7-f82a6b91e3bc admin admin] PATH_INFO: `/v3/projects` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:12:00.371055 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9503960a-705c-4104-97e7-f82a6b91e3bc admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:12:00.372816 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: WARNING py.warnings [None req-9503960a-705c-4104-97e7-f82a6b91e3bc admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:create_project": "(role:admin and system_scope:all) or (role:admin and domain_id:%(target.project.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Aug 02 10:12:00.372816 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: warnings.warn(msg) Aug 02 10:12:00.372816 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: Aug 02 10:12:00.374032 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9503960a-705c-4104-97e7-f82a6b91e3bc admin admin] RBAC: Authorization granted {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:12:00.429289 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 739/1497] 10.222.0.82 () {60 vars in 1153 bytes} [Tue Aug 2 10:12:00 2022] POST /identity/v3/projects => generated 377 bytes in 166 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 02 10:12:00.433876 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ced6dd7-182e-4632-81ed-423aee51c9e6 admin admin] Authenticating user token {{(pid=80377) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:12:00.435600 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.fernet_utils [None req-1ced6dd7-182e-4632-81ed-423aee51c9e6 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80377) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 10:12:00.469382 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ced6dd7-182e-4632-81ed-423aee51c9e6 admin admin] Validating token access rules against request {{(pid=80377) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:12:00.562493 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3c2b2f2-c103-458c-85cc-410bfa61f112 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '891005fa5ff74041b75747d16cc66798', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7047912c5acd4adeba5d87c3bf895e39', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80377) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:12:00.563115 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3c2b2f2-c103-458c-85cc-410bfa61f112 admin admin] REQUEST_METHOD: `POST` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:12:00.563397 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3c2b2f2-c103-458c-85cc-410bfa61f112 admin admin] SCRIPT_NAME: `/identity` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:12:00.563672 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3c2b2f2-c103-458c-85cc-410bfa61f112 admin admin] PATH_INFO: `/v3/users` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:12:00.564335 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3c2b2f2-c103-458c-85cc-410bfa61f112 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:12:00.565385 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: WARNING py.warnings [None req-a3c2b2f2-c103-458c-85cc-410bfa61f112 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:create_user": "(role:admin and system_scope:all) or (role:admin and token.domain.id:%(target.user.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Aug 02 10:12:00.565385 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: warnings.warn(msg) Aug 02 10:12:00.565385 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: Aug 02 10:12:00.566724 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3c2b2f2-c103-458c-85cc-410bfa61f112 admin admin] RBAC: Authorization granted {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:12:00.609676 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: [pid: 80377|app: 0|req: 759/1498] 10.222.0.82 () {60 vars in 1144 bytes} [Tue Aug 2 10:12:00 2022] POST /identity/v3/users => generated 289 bytes in 176 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 02 10:12:00.613844 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9503960a-705c-4104-97e7-f82a6b91e3bc admin admin] Authenticating user token {{(pid=80378) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:12:00.615349 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.fernet_utils [None req-9503960a-705c-4104-97e7-f82a6b91e3bc admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80378) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 10:12:00.676363 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9503960a-705c-4104-97e7-f82a6b91e3bc admin admin] Validating token access rules against request {{(pid=80378) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:12:00.759709 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG dbcounter [-] [80378] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80378) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 10:12:00.771164 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9850bcb-c95a-4e8d-8171-f12227fb2f9b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '891005fa5ff74041b75747d16cc66798', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7047912c5acd4adeba5d87c3bf895e39', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80378) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:12:00.771776 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9850bcb-c95a-4e8d-8171-f12227fb2f9b admin admin] REQUEST_METHOD: `GET` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:12:00.772059 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9850bcb-c95a-4e8d-8171-f12227fb2f9b admin admin] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:12:00.772333 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9850bcb-c95a-4e8d-8171-f12227fb2f9b admin admin] PATH_INFO: `/v3/roles` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:12:00.773032 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9850bcb-c95a-4e8d-8171-f12227fb2f9b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:12:00.774135 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: WARNING py.warnings [None req-a9850bcb-c95a-4e8d-8171-f12227fb2f9b admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:list_roles": "role:reader and system_scope:all" failed scope check. The token used to make the request was project scoped but the policy requires ['system'] scope. This behavior may change in the future where using the intended scope is required Aug 02 10:12:00.774135 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: warnings.warn(msg) Aug 02 10:12:00.774135 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: Aug 02 10:12:00.775019 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9850bcb-c95a-4e8d-8171-f12227fb2f9b admin admin] RBAC: Authorization granted {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:12:00.788188 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 740/1499] 10.222.0.82 () {58 vars in 1122 bytes} [Tue Aug 2 10:12:00 2022] GET /identity/v3/roles => generated 1420 bytes in 175 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 02 10:12:00.792406 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3c2b2f2-c103-458c-85cc-410bfa61f112 admin admin] Authenticating user token {{(pid=80377) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:12:00.794805 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.fernet_utils [None req-a3c2b2f2-c103-458c-85cc-410bfa61f112 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80377) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 10:12:00.806078 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3c2b2f2-c103-458c-85cc-410bfa61f112 admin admin] Validating token access rules against request {{(pid=80377) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:12:00.833484 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG dbcounter [-] [80377] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80377) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 10:12:00.908271 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4cf1b521-21d0-49be-9fa9-312495fe6674 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '891005fa5ff74041b75747d16cc66798', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7047912c5acd4adeba5d87c3bf895e39', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80377) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:12:00.908850 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cf1b521-21d0-49be-9fa9-312495fe6674 admin admin] REQUEST_METHOD: `GET` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:12:00.909516 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cf1b521-21d0-49be-9fa9-312495fe6674 admin admin] SCRIPT_NAME: `/identity` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:12:00.909516 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cf1b521-21d0-49be-9fa9-312495fe6674 admin admin] PATH_INFO: `/v3/roles` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:12:00.910304 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4cf1b521-21d0-49be-9fa9-312495fe6674 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:12:00.911425 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: WARNING py.warnings [None req-4cf1b521-21d0-49be-9fa9-312495fe6674 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:list_roles": "role:reader and system_scope:all" failed scope check. The token used to make the request was project scoped but the policy requires ['system'] scope. This behavior may change in the future where using the intended scope is required Aug 02 10:12:00.911425 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: warnings.warn(msg) Aug 02 10:12:00.911425 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: Aug 02 10:12:00.912228 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4cf1b521-21d0-49be-9fa9-312495fe6674 admin admin] RBAC: Authorization granted {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:12:00.926893 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: [pid: 80377|app: 0|req: 760/1500] 10.222.0.82 () {58 vars in 1122 bytes} [Tue Aug 2 10:12:00 2022] GET /identity/v3/roles => generated 1420 bytes in 135 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 02 10:12:00.931278 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9850bcb-c95a-4e8d-8171-f12227fb2f9b admin admin] Authenticating user token {{(pid=80378) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:12:00.933358 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.fernet_utils [None req-a9850bcb-c95a-4e8d-8171-f12227fb2f9b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80378) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 10:12:00.969765 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9850bcb-c95a-4e8d-8171-f12227fb2f9b admin admin] Validating token access rules against request {{(pid=80378) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:12:01.066394 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-574739e1-9db4-4fe0-bbc3-cbac710a9f8c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '891005fa5ff74041b75747d16cc66798', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7047912c5acd4adeba5d87c3bf895e39', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80378) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:12:01.067077 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-574739e1-9db4-4fe0-bbc3-cbac710a9f8c admin admin] REQUEST_METHOD: `PUT` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:12:01.067361 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-574739e1-9db4-4fe0-bbc3-cbac710a9f8c admin admin] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:12:01.067638 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-574739e1-9db4-4fe0-bbc3-cbac710a9f8c admin admin] PATH_INFO: `/v3/projects/5f2c13a8a7254e5cbcf1065cff9772f4/users/efae3c60190b4444a00e47b7511fe536/roles/e1c04b19fc4543b8b54f58bbcffbc598` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:12:01.090027 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-574739e1-9db4-4fe0-bbc3-cbac710a9f8c admin admin] RBAC: Authorizing `identity:create_grant(project_id=5f2c13a8a7254e5cbcf1065cff9772f4, user_id=efae3c60190b4444a00e47b7511fe536, role_id=e1c04b19fc4543b8b54f58bbcffbc598)` {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:12:01.091075 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: WARNING py.warnings [None req-574739e1-9db4-4fe0-bbc3-cbac710a9f8c admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:create_grant": "(role:admin and system_scope:all) or ((role:admin and domain_id:%(target.user.domain_id)s and domain_id:%(target.project.domain_id)s) or (role:admin and domain_id:%(target.user.domain_id)s and domain_id:%(target.domain.id)s) or (role:admin and domain_id:%(target.group.domain_id)s and domain_id:%(target.project.domain_id)s) or (role:admin and domain_id:%(target.group.domain_id)s and domain_id:%(target.domain.id)s)) and (domain_id:%(target.role.domain_id)s or None:%(target.role.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Aug 02 10:12:01.091075 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: warnings.warn(msg) Aug 02 10:12:01.091075 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: Aug 02 10:12:01.093112 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-574739e1-9db4-4fe0-bbc3-cbac710a9f8c admin admin] RBAC: Authorization granted {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:12:01.123307 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 741/1501] 10.222.0.82 () {60 vars in 1483 bytes} [Tue Aug 2 10:12:00 2022] PUT /identity/v3/projects/5f2c13a8a7254e5cbcf1065cff9772f4/users/efae3c60190b4444a00e47b7511fe536/roles/e1c04b19fc4543b8b54f58bbcffbc598 => generated 0 bytes in 193 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 02 10:12:01.135696 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25dd8212-be19-4ec9-9778-49ab45f5cf8c None None] REQUEST_METHOD: `POST` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:12:01.136011 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25dd8212-be19-4ec9-9778-49ab45f5cf8c None None] SCRIPT_NAME: `/identity` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:12:01.136317 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25dd8212-be19-4ec9-9778-49ab45f5cf8c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:12:01.199938 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.auth.core [None req-25dd8212-be19-4ec9-9778-49ab45f5cf8c None None] MFA Rules not processed for user `efae3c60190b4444a00e47b7511fe536`. Rule list: `[]` (Enabled: `True`). {{(pid=80377) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 02 10:12:01.201084 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.fernet_utils [None req-25dd8212-be19-4ec9-9778-49ab45f5cf8c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80377) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 10:12:01.282171 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: [pid: 80377|app: 0|req: 761/1502] 10.222.0.82 () {58 vars in 958 bytes} [Tue Aug 2 10:12:01 2022] POST /identity/v3/auth/tokens => generated 3254 bytes in 148 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 02 10:12:01.297133 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f148331e-7402-41a6-a652-13e7ef0896e1 None None] REQUEST_METHOD: `GET` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:12:01.297420 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f148331e-7402-41a6-a652-13e7ef0896e1 None None] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:12:01.297694 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f148331e-7402-41a6-a652-13e7ef0896e1 None None] PATH_INFO: `/` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:12:01.299168 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 742/1503] 10.222.0.82 () {54 vars in 964 bytes} [Tue Aug 2 10:12:01 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 02 10:12:01.304327 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b5b78c6-c7c4-4535-8b7c-105b01292928 None None] REQUEST_METHOD: `POST` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:12:01.304624 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b5b78c6-c7c4-4535-8b7c-105b01292928 None None] SCRIPT_NAME: `/identity` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:12:01.304907 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b5b78c6-c7c4-4535-8b7c-105b01292928 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:12:01.341846 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG dbcounter [-] [80377] Writing DB stats keystone:SELECT=100 {{(pid=80377) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 10:12:01.387448 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.auth.core [None req-9b5b78c6-c7c4-4535-8b7c-105b01292928 None None] MFA Rules not processed for user `1023de281ae74e9d8a6d6c312c1c97a7`. Rule list: `[]` (Enabled: `True`). {{(pid=80377) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 02 10:12:01.388655 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.fernet_utils [None req-9b5b78c6-c7c4-4535-8b7c-105b01292928 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80377) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 10:12:01.467256 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: [pid: 80377|app: 0|req: 762/1504] 10.222.0.82 () {58 vars in 1061 bytes} [Tue Aug 2 10:12:01 2022] POST /identity/v3/auth/tokens => generated 3120 bytes in 164 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 02 10:12:01.475134 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f148331e-7402-41a6-a652-13e7ef0896e1 None None] Authenticating user token {{(pid=80378) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:12:01.476761 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.fernet_utils [None req-f148331e-7402-41a6-a652-13e7ef0896e1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80378) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 10:12:01.528401 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG dbcounter [-] [80378] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80378) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 10:12:01.602535 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f148331e-7402-41a6-a652-13e7ef0896e1 None None] Validating token access rules against request {{(pid=80378) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:12:01.684121 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83a29d37-4e2c-4ec5-a584-c1b4c32533ba service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1023de281ae74e9d8a6d6c312c1c97a7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f1b38ebb804c41c69772c998aae72c41', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80378) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:12:01.684756 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83a29d37-4e2c-4ec5-a584-c1b4c32533ba service cinder] REQUEST_METHOD: `GET` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:12:01.685042 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83a29d37-4e2c-4ec5-a584-c1b4c32533ba service cinder] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:12:01.685315 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83a29d37-4e2c-4ec5-a584-c1b4c32533ba service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:12:01.687006 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.fernet_utils [None req-83a29d37-4e2c-4ec5-a584-c1b4c32533ba service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80378) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 10:12:01.794505 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83a29d37-4e2c-4ec5-a584-c1b4c32533ba service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:12:01.795952 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83a29d37-4e2c-4ec5-a584-c1b4c32533ba service cinder] RBAC: Authorization granted {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:12:01.799497 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG dbcounter [-] [80378] Writing DB stats keystone:SELECT=100 {{(pid=80378) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 10:12:01.815024 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 743/1505] 10.222.0.82 () {60 vars in 1365 bytes} [Tue Aug 2 10:12:01 2022] GET /identity/v3/auth/tokens => generated 3254 bytes in 340 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 02 10:12:01.860838 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c3afc42-c0c5-441d-8490-5f1fcc6a9f94 None None] REQUEST_METHOD: `GET` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:12:01.861278 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c3afc42-c0c5-441d-8490-5f1fcc6a9f94 None None] SCRIPT_NAME: `/identity` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:12:01.861382 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c3afc42-c0c5-441d-8490-5f1fcc6a9f94 None None] PATH_INFO: `/` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:12:01.862691 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: [pid: 80377|app: 0|req: 763/1506] 10.222.0.82 () {52 vars in 830 bytes} [Tue Aug 2 10:12:01 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 02 10:12:01.866386 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83a29d37-4e2c-4ec5-a584-c1b4c32533ba service cinder] Authenticating user token {{(pid=80378) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:12:01.867605 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.fernet_utils [None req-83a29d37-4e2c-4ec5-a584-c1b4c32533ba service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80378) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 10:12:01.966195 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83a29d37-4e2c-4ec5-a584-c1b4c32533ba service cinder] Validating token access rules against request {{(pid=80378) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:12:02.059270 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8361ccea-4487-4164-bffa-d9c98def9f7b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '891005fa5ff74041b75747d16cc66798', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7047912c5acd4adeba5d87c3bf895e39', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80378) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:12:02.059913 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8361ccea-4487-4164-bffa-d9c98def9f7b admin admin] REQUEST_METHOD: `DELETE` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:12:02.060114 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8361ccea-4487-4164-bffa-d9c98def9f7b admin admin] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:12:02.060397 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8361ccea-4487-4164-bffa-d9c98def9f7b admin admin] PATH_INFO: `/v3/users/efae3c60190b4444a00e47b7511fe536` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:12:02.072655 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8361ccea-4487-4164-bffa-d9c98def9f7b admin admin] RBAC: Authorizing `identity:delete_user(user_id=efae3c60190b4444a00e47b7511fe536)` {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:12:02.073745 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: WARNING py.warnings [None req-8361ccea-4487-4164-bffa-d9c98def9f7b admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:delete_user": "(role:admin and system_scope:all) or (role:admin and token.domain.id:%(target.user.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Aug 02 10:12:02.073745 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: warnings.warn(msg) Aug 02 10:12:02.073745 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: Aug 02 10:12:02.075016 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8361ccea-4487-4164-bffa-d9c98def9f7b admin admin] RBAC: Authorization granted {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:12:02.084962 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG dbcounter [-] [80378] Writing DB stats keystone:SELECT=100 {{(pid=80378) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 10:12:02.141101 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.notifications [None req-8361ccea-4487-4164-bffa-d9c98def9f7b admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'efae3c60190b4444a00e47b7511fe536'} {{(pid=80378) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 10:12:02.150283 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.notifications [None req-8361ccea-4487-4164-bffa-d9c98def9f7b admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'efae3c60190b4444a00e47b7511fe536'} {{(pid=80378) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 10:12:02.164315 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.notifications [None req-8361ccea-4487-4164-bffa-d9c98def9f7b admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'efae3c60190b4444a00e47b7511fe536'} {{(pid=80378) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 10:12:02.164753 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.notifications [None req-8361ccea-4487-4164-bffa-d9c98def9f7b admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'efae3c60190b4444a00e47b7511fe536'} {{(pid=80378) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 10:12:02.189007 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.notifications [None req-8361ccea-4487-4164-bffa-d9c98def9f7b admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'efae3c60190b4444a00e47b7511fe536'} {{(pid=80378) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 10:12:02.194529 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 744/1507] 10.222.0.82 () {58 vars in 1224 bytes} [Tue Aug 2 10:12:01 2022] DELETE /identity/v3/users/efae3c60190b4444a00e47b7511fe536 => generated 0 bytes in 329 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 02 10:12:02.199096 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99b06662-a3c8-4db5-b65e-c8e9852c464e None None] REQUEST_METHOD: `POST` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:12:02.199384 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99b06662-a3c8-4db5-b65e-c8e9852c464e None None] SCRIPT_NAME: `/identity` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:12:02.199657 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99b06662-a3c8-4db5-b65e-c8e9852c464e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:12:02.255910 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.auth.core [None req-99b06662-a3c8-4db5-b65e-c8e9852c464e None None] MFA Rules not processed for user `891005fa5ff74041b75747d16cc66798`. Rule list: `[]` (Enabled: `True`). {{(pid=80377) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 02 10:12:02.257107 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.fernet_utils [None req-99b06662-a3c8-4db5-b65e-c8e9852c464e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80377) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 10:12:02.331949 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG dbcounter [-] [80377] Writing DB stats keystone:SELECT=100 {{(pid=80377) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 10:12:02.345474 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: [pid: 80377|app: 0|req: 764/1508] 10.222.0.82 () {58 vars in 958 bytes} [Tue Aug 2 10:12:02 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 147 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 02 10:12:02.355601 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8361ccea-4487-4164-bffa-d9c98def9f7b admin admin] Authenticating user token {{(pid=80378) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:12:02.356830 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.fernet_utils [None req-8361ccea-4487-4164-bffa-d9c98def9f7b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80378) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 10:12:02.455930 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8361ccea-4487-4164-bffa-d9c98def9f7b admin admin] Validating token access rules against request {{(pid=80378) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:12:02.513472 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG dbcounter [-] [80378] Writing DB stats keystone:DELETE=10,keystone:SELECT=89,keystone:INSERT=1 {{(pid=80378) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 10:12:02.534066 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51615b6e-247b-43be-b855-5c33282ea615 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '33b87170f0634b67bce60750847b5cbe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f1b38ebb804c41c69772c998aae72c41', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80378) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:12:02.534687 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51615b6e-247b-43be-b855-5c33282ea615 service neutron] REQUEST_METHOD: `GET` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:12:02.534964 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51615b6e-247b-43be-b855-5c33282ea615 service neutron] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:12:02.535235 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51615b6e-247b-43be-b855-5c33282ea615 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:12:02.536849 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.fernet_utils [None req-51615b6e-247b-43be-b855-5c33282ea615 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80378) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 10:12:02.645665 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51615b6e-247b-43be-b855-5c33282ea615 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:12:02.647056 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51615b6e-247b-43be-b855-5c33282ea615 service neutron] RBAC: Authorization granted {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:12:02.665289 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 745/1509] 10.222.0.82 () {60 vars in 1365 bytes} [Tue Aug 2 10:12:02 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 311 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 02 10:12:03.012538 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99b06662-a3c8-4db5-b65e-c8e9852c464e None None] Authenticating user token {{(pid=80377) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:12:03.013886 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.fernet_utils [None req-99b06662-a3c8-4db5-b65e-c8e9852c464e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80377) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 10:12:03.025205 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99b06662-a3c8-4db5-b65e-c8e9852c464e None None] Validating token access rules against request {{(pid=80377) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:12:03.126756 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e2fb7f4-09ad-4d89-a489-6c83f636aabf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '891005fa5ff74041b75747d16cc66798', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7047912c5acd4adeba5d87c3bf895e39', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80377) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:12:03.127396 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e2fb7f4-09ad-4d89-a489-6c83f636aabf admin admin] REQUEST_METHOD: `DELETE` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:12:03.127675 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e2fb7f4-09ad-4d89-a489-6c83f636aabf admin admin] SCRIPT_NAME: `/identity` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:12:03.127944 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e2fb7f4-09ad-4d89-a489-6c83f636aabf admin admin] PATH_INFO: `/v3/projects/5f2c13a8a7254e5cbcf1065cff9772f4` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:12:03.137893 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e2fb7f4-09ad-4d89-a489-6c83f636aabf admin admin] RBAC: Authorizing `identity:delete_project(project_id=5f2c13a8a7254e5cbcf1065cff9772f4)` {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:12:03.138927 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: WARNING py.warnings [None req-9e2fb7f4-09ad-4d89-a489-6c83f636aabf admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:delete_project": "(role:admin and system_scope:all) or (role:admin and domain_id:%(target.project.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Aug 02 10:12:03.138927 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: warnings.warn(msg) Aug 02 10:12:03.138927 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: Aug 02 10:12:03.140124 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e2fb7f4-09ad-4d89-a489-6c83f636aabf admin admin] RBAC: Authorization granted {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:12:03.198742 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.notifications [None req-9e2fb7f4-09ad-4d89-a489-6c83f636aabf admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '5f2c13a8a7254e5cbcf1065cff9772f4', 'request_id': 'req-9e2fb7f4-09ad-4d89-a489-6c83f636aabf'} {{(pid=80377) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 10:12:03.206999 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.notifications [None req-9e2fb7f4-09ad-4d89-a489-6c83f636aabf admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '5f2c13a8a7254e5cbcf1065cff9772f4', 'request_id': 'req-9e2fb7f4-09ad-4d89-a489-6c83f636aabf'} {{(pid=80377) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 10:12:03.221945 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.notifications [None req-9e2fb7f4-09ad-4d89-a489-6c83f636aabf admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '5f2c13a8a7254e5cbcf1065cff9772f4', 'request_id': 'req-9e2fb7f4-09ad-4d89-a489-6c83f636aabf'} {{(pid=80377) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 10:12:03.252086 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.notifications [None req-9e2fb7f4-09ad-4d89-a489-6c83f636aabf admin admin] The token cache is being invalidate because project 5f2c13a8a7254e5cbcf1065cff9772f4 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80377) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 02 10:12:03.252379 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.notifications [None req-9e2fb7f4-09ad-4d89-a489-6c83f636aabf admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80377) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 10:12:03.252833 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.notifications [None req-9e2fb7f4-09ad-4d89-a489-6c83f636aabf admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80377) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 10:12:03.254371 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: [pid: 80377|app: 0|req: 765/1510] 10.222.0.82 () {58 vars in 1233 bytes} [Tue Aug 2 10:12:03 2022] DELETE /identity/v3/projects/5f2c13a8a7254e5cbcf1065cff9772f4 => generated 0 bytes in 242 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 02 10:12:12.662951 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG dbcounter [-] [80378] Writing DB stats keystone:SELECT=62 {{(pid=80378) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 10:12:13.249970 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG dbcounter [-] [80377] Writing DB stats keystone:SELECT=84,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80377) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 10:13:56.068166 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d74c3686-d8c5-49ca-af28-0d7bb487f95e None None] REQUEST_METHOD: `GET` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:13:56.068749 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d74c3686-d8c5-49ca-af28-0d7bb487f95e None None] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:13:56.068749 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d74c3686-d8c5-49ca-af28-0d7bb487f95e None None] PATH_INFO: `/` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:13:56.070126 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 746/1511] 127.0.0.1 () {54 vars in 901 bytes} [Tue Aug 2 10:13:56 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 02 10:13:56.075334 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6576b1ab-c9ff-4c62-812b-9586b9a8fcc4 None None] REQUEST_METHOD: `POST` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:13:56.075797 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6576b1ab-c9ff-4c62-812b-9586b9a8fcc4 None None] SCRIPT_NAME: `/identity` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:13:56.075904 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6576b1ab-c9ff-4c62-812b-9586b9a8fcc4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:13:56.171373 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.auth.core [None req-6576b1ab-c9ff-4c62-812b-9586b9a8fcc4 None None] MFA Rules not processed for user `891005fa5ff74041b75747d16cc66798`. Rule list: `[]` (Enabled: `True`). {{(pid=80377) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 02 10:13:56.172524 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.fernet_utils [None req-6576b1ab-c9ff-4c62-812b-9586b9a8fcc4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80377) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 10:13:56.280787 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: [pid: 80377|app: 0|req: 766/1512] 127.0.0.1 () {58 vars in 998 bytes} [Tue Aug 2 10:13:56 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 206 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 02 10:13:56.435204 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d74c3686-d8c5-49ca-af28-0d7bb487f95e None None] Authenticating user token {{(pid=80378) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:13:56.437299 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.fernet_utils [None req-d74c3686-d8c5-49ca-af28-0d7bb487f95e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80378) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 10:13:56.565433 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d74c3686-d8c5-49ca-af28-0d7bb487f95e None None] Validating token access rules against request {{(pid=80378) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:13:56.570924 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ee004ed-b8b7-4c2b-a475-36ef014971bc service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88af518196674fe593ad88d30111fb7d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f1b38ebb804c41c69772c998aae72c41', 'project_domain_id': 'default', 'roles': ['admin', 'ResellerAdmin', 'reader', 'service', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80378) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:13:56.571545 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ee004ed-b8b7-4c2b-a475-36ef014971bc service nova] REQUEST_METHOD: `GET` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:13:56.571831 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ee004ed-b8b7-4c2b-a475-36ef014971bc service nova] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:13:56.572110 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ee004ed-b8b7-4c2b-a475-36ef014971bc service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:13:56.580156 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ee004ed-b8b7-4c2b-a475-36ef014971bc service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:13:56.581765 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ee004ed-b8b7-4c2b-a475-36ef014971bc service nova] RBAC: Authorization granted {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:13:56.591192 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 747/1513] 10.222.0.82 () {60 vars in 1365 bytes} [Tue Aug 2 10:13:56 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 156 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 02 10:13:57.429821 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6db52af3-3c2f-4628-9ec2-58770f717c7e None None] REQUEST_METHOD: `GET` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:13:57.431336 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6db52af3-3c2f-4628-9ec2-58770f717c7e None None] SCRIPT_NAME: `/identity` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:13:57.431336 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6db52af3-3c2f-4628-9ec2-58770f717c7e None None] PATH_INFO: `/` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:13:57.433463 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: [pid: 80377|app: 0|req: 767/1514] 127.0.0.1 () {54 vars in 901 bytes} [Tue Aug 2 10:13:57 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 02 10:13:57.439286 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b04f196-c492-4104-9918-387f1b063160 None None] REQUEST_METHOD: `POST` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:13:57.439776 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b04f196-c492-4104-9918-387f1b063160 None None] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:13:57.439874 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b04f196-c492-4104-9918-387f1b063160 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:13:57.460238 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.auth.core [None req-9b04f196-c492-4104-9918-387f1b063160 None None] MFA Rules not processed for user `891005fa5ff74041b75747d16cc66798`. Rule list: `[]` (Enabled: `True`). {{(pid=80378) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 02 10:13:57.461446 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.fernet_utils [None req-9b04f196-c492-4104-9918-387f1b063160 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80378) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 10:13:57.469327 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 748/1515] 127.0.0.1 () {58 vars in 998 bytes} [Tue Aug 2 10:13:57 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 02 10:13:57.625920 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6db52af3-3c2f-4628-9ec2-58770f717c7e None None] Authenticating user token {{(pid=80377) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:13:57.640736 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6db52af3-3c2f-4628-9ec2-58770f717c7e None None] Validating token access rules against request {{(pid=80377) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:13:57.646081 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1aa69be-ce61-4775-bd6f-57f23c911afa service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88af518196674fe593ad88d30111fb7d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f1b38ebb804c41c69772c998aae72c41', 'project_domain_id': 'default', 'roles': ['admin', 'ResellerAdmin', 'reader', 'service', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80377) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:13:57.646695 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1aa69be-ce61-4775-bd6f-57f23c911afa service nova] REQUEST_METHOD: `GET` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:13:57.646980 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1aa69be-ce61-4775-bd6f-57f23c911afa service nova] SCRIPT_NAME: `/identity` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:13:57.647256 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1aa69be-ce61-4775-bd6f-57f23c911afa service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:13:57.655950 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1aa69be-ce61-4775-bd6f-57f23c911afa service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:13:57.657537 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1aa69be-ce61-4775-bd6f-57f23c911afa service nova] RBAC: Authorization granted {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:13:57.666864 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: [pid: 80377|app: 0|req: 768/1516] 10.222.0.82 () {60 vars in 1365 bytes} [Tue Aug 2 10:13:57 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 41 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 02 10:13:57.943965 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18a116da-fc53-4f4d-a580-9ed199360872 None None] REQUEST_METHOD: `GET` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:13:57.944255 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18a116da-fc53-4f4d-a580-9ed199360872 None None] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:13:57.944534 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18a116da-fc53-4f4d-a580-9ed199360872 None None] PATH_INFO: `/v3` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:13:57.945853 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 749/1517] 10.222.0.79 () {54 vars in 925 bytes} [Tue Aug 2 10:13:57 2022] GET /identity/v3 => generated 255 bytes in 3 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 02 10:13:57.952541 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a2244da-17cf-4ed0-9c66-3674717d5ad2 None None] REQUEST_METHOD: `POST` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:13:57.952781 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a2244da-17cf-4ed0-9c66-3674717d5ad2 None None] SCRIPT_NAME: `/identity` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:13:57.953089 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a2244da-17cf-4ed0-9c66-3674717d5ad2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:13:58.010966 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.auth.core [None req-4a2244da-17cf-4ed0-9c66-3674717d5ad2 None None] MFA Rules not processed for user `ae6e910e50d74f29bfb89bf18d95a908`. Rule list: `[]` (Enabled: `True`). {{(pid=80377) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 02 10:13:58.012782 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.fernet_utils [None req-4a2244da-17cf-4ed0-9c66-3674717d5ad2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80377) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 10:13:58.029865 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG dbcounter [-] [80377] Writing DB stats keystone:SELECT=100 {{(pid=80377) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 10:13:58.083973 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: [pid: 80377|app: 0|req: 769/1518] 10.222.0.79 () {58 vars in 1015 bytes} [Tue Aug 2 10:13:57 2022] POST /identity/v3/auth/tokens => generated 3308 bytes in 132 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 02 10:14:07.343809 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9f17414-0269-486b-87d1-e3908d1c0b47 None None] REQUEST_METHOD: `GET` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:14:07.344365 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9f17414-0269-486b-87d1-e3908d1c0b47 None None] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:14:07.344365 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9f17414-0269-486b-87d1-e3908d1c0b47 None None] PATH_INFO: `/` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:14:07.345746 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 750/1519] 10.222.0.82 () {54 vars in 961 bytes} [Tue Aug 2 10:14:07 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 02 10:14:07.351007 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e75c7b3-c084-4595-806d-1f2aa0181a40 None None] REQUEST_METHOD: `POST` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:14:07.351484 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e75c7b3-c084-4595-806d-1f2aa0181a40 None None] SCRIPT_NAME: `/identity` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:14:07.351592 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e75c7b3-c084-4595-806d-1f2aa0181a40 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:14:07.372723 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.auth.core [None req-6e75c7b3-c084-4595-806d-1f2aa0181a40 None None] MFA Rules not processed for user `ae6e910e50d74f29bfb89bf18d95a908`. Rule list: `[]` (Enabled: `True`). {{(pid=80377) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 02 10:14:07.374003 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.fernet_utils [None req-6e75c7b3-c084-4595-806d-1f2aa0181a40 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80377) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 10:14:07.383193 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: [pid: 80377|app: 0|req: 770/1520] 10.222.0.82 () {58 vars in 1058 bytes} [Tue Aug 2 10:14:07 2022] POST /identity/v3/auth/tokens => generated 3308 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 02 10:14:07.392203 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9f17414-0269-486b-87d1-e3908d1c0b47 None None] Authenticating user token {{(pid=80378) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:14:07.405944 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9f17414-0269-486b-87d1-e3908d1c0b47 None None] Validating token access rules against request {{(pid=80378) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:14:07.411191 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-961f789d-052e-4e86-b6e8-6d05f90678b9 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ae6e910e50d74f29bfb89bf18d95a908', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f1b38ebb804c41c69772c998aae72c41', 'project_domain_id': 'default', 'roles': ['admin', 'service', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80378) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:14:07.411798 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-961f789d-052e-4e86-b6e8-6d05f90678b9 service placement] REQUEST_METHOD: `GET` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:14:07.412082 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-961f789d-052e-4e86-b6e8-6d05f90678b9 service placement] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:14:07.412357 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-961f789d-052e-4e86-b6e8-6d05f90678b9 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:14:07.421205 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-961f789d-052e-4e86-b6e8-6d05f90678b9 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:14:07.423096 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-961f789d-052e-4e86-b6e8-6d05f90678b9 service placement] RBAC: Authorization granted {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:14:07.432677 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 751/1521] 10.222.0.82 () {60 vars in 1365 bytes} [Tue Aug 2 10:14:07 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 41 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 02 10:14:08.889647 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edab1547-60ed-4956-83e7-53b99fe06ef5 None None] REQUEST_METHOD: `GET` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:14:08.890235 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edab1547-60ed-4956-83e7-53b99fe06ef5 None None] SCRIPT_NAME: `/identity` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:14:08.890332 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edab1547-60ed-4956-83e7-53b99fe06ef5 None None] PATH_INFO: `/` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:14:08.891627 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: [pid: 80377|app: 0|req: 771/1522] 127.0.0.1 () {54 vars in 901 bytes} [Tue Aug 2 10:14:08 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 02 10:14:08.896785 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c522013-4667-4687-bb60-cbc133b754a8 None None] REQUEST_METHOD: `POST` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:14:08.897237 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c522013-4667-4687-bb60-cbc133b754a8 None None] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:14:08.897331 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c522013-4667-4687-bb60-cbc133b754a8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:14:08.917750 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.auth.core [None req-3c522013-4667-4687-bb60-cbc133b754a8 None None] MFA Rules not processed for user `891005fa5ff74041b75747d16cc66798`. Rule list: `[]` (Enabled: `True`). {{(pid=80378) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 02 10:14:08.918990 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.fernet_utils [None req-3c522013-4667-4687-bb60-cbc133b754a8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80378) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 10:14:08.927089 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 752/1523] 127.0.0.1 () {58 vars in 998 bytes} [Tue Aug 2 10:14:08 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 02 10:14:09.081277 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edab1547-60ed-4956-83e7-53b99fe06ef5 None None] Authenticating user token {{(pid=80377) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:14:09.096413 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edab1547-60ed-4956-83e7-53b99fe06ef5 None None] Validating token access rules against request {{(pid=80377) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:14:09.102134 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2a26470-e927-490a-a269-aac78e0482ec service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88af518196674fe593ad88d30111fb7d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f1b38ebb804c41c69772c998aae72c41', 'project_domain_id': 'default', 'roles': ['admin', 'ResellerAdmin', 'reader', 'service', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80377) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:14:09.102702 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2a26470-e927-490a-a269-aac78e0482ec service nova] REQUEST_METHOD: `GET` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:14:09.102989 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2a26470-e927-490a-a269-aac78e0482ec service nova] SCRIPT_NAME: `/identity` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:14:09.103270 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2a26470-e927-490a-a269-aac78e0482ec service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:14:09.111720 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2a26470-e927-490a-a269-aac78e0482ec service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:14:09.113317 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2a26470-e927-490a-a269-aac78e0482ec service nova] RBAC: Authorization granted {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:14:09.122671 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: [pid: 80377|app: 0|req: 772/1524] 10.222.0.82 () {60 vars in 1365 bytes} [Tue Aug 2 10:14:09 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 42 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 02 10:14:09.953283 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e671605-a59a-4cb1-b956-f7b419babefa None None] REQUEST_METHOD: `GET` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:14:09.953866 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e671605-a59a-4cb1-b956-f7b419babefa None None] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:14:09.953866 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e671605-a59a-4cb1-b956-f7b419babefa None None] PATH_INFO: `/` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:14:09.955216 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 753/1525] 127.0.0.1 () {54 vars in 901 bytes} [Tue Aug 2 10:14:09 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 02 10:14:09.960084 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ed4be2e-a354-49df-929f-0c0db355f022 None None] REQUEST_METHOD: `POST` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:14:09.960501 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ed4be2e-a354-49df-929f-0c0db355f022 None None] SCRIPT_NAME: `/identity` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:14:09.960665 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ed4be2e-a354-49df-929f-0c0db355f022 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:14:09.981695 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.auth.core [None req-4ed4be2e-a354-49df-929f-0c0db355f022 None None] MFA Rules not processed for user `891005fa5ff74041b75747d16cc66798`. Rule list: `[]` (Enabled: `True`). {{(pid=80377) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 02 10:14:09.982914 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.fernet_utils [None req-4ed4be2e-a354-49df-929f-0c0db355f022 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80377) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 10:14:09.990724 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: [pid: 80377|app: 0|req: 773/1526] 127.0.0.1 () {58 vars in 998 bytes} [Tue Aug 2 10:14:09 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 02 10:14:10.143642 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e671605-a59a-4cb1-b956-f7b419babefa None None] Authenticating user token {{(pid=80378) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:14:10.158549 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e671605-a59a-4cb1-b956-f7b419babefa None None] Validating token access rules against request {{(pid=80378) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:14:10.164334 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c63d5c56-27c5-4487-aac5-c8a69aa83312 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88af518196674fe593ad88d30111fb7d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f1b38ebb804c41c69772c998aae72c41', 'project_domain_id': 'default', 'roles': ['admin', 'ResellerAdmin', 'reader', 'service', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80378) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:14:10.165006 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c63d5c56-27c5-4487-aac5-c8a69aa83312 service nova] REQUEST_METHOD: `GET` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:14:10.165255 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c63d5c56-27c5-4487-aac5-c8a69aa83312 service nova] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:14:10.165527 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c63d5c56-27c5-4487-aac5-c8a69aa83312 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:14:10.174162 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c63d5c56-27c5-4487-aac5-c8a69aa83312 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:14:10.175743 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c63d5c56-27c5-4487-aac5-c8a69aa83312 service nova] RBAC: Authorization granted {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:14:10.186824 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 754/1527] 10.222.0.82 () {60 vars in 1365 bytes} [Tue Aug 2 10:14:10 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 44 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 02 10:14:14.313453 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45452274-125a-4a78-b225-992a2b510f2a None None] REQUEST_METHOD: `GET` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:14:14.314061 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45452274-125a-4a78-b225-992a2b510f2a None None] SCRIPT_NAME: `/identity` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:14:14.314061 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45452274-125a-4a78-b225-992a2b510f2a None None] PATH_INFO: `/` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:14:14.315401 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: [pid: 80377|app: 0|req: 774/1528] 127.0.0.1 () {54 vars in 916 bytes} [Tue Aug 2 10:14:14 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 02 10:14:14.320596 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c352edc1-7566-44e3-b93f-64ed77fc5cc1 None None] REQUEST_METHOD: `POST` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:14:14.320999 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c352edc1-7566-44e3-b93f-64ed77fc5cc1 None None] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:14:14.321154 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c352edc1-7566-44e3-b93f-64ed77fc5cc1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:14:14.341396 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.auth.core [None req-c352edc1-7566-44e3-b93f-64ed77fc5cc1 None None] MFA Rules not processed for user `891005fa5ff74041b75747d16cc66798`. Rule list: `[]` (Enabled: `True`). {{(pid=80378) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 02 10:14:14.342666 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.fernet_utils [None req-c352edc1-7566-44e3-b93f-64ed77fc5cc1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80378) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 10:14:14.350645 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 755/1529] 127.0.0.1 () {58 vars in 1013 bytes} [Tue Aug 2 10:14:14 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 02 10:14:14.429338 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45452274-125a-4a78-b225-992a2b510f2a None None] Authenticating user token {{(pid=80377) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:14:14.431150 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.fernet_utils [None req-45452274-125a-4a78-b225-992a2b510f2a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80377) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 10:14:14.509556 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45452274-125a-4a78-b225-992a2b510f2a None None] Validating token access rules against request {{(pid=80377) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:14:14.513775 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f9aa6aa-037d-41f0-af2c-4a80773d8b46 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '33b87170f0634b67bce60750847b5cbe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f1b38ebb804c41c69772c998aae72c41', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80377) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:14:14.514418 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f9aa6aa-037d-41f0-af2c-4a80773d8b46 service neutron] REQUEST_METHOD: `GET` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:14:14.514703 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f9aa6aa-037d-41f0-af2c-4a80773d8b46 service neutron] SCRIPT_NAME: `/identity` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:14:14.514981 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f9aa6aa-037d-41f0-af2c-4a80773d8b46 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:14:14.523885 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f9aa6aa-037d-41f0-af2c-4a80773d8b46 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:14:14.525334 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f9aa6aa-037d-41f0-af2c-4a80773d8b46 service neutron] RBAC: Authorization granted {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:14:14.535653 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: [pid: 80377|app: 0|req: 775/1530] 10.222.0.82 () {60 vars in 1365 bytes} [Tue Aug 2 10:14:14 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 107 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 02 10:14:16.563145 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccff6248-374e-47dd-bff7-e246de6dfcfd None None] REQUEST_METHOD: `GET` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:14:16.563738 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccff6248-374e-47dd-bff7-e246de6dfcfd None None] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:14:16.563738 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccff6248-374e-47dd-bff7-e246de6dfcfd None None] PATH_INFO: `/` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:14:16.565556 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 756/1531] 127.0.0.1 () {54 vars in 916 bytes} [Tue Aug 2 10:14:16 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 02 10:14:16.570492 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccc83976-59e2-4bdc-a226-2a17ad4636e8 None None] REQUEST_METHOD: `POST` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:14:16.571239 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccc83976-59e2-4bdc-a226-2a17ad4636e8 None None] SCRIPT_NAME: `/identity` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:14:16.571239 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccc83976-59e2-4bdc-a226-2a17ad4636e8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:14:16.591784 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.auth.core [None req-ccc83976-59e2-4bdc-a226-2a17ad4636e8 None None] MFA Rules not processed for user `891005fa5ff74041b75747d16cc66798`. Rule list: `[]` (Enabled: `True`). {{(pid=80377) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 02 10:14:16.593000 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.fernet_utils [None req-ccc83976-59e2-4bdc-a226-2a17ad4636e8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80377) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 10:14:16.601145 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: [pid: 80377|app: 0|req: 776/1532] 127.0.0.1 () {58 vars in 1013 bytes} [Tue Aug 2 10:14:16 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 02 10:14:16.637680 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ccff6248-374e-47dd-bff7-e246de6dfcfd None None] Authenticating user token {{(pid=80378) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:14:16.639808 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.fernet_utils [None req-ccff6248-374e-47dd-bff7-e246de6dfcfd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80378) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 10:14:16.653524 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ccff6248-374e-47dd-bff7-e246de6dfcfd None None] Validating token access rules against request {{(pid=80378) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:14:16.657091 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d0cbc63-3726-43b7-9814-ae8a622db0b4 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '33b87170f0634b67bce60750847b5cbe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f1b38ebb804c41c69772c998aae72c41', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80378) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:14:16.657682 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d0cbc63-3726-43b7-9814-ae8a622db0b4 service neutron] REQUEST_METHOD: `GET` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:14:16.657990 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d0cbc63-3726-43b7-9814-ae8a622db0b4 service neutron] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:14:16.658316 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d0cbc63-3726-43b7-9814-ae8a622db0b4 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:14:16.667157 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d0cbc63-3726-43b7-9814-ae8a622db0b4 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:14:16.668597 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d0cbc63-3726-43b7-9814-ae8a622db0b4 service neutron] RBAC: Authorization granted {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:14:16.678092 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 757/1533] 10.222.0.82 () {60 vars in 1365 bytes} [Tue Aug 2 10:14:16 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 42 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 02 10:14:20.411530 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56e37c4f-fad5-4e85-8e02-11a801953a30 None None] REQUEST_METHOD: `GET` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:14:20.412101 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56e37c4f-fad5-4e85-8e02-11a801953a30 None None] SCRIPT_NAME: `/identity` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:14:20.412101 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56e37c4f-fad5-4e85-8e02-11a801953a30 None None] PATH_INFO: `/` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:14:20.413443 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: [pid: 80377|app: 0|req: 777/1534] 127.0.0.1 () {54 vars in 916 bytes} [Tue Aug 2 10:14:20 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 02 10:14:20.418882 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9629a8a8-d6c1-45d1-93cb-f3297c9e6b20 None None] REQUEST_METHOD: `POST` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:14:20.419693 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9629a8a8-d6c1-45d1-93cb-f3297c9e6b20 None None] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:14:20.419859 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9629a8a8-d6c1-45d1-93cb-f3297c9e6b20 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:14:20.439289 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.auth.core [None req-9629a8a8-d6c1-45d1-93cb-f3297c9e6b20 None None] MFA Rules not processed for user `891005fa5ff74041b75747d16cc66798`. Rule list: `[]` (Enabled: `True`). {{(pid=80378) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 02 10:14:20.440498 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.fernet_utils [None req-9629a8a8-d6c1-45d1-93cb-f3297c9e6b20 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80378) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 10:14:20.449313 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 758/1535] 127.0.0.1 () {58 vars in 1013 bytes} [Tue Aug 2 10:14:20 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 02 10:14:20.757408 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56e37c4f-fad5-4e85-8e02-11a801953a30 None None] Authenticating user token {{(pid=80377) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:14:20.759277 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.fernet_utils [None req-56e37c4f-fad5-4e85-8e02-11a801953a30 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80377) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 10:14:20.824293 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG dbcounter [-] [80377] Writing DB stats keystone:SELECT=100 {{(pid=80377) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 10:14:20.838603 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56e37c4f-fad5-4e85-8e02-11a801953a30 None None] Validating token access rules against request {{(pid=80377) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:14:20.842183 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a559d9a-c3c3-49f1-b5df-33a80e312583 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1023de281ae74e9d8a6d6c312c1c97a7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f1b38ebb804c41c69772c998aae72c41', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80377) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:14:20.842851 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a559d9a-c3c3-49f1-b5df-33a80e312583 service cinder] REQUEST_METHOD: `GET` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:14:20.843137 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a559d9a-c3c3-49f1-b5df-33a80e312583 service cinder] SCRIPT_NAME: `/identity` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:14:20.843409 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a559d9a-c3c3-49f1-b5df-33a80e312583 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:14:20.852879 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a559d9a-c3c3-49f1-b5df-33a80e312583 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:14:20.854377 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a559d9a-c3c3-49f1-b5df-33a80e312583 service cinder] RBAC: Authorization granted {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:14:20.864204 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: [pid: 80377|app: 0|req: 778/1536] 10.222.0.82 () {60 vars in 1365 bytes} [Tue Aug 2 10:14:20 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 107 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 02 10:14:22.923608 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28635de8-daa8-418f-a4d0-5c13ed2a129c None None] REQUEST_METHOD: `GET` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:14:22.924166 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28635de8-daa8-418f-a4d0-5c13ed2a129c None None] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:14:22.924166 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28635de8-daa8-418f-a4d0-5c13ed2a129c None None] PATH_INFO: `/` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:14:22.925519 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 759/1537] 127.0.0.1 () {54 vars in 916 bytes} [Tue Aug 2 10:14:22 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 02 10:14:22.930764 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a2a7906-ef1e-47a6-a424-3e708c90ced0 None None] REQUEST_METHOD: `POST` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:14:22.931163 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a2a7906-ef1e-47a6-a424-3e708c90ced0 None None] SCRIPT_NAME: `/identity` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:14:22.931318 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a2a7906-ef1e-47a6-a424-3e708c90ced0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:14:22.951542 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.auth.core [None req-4a2a7906-ef1e-47a6-a424-3e708c90ced0 None None] MFA Rules not processed for user `891005fa5ff74041b75747d16cc66798`. Rule list: `[]` (Enabled: `True`). {{(pid=80377) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 02 10:14:22.952767 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.fernet_utils [None req-4a2a7906-ef1e-47a6-a424-3e708c90ced0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80377) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 10:14:22.960587 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: [pid: 80377|app: 0|req: 779/1538] 127.0.0.1 () {58 vars in 1013 bytes} [Tue Aug 2 10:14:22 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 02 10:14:23.281063 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28635de8-daa8-418f-a4d0-5c13ed2a129c None None] Authenticating user token {{(pid=80378) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:14:23.283737 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.fernet_utils [None req-28635de8-daa8-418f-a4d0-5c13ed2a129c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80378) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 10:14:23.297416 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG dbcounter [-] [80378] Writing DB stats keystone:SELECT=84 {{(pid=80378) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 10:14:23.304507 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28635de8-daa8-418f-a4d0-5c13ed2a129c None None] Validating token access rules against request {{(pid=80378) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:14:23.308057 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-124da7ef-39e4-47f1-8ea1-47628e32da95 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1023de281ae74e9d8a6d6c312c1c97a7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f1b38ebb804c41c69772c998aae72c41', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80378) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:14:23.308845 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-124da7ef-39e4-47f1-8ea1-47628e32da95 service cinder] REQUEST_METHOD: `GET` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:14:23.310247 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-124da7ef-39e4-47f1-8ea1-47628e32da95 service cinder] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:14:23.310528 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-124da7ef-39e4-47f1-8ea1-47628e32da95 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:14:23.320997 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-124da7ef-39e4-47f1-8ea1-47628e32da95 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:14:23.322503 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-124da7ef-39e4-47f1-8ea1-47628e32da95 service cinder] RBAC: Authorization granted {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:14:23.332482 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 760/1539] 10.222.0.82 () {60 vars in 1365 bytes} [Tue Aug 2 10:14:23 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 52 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 02 10:14:32.946442 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG dbcounter [-] [80377] Writing DB stats keystone:SELECT=11 {{(pid=80377) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 10:14:33.328057 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG dbcounter [-] [80378] Writing DB stats keystone:SELECT=5 {{(pid=80378) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 10:15:56.401226 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0c0a59f-4687-48ee-8b6f-3705ad7f7fc0 None None] REQUEST_METHOD: `POST` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:15:56.401894 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0c0a59f-4687-48ee-8b6f-3705ad7f7fc0 None None] SCRIPT_NAME: `/identity` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:15:56.401894 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0c0a59f-4687-48ee-8b6f-3705ad7f7fc0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:15:56.426918 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.auth.core [None req-a0c0a59f-4687-48ee-8b6f-3705ad7f7fc0 None None] MFA Rules not processed for user `891005fa5ff74041b75747d16cc66798`. Rule list: `[]` (Enabled: `True`). {{(pid=80377) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 02 10:15:56.428518 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.fernet_utils [None req-a0c0a59f-4687-48ee-8b6f-3705ad7f7fc0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80377) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 10:15:56.441619 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: [pid: 80377|app: 0|req: 780/1540] 10.222.0.82 () {58 vars in 958 bytes} [Tue Aug 2 10:15:56 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 42 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 02 10:15:56.451904 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1cd375f-9e56-4429-800a-453bd0ea0225 None None] REQUEST_METHOD: `POST` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:15:56.452913 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1cd375f-9e56-4429-800a-453bd0ea0225 None None] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:15:56.453415 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1cd375f-9e56-4429-800a-453bd0ea0225 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:15:56.478281 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.auth.core [None req-c1cd375f-9e56-4429-800a-453bd0ea0225 None None] MFA Rules not processed for user `891005fa5ff74041b75747d16cc66798`. Rule list: `[]` (Enabled: `True`). {{(pid=80378) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 02 10:15:56.479872 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.fernet_utils [None req-c1cd375f-9e56-4429-800a-453bd0ea0225 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80378) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 10:15:56.492957 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 761/1541] 10.222.0.82 () {58 vars in 958 bytes} [Tue Aug 2 10:15:56 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 42 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 02 10:15:56.498231 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0c0a59f-4687-48ee-8b6f-3705ad7f7fc0 None None] Authenticating user token {{(pid=80377) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:15:56.509567 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0c0a59f-4687-48ee-8b6f-3705ad7f7fc0 None None] Validating token access rules against request {{(pid=80377) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:15:56.520788 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e65eb297-fcd0-4837-882f-63b09df48464 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '891005fa5ff74041b75747d16cc66798', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7047912c5acd4adeba5d87c3bf895e39', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80377) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:15:56.521901 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e65eb297-fcd0-4837-882f-63b09df48464 admin admin] REQUEST_METHOD: `GET` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:15:56.522415 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e65eb297-fcd0-4837-882f-63b09df48464 admin admin] SCRIPT_NAME: `/identity` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:15:56.522919 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e65eb297-fcd0-4837-882f-63b09df48464 admin admin] PATH_INFO: `/v3/domains` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:15:56.523883 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e65eb297-fcd0-4837-882f-63b09df48464 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80377) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 02 10:15:56.524423 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e65eb297-fcd0-4837-882f-63b09df48464 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:15:56.526568 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e65eb297-fcd0-4837-882f-63b09df48464 admin admin] RBAC: Authorization granted {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:15:56.545887 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: [pid: 80377|app: 0|req: 781/1542] 10.222.0.82 () {58 vars in 1153 bytes} [Tue Aug 2 10:15:56 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 48 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 02 10:15:56.554953 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1cd375f-9e56-4429-800a-453bd0ea0225 None None] Authenticating user token {{(pid=80378) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:15:56.566118 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1cd375f-9e56-4429-800a-453bd0ea0225 None None] Validating token access rules against request {{(pid=80378) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:15:56.572513 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9560b786-2194-49bc-8cbd-516e9ba26a8a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '891005fa5ff74041b75747d16cc66798', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7047912c5acd4adeba5d87c3bf895e39', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80378) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:15:56.573539 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9560b786-2194-49bc-8cbd-516e9ba26a8a admin admin] REQUEST_METHOD: `POST` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:15:56.574112 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9560b786-2194-49bc-8cbd-516e9ba26a8a admin admin] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:15:56.574623 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9560b786-2194-49bc-8cbd-516e9ba26a8a admin admin] PATH_INFO: `/v3/projects` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:15:56.576066 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9560b786-2194-49bc-8cbd-516e9ba26a8a admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:15:56.578334 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9560b786-2194-49bc-8cbd-516e9ba26a8a admin admin] RBAC: Authorization granted {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:15:56.623783 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 762/1543] 10.222.0.82 () {60 vars in 1153 bytes} [Tue Aug 2 10:15:56 2022] POST /identity/v3/projects => generated 377 bytes in 69 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 02 10:15:56.628459 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e65eb297-fcd0-4837-882f-63b09df48464 admin admin] Authenticating user token {{(pid=80377) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:15:56.643509 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e77c6aa0-524c-457b-b6d9-ff094a5181e1 None None] REQUEST_METHOD: `POST` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:15:56.644796 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e65eb297-fcd0-4837-882f-63b09df48464 admin admin] Validating token access rules against request {{(pid=80377) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:15:56.646114 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e77c6aa0-524c-457b-b6d9-ff094a5181e1 None None] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:15:56.646738 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e77c6aa0-524c-457b-b6d9-ff094a5181e1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:15:56.692112 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.auth.core [None req-e77c6aa0-524c-457b-b6d9-ff094a5181e1 None None] MFA Rules not processed for user `891005fa5ff74041b75747d16cc66798`. Rule list: `[]` (Enabled: `True`). {{(pid=80378) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 02 10:15:56.693770 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.fernet_utils [None req-e77c6aa0-524c-457b-b6d9-ff094a5181e1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80378) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 10:15:56.788879 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbc9ef28-f2ce-4baa-8cd3-2d3acdbcceb0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '891005fa5ff74041b75747d16cc66798', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7047912c5acd4adeba5d87c3bf895e39', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80377) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:15:56.789988 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbc9ef28-f2ce-4baa-8cd3-2d3acdbcceb0 admin admin] REQUEST_METHOD: `POST` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:15:56.790821 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbc9ef28-f2ce-4baa-8cd3-2d3acdbcceb0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:15:56.791136 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbc9ef28-f2ce-4baa-8cd3-2d3acdbcceb0 admin admin] PATH_INFO: `/v3/users` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:15:56.792142 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cbc9ef28-f2ce-4baa-8cd3-2d3acdbcceb0 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:15:56.795088 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cbc9ef28-f2ce-4baa-8cd3-2d3acdbcceb0 admin admin] RBAC: Authorization granted {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:15:56.866903 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: [pid: 80377|app: 0|req: 782/1544] 10.222.0.82 () {60 vars in 1144 bytes} [Tue Aug 2 10:15:56 2022] POST /identity/v3/users => generated 289 bytes in 231 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 02 10:15:56.868980 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aeb00b6f-1337-4a9d-a004-31884871a389 None None] REQUEST_METHOD: `POST` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:15:56.869401 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aeb00b6f-1337-4a9d-a004-31884871a389 None None] SCRIPT_NAME: `/identity` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:15:56.869730 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aeb00b6f-1337-4a9d-a004-31884871a389 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:15:56.882362 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 763/1545] 10.222.0.82 () {58 vars in 958 bytes} [Tue Aug 2 10:15:56 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 240 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 02 10:15:56.883871 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47559f55-5195-4446-93a3-f4057b2a1591 None None] REQUEST_METHOD: `POST` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:15:56.884176 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47559f55-5195-4446-93a3-f4057b2a1591 None None] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:15:56.884538 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47559f55-5195-4446-93a3-f4057b2a1591 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:15:56.924851 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.auth.core [None req-aeb00b6f-1337-4a9d-a004-31884871a389 None None] MFA Rules not processed for user `891005fa5ff74041b75747d16cc66798`. Rule list: `[]` (Enabled: `True`). {{(pid=80377) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 02 10:15:56.926512 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.fernet_utils [None req-aeb00b6f-1337-4a9d-a004-31884871a389 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80377) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 10:15:56.938050 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: [pid: 80377|app: 0|req: 783/1546] 10.222.0.82 () {58 vars in 958 bytes} [Tue Aug 2 10:15:56 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 71 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 02 10:15:56.938916 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aeb00b6f-1337-4a9d-a004-31884871a389 None None] Authenticating user token {{(pid=80377) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:15:56.945455 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.auth.core [None req-47559f55-5195-4446-93a3-f4057b2a1591 None None] MFA Rules not processed for user `891005fa5ff74041b75747d16cc66798`. Rule list: `[]` (Enabled: `True`). {{(pid=80378) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 02 10:15:56.950074 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.fernet_utils [None req-47559f55-5195-4446-93a3-f4057b2a1591 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80378) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 10:15:56.960500 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aeb00b6f-1337-4a9d-a004-31884871a389 None None] Validating token access rules against request {{(pid=80377) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:15:56.963966 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 764/1547] 10.222.0.82 () {58 vars in 958 bytes} [Tue Aug 2 10:15:56 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 81 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 02 10:15:56.965259 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30b73172-9320-4fe4-9ba8-62ee310b7e5e None None] REQUEST_METHOD: `POST` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:15:56.966260 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30b73172-9320-4fe4-9ba8-62ee310b7e5e None None] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:15:56.966402 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2eb49ebf-66ad-488c-9e18-ae4b09abac2d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '891005fa5ff74041b75747d16cc66798', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7047912c5acd4adeba5d87c3bf895e39', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80377) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:15:56.966536 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30b73172-9320-4fe4-9ba8-62ee310b7e5e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:15:56.967025 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2eb49ebf-66ad-488c-9e18-ae4b09abac2d admin admin] REQUEST_METHOD: `GET` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:15:56.967305 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2eb49ebf-66ad-488c-9e18-ae4b09abac2d admin admin] SCRIPT_NAME: `/identity` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:15:56.967577 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2eb49ebf-66ad-488c-9e18-ae4b09abac2d admin admin] PATH_INFO: `/v3/roles` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:15:56.968338 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2eb49ebf-66ad-488c-9e18-ae4b09abac2d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:15:56.970014 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2eb49ebf-66ad-488c-9e18-ae4b09abac2d admin admin] RBAC: Authorization granted {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:15:56.988053 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: [pid: 80377|app: 0|req: 784/1548] 10.222.0.82 () {58 vars in 1122 bytes} [Tue Aug 2 10:15:56 2022] GET /identity/v3/roles => generated 1420 bytes in 49 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 02 10:15:56.989701 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fdccdba-2ceb-48a7-93fa-7035fbef7fdc None None] REQUEST_METHOD: `POST` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:15:56.990845 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fdccdba-2ceb-48a7-93fa-7035fbef7fdc None None] SCRIPT_NAME: `/identity` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:15:56.991428 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fdccdba-2ceb-48a7-93fa-7035fbef7fdc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:15:56.997766 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.auth.core [None req-30b73172-9320-4fe4-9ba8-62ee310b7e5e None None] MFA Rules not processed for user `891005fa5ff74041b75747d16cc66798`. Rule list: `[]` (Enabled: `True`). {{(pid=80378) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 02 10:15:56.999010 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.fernet_utils [None req-30b73172-9320-4fe4-9ba8-62ee310b7e5e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80378) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 10:15:57.009360 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 765/1549] 10.222.0.82 () {58 vars in 958 bytes} [Tue Aug 2 10:15:56 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 45 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 02 10:15:57.011340 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49e03bbb-1edc-4412-ae95-d1098ddea423 None None] REQUEST_METHOD: `POST` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:15:57.011864 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49e03bbb-1edc-4412-ae95-d1098ddea423 None None] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:15:57.012963 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49e03bbb-1edc-4412-ae95-d1098ddea423 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:15:57.014341 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.auth.core [None req-6fdccdba-2ceb-48a7-93fa-7035fbef7fdc None None] MFA Rules not processed for user `891005fa5ff74041b75747d16cc66798`. Rule list: `[]` (Enabled: `True`). {{(pid=80377) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 02 10:15:57.015626 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.fernet_utils [None req-6fdccdba-2ceb-48a7-93fa-7035fbef7fdc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80377) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 10:15:57.025111 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: [pid: 80377|app: 0|req: 785/1550] 10.222.0.82 () {58 vars in 958 bytes} [Tue Aug 2 10:15:56 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 36 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 02 10:15:57.026587 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28a12e66-d571-418c-b720-bd2f8649b54d None None] REQUEST_METHOD: `POST` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:15:57.026879 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28a12e66-d571-418c-b720-bd2f8649b54d None None] SCRIPT_NAME: `/identity` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:15:57.027157 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28a12e66-d571-418c-b720-bd2f8649b54d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:15:57.037682 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.auth.core [None req-49e03bbb-1edc-4412-ae95-d1098ddea423 None None] MFA Rules not processed for user `891005fa5ff74041b75747d16cc66798`. Rule list: `[]` (Enabled: `True`). {{(pid=80378) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 02 10:15:57.038957 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.fernet_utils [None req-49e03bbb-1edc-4412-ae95-d1098ddea423 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80378) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 10:15:57.048098 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.auth.core [None req-28a12e66-d571-418c-b720-bd2f8649b54d None None] MFA Rules not processed for user `891005fa5ff74041b75747d16cc66798`. Rule list: `[]` (Enabled: `True`). {{(pid=80377) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 02 10:15:57.048411 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 766/1551] 10.222.0.82 () {58 vars in 958 bytes} [Tue Aug 2 10:15:57 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 38 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 02 10:15:57.049381 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.fernet_utils [None req-28a12e66-d571-418c-b720-bd2f8649b54d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80377) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 10:15:57.049669 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c510b37-4281-4ad6-a449-13e2d06f2169 None None] REQUEST_METHOD: `POST` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:15:57.050283 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c510b37-4281-4ad6-a449-13e2d06f2169 None None] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:15:57.050572 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c510b37-4281-4ad6-a449-13e2d06f2169 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:15:57.058177 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: [pid: 80377|app: 0|req: 786/1552] 10.222.0.82 () {58 vars in 958 bytes} [Tue Aug 2 10:15:57 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 02 10:15:57.059210 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28a12e66-d571-418c-b720-bd2f8649b54d None None] Authenticating user token {{(pid=80377) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:15:57.071037 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28a12e66-d571-418c-b720-bd2f8649b54d None None] Validating token access rules against request {{(pid=80377) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:15:57.073449 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.auth.core [None req-0c510b37-4281-4ad6-a449-13e2d06f2169 None None] MFA Rules not processed for user `891005fa5ff74041b75747d16cc66798`. Rule list: `[]` (Enabled: `True`). {{(pid=80378) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 02 10:15:57.074711 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.fernet_utils [None req-0c510b37-4281-4ad6-a449-13e2d06f2169 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80378) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 10:15:57.076671 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4898f6a-690c-4964-9230-14d20f377c04 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '891005fa5ff74041b75747d16cc66798', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7047912c5acd4adeba5d87c3bf895e39', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80377) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:15:57.077302 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4898f6a-690c-4964-9230-14d20f377c04 admin admin] REQUEST_METHOD: `GET` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:15:57.077583 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4898f6a-690c-4964-9230-14d20f377c04 admin admin] SCRIPT_NAME: `/identity` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:15:57.077899 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4898f6a-690c-4964-9230-14d20f377c04 admin admin] PATH_INFO: `/v3/roles` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:15:57.078604 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4898f6a-690c-4964-9230-14d20f377c04 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:15:57.080229 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4898f6a-690c-4964-9230-14d20f377c04 admin admin] RBAC: Authorization granted {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:15:57.083295 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 767/1553] 10.222.0.82 () {58 vars in 958 bytes} [Tue Aug 2 10:15:57 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 34 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 02 10:15:57.084502 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b41e4efd-7cac-41ba-bd5c-ab97fbbe3c59 None None] REQUEST_METHOD: `POST` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:15:57.084791 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b41e4efd-7cac-41ba-bd5c-ab97fbbe3c59 None None] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:15:57.085070 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b41e4efd-7cac-41ba-bd5c-ab97fbbe3c59 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:15:57.096242 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: [pid: 80377|app: 0|req: 787/1554] 10.222.0.82 () {58 vars in 1122 bytes} [Tue Aug 2 10:15:57 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 02 10:15:57.096242 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4898f6a-690c-4964-9230-14d20f377c04 admin admin] Authenticating user token {{(pid=80377) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:15:57.104848 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4898f6a-690c-4964-9230-14d20f377c04 admin admin] Validating token access rules against request {{(pid=80377) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:15:57.106787 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.auth.core [None req-b41e4efd-7cac-41ba-bd5c-ab97fbbe3c59 None None] MFA Rules not processed for user `891005fa5ff74041b75747d16cc66798`. Rule list: `[]` (Enabled: `True`). {{(pid=80378) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 02 10:15:57.108003 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.fernet_utils [None req-b41e4efd-7cac-41ba-bd5c-ab97fbbe3c59 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80378) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 10:15:57.109509 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ca74bdd-db26-414f-b8b0-ba45f5f55d05 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '891005fa5ff74041b75747d16cc66798', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7047912c5acd4adeba5d87c3bf895e39', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80377) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:15:57.110229 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ca74bdd-db26-414f-b8b0-ba45f5f55d05 admin admin] REQUEST_METHOD: `GET` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:15:57.110515 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ca74bdd-db26-414f-b8b0-ba45f5f55d05 admin admin] SCRIPT_NAME: `/identity` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:15:57.110794 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ca74bdd-db26-414f-b8b0-ba45f5f55d05 admin admin] PATH_INFO: `/v3/domains` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:15:57.111492 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ca74bdd-db26-414f-b8b0-ba45f5f55d05 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80377) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 02 10:15:57.111805 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ca74bdd-db26-414f-b8b0-ba45f5f55d05 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:15:57.113428 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ca74bdd-db26-414f-b8b0-ba45f5f55d05 admin admin] RBAC: Authorization granted {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:15:57.116056 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 768/1555] 10.222.0.82 () {58 vars in 958 bytes} [Tue Aug 2 10:15:57 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 02 10:15:57.117331 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cf62cfc-6c11-4acf-b6e8-018cde0091c1 None None] REQUEST_METHOD: `POST` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:15:57.117620 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cf62cfc-6c11-4acf-b6e8-018cde0091c1 None None] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:15:57.117933 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cf62cfc-6c11-4acf-b6e8-018cde0091c1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:15:57.126166 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: [pid: 80377|app: 0|req: 788/1556] 10.222.0.82 () {58 vars in 1153 bytes} [Tue Aug 2 10:15:57 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 02 10:15:57.127415 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8ffe5bb-2f8b-4ba2-9433-a82fca0df7ec None None] REQUEST_METHOD: `POST` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:15:57.127702 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8ffe5bb-2f8b-4ba2-9433-a82fca0df7ec None None] SCRIPT_NAME: `/identity` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:15:57.127979 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8ffe5bb-2f8b-4ba2-9433-a82fca0df7ec None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:15:57.143164 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.auth.core [None req-1cf62cfc-6c11-4acf-b6e8-018cde0091c1 None None] MFA Rules not processed for user `891005fa5ff74041b75747d16cc66798`. Rule list: `[]` (Enabled: `True`). {{(pid=80378) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 02 10:15:57.144292 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.fernet_utils [None req-1cf62cfc-6c11-4acf-b6e8-018cde0091c1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80378) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 10:15:57.147414 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.auth.core [None req-e8ffe5bb-2f8b-4ba2-9433-a82fca0df7ec None None] MFA Rules not processed for user `891005fa5ff74041b75747d16cc66798`. Rule list: `[]` (Enabled: `True`). {{(pid=80377) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 02 10:15:57.148588 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.fernet_utils [None req-e8ffe5bb-2f8b-4ba2-9433-a82fca0df7ec None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80377) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 10:15:57.152415 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 769/1557] 10.222.0.82 () {58 vars in 958 bytes} [Tue Aug 2 10:15:57 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 36 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 02 10:15:57.153877 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1cf62cfc-6c11-4acf-b6e8-018cde0091c1 None None] Authenticating user token {{(pid=80378) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:15:57.158006 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: [pid: 80377|app: 0|req: 789/1558] 10.222.0.82 () {58 vars in 958 bytes} [Tue Aug 2 10:15:57 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 02 10:15:57.159067 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6316fbe7-b388-442f-b727-0d1ce7a79a86 None None] REQUEST_METHOD: `POST` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:15:57.159356 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6316fbe7-b388-442f-b727-0d1ce7a79a86 None None] SCRIPT_NAME: `/identity` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:15:57.159633 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6316fbe7-b388-442f-b727-0d1ce7a79a86 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:15:57.167843 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1cf62cfc-6c11-4acf-b6e8-018cde0091c1 None None] Validating token access rules against request {{(pid=80378) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:15:57.172327 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f658c8a-b570-42ee-97d4-587325f8934c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '891005fa5ff74041b75747d16cc66798', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7047912c5acd4adeba5d87c3bf895e39', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80378) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:15:57.172946 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f658c8a-b570-42ee-97d4-587325f8934c admin admin] REQUEST_METHOD: `GET` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:15:57.173228 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f658c8a-b570-42ee-97d4-587325f8934c admin admin] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:15:57.173497 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f658c8a-b570-42ee-97d4-587325f8934c admin admin] PATH_INFO: `/v3/domains` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:15:57.174241 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f658c8a-b570-42ee-97d4-587325f8934c admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80378) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 02 10:15:57.174545 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f658c8a-b570-42ee-97d4-587325f8934c admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:15:57.175978 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: WARNING py.warnings [None req-6f658c8a-b570-42ee-97d4-587325f8934c admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:list_domains": "role:reader and system_scope:all" failed scope check. The token used to make the request was project scoped but the policy requires ['system'] scope. This behavior may change in the future where using the intended scope is required Aug 02 10:15:57.175978 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: warnings.warn(msg) Aug 02 10:15:57.175978 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: Aug 02 10:15:57.176861 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f658c8a-b570-42ee-97d4-587325f8934c admin admin] RBAC: Authorization granted {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:15:57.180115 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.auth.core [None req-6316fbe7-b388-442f-b727-0d1ce7a79a86 None None] MFA Rules not processed for user `891005fa5ff74041b75747d16cc66798`. Rule list: `[]` (Enabled: `True`). {{(pid=80377) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 02 10:15:57.181330 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.fernet_utils [None req-6316fbe7-b388-442f-b727-0d1ce7a79a86 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80377) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 10:15:57.196256 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 770/1559] 10.222.0.82 () {58 vars in 1153 bytes} [Tue Aug 2 10:15:57 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 02 10:15:57.196515 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: [pid: 80377|app: 0|req: 790/1560] 10.222.0.82 () {58 vars in 958 bytes} [Tue Aug 2 10:15:57 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 02 10:15:57.196706 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f658c8a-b570-42ee-97d4-587325f8934c admin admin] Authenticating user token {{(pid=80378) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:15:57.196807 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6316fbe7-b388-442f-b727-0d1ce7a79a86 None None] Authenticating user token {{(pid=80377) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:15:57.203051 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6316fbe7-b388-442f-b727-0d1ce7a79a86 None None] Validating token access rules against request {{(pid=80377) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:15:57.203352 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f658c8a-b570-42ee-97d4-587325f8934c admin admin] Validating token access rules against request {{(pid=80378) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:15:57.207684 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fcd980b-3136-4510-967f-fd1b2174b079 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '891005fa5ff74041b75747d16cc66798', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7047912c5acd4adeba5d87c3bf895e39', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80377) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:15:57.207941 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06a676d6-c9a7-4f25-b854-16fd189bb61d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '891005fa5ff74041b75747d16cc66798', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7047912c5acd4adeba5d87c3bf895e39', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80378) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:15:57.208357 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fcd980b-3136-4510-967f-fd1b2174b079 admin admin] REQUEST_METHOD: `PUT` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:15:57.208551 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06a676d6-c9a7-4f25-b854-16fd189bb61d admin admin] REQUEST_METHOD: `GET` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:15:57.208660 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fcd980b-3136-4510-967f-fd1b2174b079 admin admin] SCRIPT_NAME: `/identity` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:15:57.208833 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06a676d6-c9a7-4f25-b854-16fd189bb61d admin admin] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:15:57.208955 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fcd980b-3136-4510-967f-fd1b2174b079 admin admin] PATH_INFO: `/v3/projects/088b96c6fcf64d899fbbe739c682de7d/users/f690a9da98b548e9aee69426281ae428/roles/e1c04b19fc4543b8b54f58bbcffbc598` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:15:57.209106 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06a676d6-c9a7-4f25-b854-16fd189bb61d admin admin] PATH_INFO: `/v3/domains` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:15:57.209811 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06a676d6-c9a7-4f25-b854-16fd189bb61d admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80378) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 02 10:15:57.210129 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06a676d6-c9a7-4f25-b854-16fd189bb61d admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:15:57.212402 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06a676d6-c9a7-4f25-b854-16fd189bb61d admin admin] RBAC: Authorization granted {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:15:57.224270 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2fcd980b-3136-4510-967f-fd1b2174b079 admin admin] RBAC: Authorizing `identity:create_grant(project_id=088b96c6fcf64d899fbbe739c682de7d, user_id=f690a9da98b548e9aee69426281ae428, role_id=e1c04b19fc4543b8b54f58bbcffbc598)` {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:15:57.225033 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 771/1561] 10.222.0.82 () {58 vars in 1153 bytes} [Tue Aug 2 10:15:57 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 02 10:15:57.225322 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: WARNING py.warnings [None req-2fcd980b-3136-4510-967f-fd1b2174b079 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:create_grant": "(role:admin and system_scope:all) or ((role:admin and domain_id:%(target.user.domain_id)s and domain_id:%(target.project.domain_id)s) or (role:admin and domain_id:%(target.user.domain_id)s and domain_id:%(target.domain.id)s) or (role:admin and domain_id:%(target.group.domain_id)s and domain_id:%(target.project.domain_id)s) or (role:admin and domain_id:%(target.group.domain_id)s and domain_id:%(target.domain.id)s)) and (domain_id:%(target.role.domain_id)s or None:%(target.role.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Aug 02 10:15:57.225322 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: warnings.warn(msg) Aug 02 10:15:57.225322 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: Aug 02 10:15:57.226503 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b84b97e4-7bdd-44ab-8b4b-aed65483c037 None None] REQUEST_METHOD: `POST` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:15:57.226789 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b84b97e4-7bdd-44ab-8b4b-aed65483c037 None None] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:15:57.227062 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b84b97e4-7bdd-44ab-8b4b-aed65483c037 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:15:57.227302 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2fcd980b-3136-4510-967f-fd1b2174b079 admin admin] RBAC: Authorization granted {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:15:57.242050 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: [pid: 80377|app: 0|req: 791/1562] 10.222.0.82 () {60 vars in 1483 bytes} [Tue Aug 2 10:15:57 2022] PUT /identity/v3/projects/088b96c6fcf64d899fbbe739c682de7d/users/f690a9da98b548e9aee69426281ae428/roles/e1c04b19fc4543b8b54f58bbcffbc598 => generated 0 bytes in 52 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 02 10:15:57.242890 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fcd980b-3136-4510-967f-fd1b2174b079 admin admin] Authenticating user token {{(pid=80377) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:15:57.251129 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.auth.core [None req-b84b97e4-7bdd-44ab-8b4b-aed65483c037 None None] MFA Rules not processed for user `891005fa5ff74041b75747d16cc66798`. Rule list: `[]` (Enabled: `True`). {{(pid=80378) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 02 10:15:57.252834 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.fernet_utils [None req-b84b97e4-7bdd-44ab-8b4b-aed65483c037 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80378) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 10:15:57.288130 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG dbcounter [-] [80377] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=80377) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 10:15:57.311162 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fcd980b-3136-4510-967f-fd1b2174b079 admin admin] Validating token access rules against request {{(pid=80377) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:15:57.312623 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG dbcounter [-] [80378] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80378) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 10:15:57.316724 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ee8d16d-b65c-4d85-b4e6-a17d53762dcf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '891005fa5ff74041b75747d16cc66798', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7047912c5acd4adeba5d87c3bf895e39', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80377) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:15:57.317320 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ee8d16d-b65c-4d85-b4e6-a17d53762dcf admin admin] REQUEST_METHOD: `POST` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:15:57.318024 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ee8d16d-b65c-4d85-b4e6-a17d53762dcf admin admin] SCRIPT_NAME: `/identity` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:15:57.318304 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ee8d16d-b65c-4d85-b4e6-a17d53762dcf admin admin] PATH_INFO: `/v3/projects` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:15:57.319156 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ee8d16d-b65c-4d85-b4e6-a17d53762dcf admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:15:57.320181 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: WARNING py.warnings [None req-4ee8d16d-b65c-4d85-b4e6-a17d53762dcf admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:create_project": "(role:admin and system_scope:all) or (role:admin and domain_id:%(target.project.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Aug 02 10:15:57.320181 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: warnings.warn(msg) Aug 02 10:15:57.320181 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: Aug 02 10:15:57.321446 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ee8d16d-b65c-4d85-b4e6-a17d53762dcf admin admin] RBAC: Authorization granted {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:15:57.323976 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 772/1563] 10.222.0.82 () {58 vars in 958 bytes} [Tue Aug 2 10:15:57 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 02 10:15:57.325500 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b84b97e4-7bdd-44ab-8b4b-aed65483c037 None None] Authenticating user token {{(pid=80378) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:15:57.334879 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b84b97e4-7bdd-44ab-8b4b-aed65483c037 None None] Validating token access rules against request {{(pid=80378) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:15:57.340133 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04ea10d3-b2bd-4f40-a944-2303be89b6ab admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '891005fa5ff74041b75747d16cc66798', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7047912c5acd4adeba5d87c3bf895e39', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80378) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:15:57.341322 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04ea10d3-b2bd-4f40-a944-2303be89b6ab admin admin] REQUEST_METHOD: `GET` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:15:57.341601 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04ea10d3-b2bd-4f40-a944-2303be89b6ab admin admin] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:15:57.341896 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04ea10d3-b2bd-4f40-a944-2303be89b6ab admin admin] PATH_INFO: `/v3/domains` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:15:57.342642 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04ea10d3-b2bd-4f40-a944-2303be89b6ab admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80378) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 02 10:15:57.342928 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04ea10d3-b2bd-4f40-a944-2303be89b6ab admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:15:57.344547 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04ea10d3-b2bd-4f40-a944-2303be89b6ab admin admin] RBAC: Authorization granted {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:15:57.355000 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: [pid: 80377|app: 0|req: 792/1564] 10.222.0.82 () {60 vars in 1153 bytes} [Tue Aug 2 10:15:57 2022] POST /identity/v3/projects => generated 373 bytes in 112 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 02 10:15:57.356484 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9091ef89-ebb3-40ab-bd01-7fe513f5eabb None None] REQUEST_METHOD: `POST` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:15:57.357076 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9091ef89-ebb3-40ab-bd01-7fe513f5eabb None None] SCRIPT_NAME: `/identity` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:15:57.357638 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 773/1565] 10.222.0.82 () {58 vars in 1153 bytes} [Tue Aug 2 10:15:57 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 02 10:15:57.357776 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9091ef89-ebb3-40ab-bd01-7fe513f5eabb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:15:57.358362 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04ea10d3-b2bd-4f40-a944-2303be89b6ab admin admin] Authenticating user token {{(pid=80378) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:15:57.376586 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04ea10d3-b2bd-4f40-a944-2303be89b6ab admin admin] Validating token access rules against request {{(pid=80378) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:15:57.379952 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.auth.core [None req-9091ef89-ebb3-40ab-bd01-7fe513f5eabb None None] MFA Rules not processed for user `891005fa5ff74041b75747d16cc66798`. Rule list: `[]` (Enabled: `True`). {{(pid=80377) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 02 10:15:57.381173 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.fernet_utils [None req-9091ef89-ebb3-40ab-bd01-7fe513f5eabb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80377) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 10:15:57.438953 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32e20e99-f2ef-4b85-a804-822c35536590 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '891005fa5ff74041b75747d16cc66798', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7047912c5acd4adeba5d87c3bf895e39', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80378) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:15:57.439476 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32e20e99-f2ef-4b85-a804-822c35536590 admin admin] REQUEST_METHOD: `GET` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:15:57.439755 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32e20e99-f2ef-4b85-a804-822c35536590 admin admin] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:15:57.440033 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32e20e99-f2ef-4b85-a804-822c35536590 admin admin] PATH_INFO: `/v3/domains` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:15:57.440727 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32e20e99-f2ef-4b85-a804-822c35536590 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80378) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 02 10:15:57.441031 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32e20e99-f2ef-4b85-a804-822c35536590 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:15:57.442658 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32e20e99-f2ef-4b85-a804-822c35536590 admin admin] RBAC: Authorization granted {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:15:57.449132 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: [pid: 80377|app: 0|req: 793/1566] 10.222.0.82 () {58 vars in 958 bytes} [Tue Aug 2 10:15:57 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 02 10:15:57.449834 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9091ef89-ebb3-40ab-bd01-7fe513f5eabb None None] Authenticating user token {{(pid=80377) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:15:57.456343 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 774/1567] 10.222.0.82 () {58 vars in 1153 bytes} [Tue Aug 2 10:15:57 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 98 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 02 10:15:57.457159 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32e20e99-f2ef-4b85-a804-822c35536590 admin admin] Authenticating user token {{(pid=80378) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:15:57.460809 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9091ef89-ebb3-40ab-bd01-7fe513f5eabb None None] Validating token access rules against request {{(pid=80377) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:15:57.467182 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0be5d1b-5bb8-40eb-9644-dc08d5862f62 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '891005fa5ff74041b75747d16cc66798', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7047912c5acd4adeba5d87c3bf895e39', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80377) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:15:57.468097 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0be5d1b-5bb8-40eb-9644-dc08d5862f62 admin admin] REQUEST_METHOD: `POST` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:15:57.468381 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0be5d1b-5bb8-40eb-9644-dc08d5862f62 admin admin] SCRIPT_NAME: `/identity` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:15:57.468655 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0be5d1b-5bb8-40eb-9644-dc08d5862f62 admin admin] PATH_INFO: `/v3/projects` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:15:57.469336 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0be5d1b-5bb8-40eb-9644-dc08d5862f62 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:15:57.471329 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0be5d1b-5bb8-40eb-9644-dc08d5862f62 admin admin] RBAC: Authorization granted {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:15:57.473057 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32e20e99-f2ef-4b85-a804-822c35536590 admin admin] Validating token access rules against request {{(pid=80378) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:15:57.477754 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0bfca861-3765-44cb-b773-f8053bc854c1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '891005fa5ff74041b75747d16cc66798', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7047912c5acd4adeba5d87c3bf895e39', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80378) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:15:57.478523 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bfca861-3765-44cb-b773-f8053bc854c1 admin admin] REQUEST_METHOD: `POST` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:15:57.478808 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bfca861-3765-44cb-b773-f8053bc854c1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:15:57.479083 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bfca861-3765-44cb-b773-f8053bc854c1 admin admin] PATH_INFO: `/v3/projects` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:15:57.479752 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0bfca861-3765-44cb-b773-f8053bc854c1 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:15:57.481719 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0bfca861-3765-44cb-b773-f8053bc854c1 admin admin] RBAC: Authorization granted {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:15:57.503842 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: [pid: 80377|app: 0|req: 794/1568] 10.222.0.82 () {60 vars in 1153 bytes} [Tue Aug 2 10:15:57 2022] POST /identity/v3/projects => generated 375 bytes in 51 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 02 10:15:57.503842 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdd55aee-aa11-4649-bfe1-62823dbc6314 None None] REQUEST_METHOD: `POST` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:15:57.503842 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdd55aee-aa11-4649-bfe1-62823dbc6314 None None] SCRIPT_NAME: `/identity` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:15:57.503842 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdd55aee-aa11-4649-bfe1-62823dbc6314 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:15:57.515604 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 775/1569] 10.222.0.82 () {60 vars in 1153 bytes} [Tue Aug 2 10:15:57 2022] POST /identity/v3/projects => generated 403 bytes in 55 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 02 10:15:57.515604 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0bfca861-3765-44cb-b773-f8053bc854c1 admin admin] Authenticating user token {{(pid=80378) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:15:57.522311 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.auth.core [None req-cdd55aee-aa11-4649-bfe1-62823dbc6314 None None] MFA Rules not processed for user `891005fa5ff74041b75747d16cc66798`. Rule list: `[]` (Enabled: `True`). {{(pid=80377) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 02 10:15:57.523524 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.fernet_utils [None req-cdd55aee-aa11-4649-bfe1-62823dbc6314 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80377) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 10:15:57.582612 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0bfca861-3765-44cb-b773-f8053bc854c1 admin admin] Validating token access rules against request {{(pid=80378) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:15:57.587243 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03ebaba4-91b2-4046-8b15-dc1078cb8f0f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '891005fa5ff74041b75747d16cc66798', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7047912c5acd4adeba5d87c3bf895e39', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80378) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:15:57.587855 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03ebaba4-91b2-4046-8b15-dc1078cb8f0f admin admin] REQUEST_METHOD: `GET` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:15:57.588138 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03ebaba4-91b2-4046-8b15-dc1078cb8f0f admin admin] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:15:57.588537 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03ebaba4-91b2-4046-8b15-dc1078cb8f0f admin admin] PATH_INFO: `/v3/domains` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:15:57.589572 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03ebaba4-91b2-4046-8b15-dc1078cb8f0f admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80378) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 02 10:15:57.589981 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03ebaba4-91b2-4046-8b15-dc1078cb8f0f admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:15:57.590572 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: [pid: 80377|app: 0|req: 795/1570] 10.222.0.82 () {58 vars in 958 bytes} [Tue Aug 2 10:15:57 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 90 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 02 10:15:57.591474 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdd55aee-aa11-4649-bfe1-62823dbc6314 None None] Authenticating user token {{(pid=80377) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:15:57.591581 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03ebaba4-91b2-4046-8b15-dc1078cb8f0f admin admin] RBAC: Authorization granted {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:15:57.603187 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdd55aee-aa11-4649-bfe1-62823dbc6314 None None] Validating token access rules against request {{(pid=80377) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:15:57.612061 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 776/1571] 10.222.0.82 () {58 vars in 1153 bytes} [Tue Aug 2 10:15:57 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 93 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 02 10:15:57.612061 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03ebaba4-91b2-4046-8b15-dc1078cb8f0f admin admin] Authenticating user token {{(pid=80378) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:15:57.612315 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb264391-3b5d-4423-a1b7-10e6200f645f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '891005fa5ff74041b75747d16cc66798', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7047912c5acd4adeba5d87c3bf895e39', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80377) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:15:57.612315 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb264391-3b5d-4423-a1b7-10e6200f645f admin admin] REQUEST_METHOD: `POST` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:15:57.612315 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb264391-3b5d-4423-a1b7-10e6200f645f admin admin] SCRIPT_NAME: `/identity` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:15:57.612315 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb264391-3b5d-4423-a1b7-10e6200f645f admin admin] PATH_INFO: `/v3/users` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:15:57.612746 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb264391-3b5d-4423-a1b7-10e6200f645f admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:15:57.612746 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb264391-3b5d-4423-a1b7-10e6200f645f admin admin] RBAC: Authorization granted {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:15:57.623204 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03ebaba4-91b2-4046-8b15-dc1078cb8f0f admin admin] Validating token access rules against request {{(pid=80378) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:15:57.628305 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f2942cb-83b7-40d9-bbe2-401e3c1b4278 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '891005fa5ff74041b75747d16cc66798', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7047912c5acd4adeba5d87c3bf895e39', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80378) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:15:57.629103 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f2942cb-83b7-40d9-bbe2-401e3c1b4278 admin admin] REQUEST_METHOD: `POST` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:15:57.629467 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f2942cb-83b7-40d9-bbe2-401e3c1b4278 admin admin] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:15:57.629752 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f2942cb-83b7-40d9-bbe2-401e3c1b4278 admin admin] PATH_INFO: `/v3/projects` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:15:57.630447 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f2942cb-83b7-40d9-bbe2-401e3c1b4278 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:15:57.632361 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f2942cb-83b7-40d9-bbe2-401e3c1b4278 admin admin] RBAC: Authorization granted {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:15:57.637836 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: [pid: 80377|app: 0|req: 796/1572] 10.222.0.82 () {60 vars in 1144 bytes} [Tue Aug 2 10:15:57 2022] POST /identity/v3/users => generated 287 bytes in 44 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 02 10:15:57.637836 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb264391-3b5d-4423-a1b7-10e6200f645f admin admin] Authenticating user token {{(pid=80377) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:15:57.648038 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb264391-3b5d-4423-a1b7-10e6200f645f admin admin] Validating token access rules against request {{(pid=80377) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:15:57.653024 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72e1427d-4dfd-4c43-9a77-048dc768829c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '891005fa5ff74041b75747d16cc66798', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7047912c5acd4adeba5d87c3bf895e39', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80377) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:15:57.653611 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72e1427d-4dfd-4c43-9a77-048dc768829c admin admin] REQUEST_METHOD: `GET` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:15:57.654028 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72e1427d-4dfd-4c43-9a77-048dc768829c admin admin] SCRIPT_NAME: `/identity` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:15:57.654399 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72e1427d-4dfd-4c43-9a77-048dc768829c admin admin] PATH_INFO: `/v3/domains` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:15:57.655178 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72e1427d-4dfd-4c43-9a77-048dc768829c admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80377) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 02 10:15:57.655488 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72e1427d-4dfd-4c43-9a77-048dc768829c admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:15:57.657259 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72e1427d-4dfd-4c43-9a77-048dc768829c admin admin] RBAC: Authorization granted {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:15:57.664229 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 777/1573] 10.222.0.82 () {60 vars in 1153 bytes} [Tue Aug 2 10:15:57 2022] POST /identity/v3/projects => generated 393 bytes in 58 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 02 10:15:57.665445 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f2942cb-83b7-40d9-bbe2-401e3c1b4278 admin admin] Authenticating user token {{(pid=80378) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:15:57.671663 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: [pid: 80377|app: 0|req: 797/1574] 10.222.0.82 () {58 vars in 1153 bytes} [Tue Aug 2 10:15:57 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 02 10:15:57.672737 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72e1427d-4dfd-4c43-9a77-048dc768829c admin admin] Authenticating user token {{(pid=80377) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:15:57.683798 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72e1427d-4dfd-4c43-9a77-048dc768829c admin admin] Validating token access rules against request {{(pid=80377) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:15:57.692462 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG dbcounter [-] [80377] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80377) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 10:15:57.698368 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG dbcounter [-] [80378] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80378) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 10:15:57.731610 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f2942cb-83b7-40d9-bbe2-401e3c1b4278 admin admin] Validating token access rules against request {{(pid=80378) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:15:57.736349 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d932ce56-41ba-4641-9f74-67e627a9dfcb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '891005fa5ff74041b75747d16cc66798', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7047912c5acd4adeba5d87c3bf895e39', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80378) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:15:57.736964 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d932ce56-41ba-4641-9f74-67e627a9dfcb admin admin] REQUEST_METHOD: `POST` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:15:57.737249 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d932ce56-41ba-4641-9f74-67e627a9dfcb admin admin] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:15:57.737524 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d932ce56-41ba-4641-9f74-67e627a9dfcb admin admin] PATH_INFO: `/v3/projects` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:15:57.738246 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d932ce56-41ba-4641-9f74-67e627a9dfcb admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:15:57.740126 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d932ce56-41ba-4641-9f74-67e627a9dfcb admin admin] RBAC: Authorization granted {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:15:57.741322 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff7b187e-357e-4f25-b4b2-5e3ed4f45950 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '891005fa5ff74041b75747d16cc66798', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7047912c5acd4adeba5d87c3bf895e39', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80377) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:15:57.741989 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff7b187e-357e-4f25-b4b2-5e3ed4f45950 admin admin] REQUEST_METHOD: `POST` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:15:57.742277 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff7b187e-357e-4f25-b4b2-5e3ed4f45950 admin admin] SCRIPT_NAME: `/identity` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:15:57.742553 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff7b187e-357e-4f25-b4b2-5e3ed4f45950 admin admin] PATH_INFO: `/v3/users` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:15:57.743224 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff7b187e-357e-4f25-b4b2-5e3ed4f45950 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:15:57.745171 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff7b187e-357e-4f25-b4b2-5e3ed4f45950 admin admin] RBAC: Authorization granted {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:15:57.772084 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 778/1575] 10.222.0.82 () {60 vars in 1153 bytes} [Tue Aug 2 10:15:57 2022] POST /identity/v3/projects => generated 389 bytes in 104 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 02 10:15:57.772313 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: [pid: 80377|app: 0|req: 798/1576] 10.222.0.82 () {60 vars in 1144 bytes} [Tue Aug 2 10:15:57 2022] POST /identity/v3/users => generated 288 bytes in 97 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 02 10:15:57.772432 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d932ce56-41ba-4641-9f74-67e627a9dfcb admin admin] Authenticating user token {{(pid=80378) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:15:57.772542 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff7b187e-357e-4f25-b4b2-5e3ed4f45950 admin admin] Authenticating user token {{(pid=80377) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:15:57.826629 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff7b187e-357e-4f25-b4b2-5e3ed4f45950 admin admin] Validating token access rules against request {{(pid=80377) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:15:57.835418 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d932ce56-41ba-4641-9f74-67e627a9dfcb admin admin] Validating token access rules against request {{(pid=80378) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:15:57.839982 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49f1d682-ba2d-4d61-9287-931038ed2889 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '891005fa5ff74041b75747d16cc66798', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7047912c5acd4adeba5d87c3bf895e39', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80378) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:15:57.840655 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49f1d682-ba2d-4d61-9287-931038ed2889 admin admin] REQUEST_METHOD: `POST` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:15:57.840938 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49f1d682-ba2d-4d61-9287-931038ed2889 admin admin] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:15:57.841213 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49f1d682-ba2d-4d61-9287-931038ed2889 admin admin] PATH_INFO: `/v3/users` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:15:57.841905 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49f1d682-ba2d-4d61-9287-931038ed2889 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:15:57.842926 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: WARNING py.warnings [None req-49f1d682-ba2d-4d61-9287-931038ed2889 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:create_user": "(role:admin and system_scope:all) or (role:admin and token.domain.id:%(target.user.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Aug 02 10:15:57.842926 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: warnings.warn(msg) Aug 02 10:15:57.842926 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: Aug 02 10:15:57.844153 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49f1d682-ba2d-4d61-9287-931038ed2889 admin admin] RBAC: Authorization granted {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:15:57.868426 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 779/1577] 10.222.0.82 () {60 vars in 1144 bytes} [Tue Aug 2 10:15:57 2022] POST /identity/v3/users => generated 302 bytes in 99 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 02 10:15:57.869036 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49f1d682-ba2d-4d61-9287-931038ed2889 admin admin] Authenticating user token {{(pid=80378) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:15:57.878713 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49f1d682-ba2d-4d61-9287-931038ed2889 admin admin] Validating token access rules against request {{(pid=80378) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:15:57.880133 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4cb6953a-da69-4c94-8407-2fb3a88b40ae service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '33b87170f0634b67bce60750847b5cbe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f1b38ebb804c41c69772c998aae72c41', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80377) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:15:57.880708 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cb6953a-da69-4c94-8407-2fb3a88b40ae service neutron] REQUEST_METHOD: `GET` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:15:57.880990 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cb6953a-da69-4c94-8407-2fb3a88b40ae service neutron] SCRIPT_NAME: `/identity` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:15:57.881264 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cb6953a-da69-4c94-8407-2fb3a88b40ae service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:15:57.883342 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93ebaf82-6af6-4db8-89e1-0590fa18c2db admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '891005fa5ff74041b75747d16cc66798', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7047912c5acd4adeba5d87c3bf895e39', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80378) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:15:57.884305 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93ebaf82-6af6-4db8-89e1-0590fa18c2db admin admin] REQUEST_METHOD: `POST` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:15:57.884305 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93ebaf82-6af6-4db8-89e1-0590fa18c2db admin admin] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:15:57.884475 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93ebaf82-6af6-4db8-89e1-0590fa18c2db admin admin] PATH_INFO: `/v3/projects` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:15:57.885377 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93ebaf82-6af6-4db8-89e1-0590fa18c2db admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:15:57.888139 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93ebaf82-6af6-4db8-89e1-0590fa18c2db admin admin] RBAC: Authorization granted {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:15:57.892499 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4cb6953a-da69-4c94-8407-2fb3a88b40ae service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:15:57.894131 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4cb6953a-da69-4c94-8407-2fb3a88b40ae service neutron] RBAC: Authorization granted {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:15:57.904897 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: [pid: 80377|app: 0|req: 799/1578] 10.222.0.82 () {60 vars in 1365 bytes} [Tue Aug 2 10:15:57 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 135 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 02 10:15:57.906020 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4cb6953a-da69-4c94-8407-2fb3a88b40ae service neutron] Authenticating user token {{(pid=80377) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:15:57.916272 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4cb6953a-da69-4c94-8407-2fb3a88b40ae service neutron] Validating token access rules against request {{(pid=80377) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:15:57.921669 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 780/1579] 10.222.0.82 () {60 vars in 1153 bytes} [Tue Aug 2 10:15:57 2022] POST /identity/v3/projects => generated 367 bytes in 50 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 02 10:15:57.921669 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93ebaf82-6af6-4db8-89e1-0590fa18c2db admin admin] Authenticating user token {{(pid=80378) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:15:57.994962 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93ebaf82-6af6-4db8-89e1-0590fa18c2db admin admin] Validating token access rules against request {{(pid=80378) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:15:57.996022 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2168c8ec-ad9b-4b2e-a0be-7b35273f80a5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '891005fa5ff74041b75747d16cc66798', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7047912c5acd4adeba5d87c3bf895e39', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80377) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:15:57.996920 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2168c8ec-ad9b-4b2e-a0be-7b35273f80a5 admin admin] REQUEST_METHOD: `GET` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:15:57.997166 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2168c8ec-ad9b-4b2e-a0be-7b35273f80a5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:15:57.997523 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2168c8ec-ad9b-4b2e-a0be-7b35273f80a5 admin admin] PATH_INFO: `/v3/roles` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:15:57.998278 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2168c8ec-ad9b-4b2e-a0be-7b35273f80a5 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:15:58.000031 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2168c8ec-ad9b-4b2e-a0be-7b35273f80a5 admin admin] RBAC: Authorization granted {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:15:58.001592 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG dbcounter [-] [80377] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80377) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 10:15:58.001962 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-438b98c7-597e-4f6c-8fa2-433f2954c0d0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '891005fa5ff74041b75747d16cc66798', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7047912c5acd4adeba5d87c3bf895e39', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80378) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:15:58.002699 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-438b98c7-597e-4f6c-8fa2-433f2954c0d0 admin admin] REQUEST_METHOD: `POST` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:15:58.003051 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-438b98c7-597e-4f6c-8fa2-433f2954c0d0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:15:58.003401 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-438b98c7-597e-4f6c-8fa2-433f2954c0d0 admin admin] PATH_INFO: `/v3/users` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:15:58.004140 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-438b98c7-597e-4f6c-8fa2-433f2954c0d0 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:15:58.006162 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-438b98c7-597e-4f6c-8fa2-433f2954c0d0 admin admin] RBAC: Authorization granted {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:15:58.018377 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: [pid: 80377|app: 0|req: 800/1580] 10.222.0.82 () {58 vars in 1122 bytes} [Tue Aug 2 10:15:57 2022] GET /identity/v3/roles => generated 1420 bytes in 110 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 02 10:15:58.018377 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2168c8ec-ad9b-4b2e-a0be-7b35273f80a5 admin admin] Authenticating user token {{(pid=80377) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:15:58.027152 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2168c8ec-ad9b-4b2e-a0be-7b35273f80a5 admin admin] Validating token access rules against request {{(pid=80377) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:15:58.033859 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2bf3870-b282-41d4-b26e-761466aea383 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '891005fa5ff74041b75747d16cc66798', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7047912c5acd4adeba5d87c3bf895e39', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80377) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:15:58.034592 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2bf3870-b282-41d4-b26e-761466aea383 admin admin] REQUEST_METHOD: `POST` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:15:58.034928 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2bf3870-b282-41d4-b26e-761466aea383 admin admin] SCRIPT_NAME: `/identity` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:15:58.038152 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2bf3870-b282-41d4-b26e-761466aea383 admin admin] PATH_INFO: `/v3/projects` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:15:58.038336 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 781/1581] 10.222.0.82 () {60 vars in 1144 bytes} [Tue Aug 2 10:15:57 2022] POST /identity/v3/users => generated 297 bytes in 116 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 02 10:15:58.038527 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2bf3870-b282-41d4-b26e-761466aea383 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:15:58.038637 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-438b98c7-597e-4f6c-8fa2-433f2954c0d0 admin admin] Authenticating user token {{(pid=80378) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:15:58.038736 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2bf3870-b282-41d4-b26e-761466aea383 admin admin] RBAC: Authorization granted {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:15:58.049150 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-438b98c7-597e-4f6c-8fa2-433f2954c0d0 admin admin] Validating token access rules against request {{(pid=80378) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:15:58.054993 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8be8f3c1-1ecc-4edd-9f87-fcf33fafdb8e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '891005fa5ff74041b75747d16cc66798', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7047912c5acd4adeba5d87c3bf895e39', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80378) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:15:58.055960 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8be8f3c1-1ecc-4edd-9f87-fcf33fafdb8e admin admin] REQUEST_METHOD: `GET` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:15:58.056501 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8be8f3c1-1ecc-4edd-9f87-fcf33fafdb8e admin admin] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:15:58.057057 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8be8f3c1-1ecc-4edd-9f87-fcf33fafdb8e admin admin] PATH_INFO: `/v3/roles` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:15:58.057717 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8be8f3c1-1ecc-4edd-9f87-fcf33fafdb8e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:15:58.059862 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8be8f3c1-1ecc-4edd-9f87-fcf33fafdb8e admin admin] RBAC: Authorization granted {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:15:58.065968 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG dbcounter [-] [80378] Writing DB stats keystone:SELECT=92,keystone:INSERT=8 {{(pid=80378) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 10:15:58.076498 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 782/1582] 10.222.0.82 () {58 vars in 1122 bytes} [Tue Aug 2 10:15:58 2022] GET /identity/v3/roles => generated 1420 bytes in 40 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 02 10:15:58.077471 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8be8f3c1-1ecc-4edd-9f87-fcf33fafdb8e admin admin] Authenticating user token {{(pid=80378) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:15:58.078720 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: [pid: 80377|app: 0|req: 801/1583] 10.222.0.82 () {60 vars in 1153 bytes} [Tue Aug 2 10:15:58 2022] POST /identity/v3/projects => generated 361 bytes in 62 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 02 10:15:58.080204 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2bf3870-b282-41d4-b26e-761466aea383 admin admin] Authenticating user token {{(pid=80377) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:15:58.090232 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8be8f3c1-1ecc-4edd-9f87-fcf33fafdb8e admin admin] Validating token access rules against request {{(pid=80378) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:15:58.156863 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4eca316-fbe3-47bc-a1bf-a676edddf7bc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '891005fa5ff74041b75747d16cc66798', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7047912c5acd4adeba5d87c3bf895e39', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80378) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:15:58.157953 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4eca316-fbe3-47bc-a1bf-a676edddf7bc admin admin] REQUEST_METHOD: `POST` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:15:58.158303 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4eca316-fbe3-47bc-a1bf-a676edddf7bc admin admin] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:15:58.158586 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4eca316-fbe3-47bc-a1bf-a676edddf7bc admin admin] PATH_INFO: `/v3/users` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:15:58.159289 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4eca316-fbe3-47bc-a1bf-a676edddf7bc admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:15:58.161405 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4eca316-fbe3-47bc-a1bf-a676edddf7bc admin admin] RBAC: Authorization granted {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:15:58.169722 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2bf3870-b282-41d4-b26e-761466aea383 admin admin] Validating token access rules against request {{(pid=80377) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:15:58.176443 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2508918-21cb-46e9-a3f1-acf788a1b74d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '891005fa5ff74041b75747d16cc66798', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7047912c5acd4adeba5d87c3bf895e39', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80377) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:15:58.177234 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2508918-21cb-46e9-a3f1-acf788a1b74d admin admin] REQUEST_METHOD: `GET` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:15:58.177614 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2508918-21cb-46e9-a3f1-acf788a1b74d admin admin] SCRIPT_NAME: `/identity` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:15:58.177898 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2508918-21cb-46e9-a3f1-acf788a1b74d admin admin] PATH_INFO: `/v3/roles` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:15:58.178771 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2508918-21cb-46e9-a3f1-acf788a1b74d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:15:58.180489 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2508918-21cb-46e9-a3f1-acf788a1b74d admin admin] RBAC: Authorization granted {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:15:58.187630 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 783/1584] 10.222.0.82 () {60 vars in 1144 bytes} [Tue Aug 2 10:15:58 2022] POST /identity/v3/users => generated 295 bytes in 110 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 02 10:15:58.191805 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4eca316-fbe3-47bc-a1bf-a676edddf7bc admin admin] Authenticating user token {{(pid=80378) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:15:58.195126 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: [pid: 80377|app: 0|req: 802/1585] 10.222.0.82 () {58 vars in 1122 bytes} [Tue Aug 2 10:15:58 2022] GET /identity/v3/roles => generated 1420 bytes in 115 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 02 10:15:58.195954 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2508918-21cb-46e9-a3f1-acf788a1b74d admin admin] Authenticating user token {{(pid=80377) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:15:58.204384 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4eca316-fbe3-47bc-a1bf-a676edddf7bc admin admin] Validating token access rules against request {{(pid=80378) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:15:58.205724 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2508918-21cb-46e9-a3f1-acf788a1b74d admin admin] Validating token access rules against request {{(pid=80377) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:15:58.209817 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f358f95-a3a4-4da2-aa94-3f79aa04e00e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '891005fa5ff74041b75747d16cc66798', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7047912c5acd4adeba5d87c3bf895e39', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80378) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:15:58.210555 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f358f95-a3a4-4da2-aa94-3f79aa04e00e admin admin] REQUEST_METHOD: `POST` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:15:58.210747 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ff65593-6823-45f6-8cb9-2db86a7fdd84 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '891005fa5ff74041b75747d16cc66798', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7047912c5acd4adeba5d87c3bf895e39', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80377) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:15:58.211024 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f358f95-a3a4-4da2-aa94-3f79aa04e00e admin admin] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:15:58.211366 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ff65593-6823-45f6-8cb9-2db86a7fdd84 admin admin] REQUEST_METHOD: `GET` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:15:58.211490 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f358f95-a3a4-4da2-aa94-3f79aa04e00e admin admin] PATH_INFO: `/v3/users` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:15:58.211653 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ff65593-6823-45f6-8cb9-2db86a7fdd84 admin admin] SCRIPT_NAME: `/identity` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:15:58.211934 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ff65593-6823-45f6-8cb9-2db86a7fdd84 admin admin] PATH_INFO: `/v3/roles` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:15:58.212474 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f358f95-a3a4-4da2-aa94-3f79aa04e00e admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:15:58.212632 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ff65593-6823-45f6-8cb9-2db86a7fdd84 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:15:58.214830 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ff65593-6823-45f6-8cb9-2db86a7fdd84 admin admin] RBAC: Authorization granted {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:15:58.214990 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f358f95-a3a4-4da2-aa94-3f79aa04e00e admin admin] RBAC: Authorization granted {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:15:58.228773 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: [pid: 80377|app: 0|req: 803/1586] 10.222.0.82 () {58 vars in 1122 bytes} [Tue Aug 2 10:15:58 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 02 10:15:58.229949 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ff65593-6823-45f6-8cb9-2db86a7fdd84 admin admin] Authenticating user token {{(pid=80377) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:15:58.242183 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ff65593-6823-45f6-8cb9-2db86a7fdd84 admin admin] Validating token access rules against request {{(pid=80377) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:15:58.246367 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 784/1587] 10.222.0.82 () {60 vars in 1144 bytes} [Tue Aug 2 10:15:58 2022] POST /identity/v3/users => generated 284 bytes in 58 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 02 10:15:58.248187 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-404cf9be-57b4-4abe-bc91-6db27328cd6c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '891005fa5ff74041b75747d16cc66798', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7047912c5acd4adeba5d87c3bf895e39', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80377) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:15:58.248304 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f358f95-a3a4-4da2-aa94-3f79aa04e00e admin admin] Authenticating user token {{(pid=80378) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:15:58.249823 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-404cf9be-57b4-4abe-bc91-6db27328cd6c admin admin] REQUEST_METHOD: `GET` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:15:58.250200 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-404cf9be-57b4-4abe-bc91-6db27328cd6c admin admin] SCRIPT_NAME: `/identity` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:15:58.250605 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-404cf9be-57b4-4abe-bc91-6db27328cd6c admin admin] PATH_INFO: `/v3/roles` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:15:58.252059 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-404cf9be-57b4-4abe-bc91-6db27328cd6c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:15:58.253136 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-404cf9be-57b4-4abe-bc91-6db27328cd6c admin admin] RBAC: Authorization granted {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:15:58.263696 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f358f95-a3a4-4da2-aa94-3f79aa04e00e admin admin] Validating token access rules against request {{(pid=80378) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:15:58.266987 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: [pid: 80377|app: 0|req: 804/1588] 10.222.0.82 () {58 vars in 1122 bytes} [Tue Aug 2 10:15:58 2022] GET /identity/v3/roles => generated 1420 bytes in 37 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 02 10:15:58.267925 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-404cf9be-57b4-4abe-bc91-6db27328cd6c admin admin] Authenticating user token {{(pid=80377) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:15:58.275126 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae16b638-9a7d-47b6-a9f9-55e7d181cba6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '891005fa5ff74041b75747d16cc66798', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7047912c5acd4adeba5d87c3bf895e39', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80378) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:15:58.278925 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae16b638-9a7d-47b6-a9f9-55e7d181cba6 admin admin] REQUEST_METHOD: `GET` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:15:58.279403 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae16b638-9a7d-47b6-a9f9-55e7d181cba6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:15:58.279743 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae16b638-9a7d-47b6-a9f9-55e7d181cba6 admin admin] PATH_INFO: `/v3/roles` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:15:58.280558 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae16b638-9a7d-47b6-a9f9-55e7d181cba6 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:15:58.283690 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae16b638-9a7d-47b6-a9f9-55e7d181cba6 admin admin] RBAC: Authorization granted {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:15:58.289327 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-404cf9be-57b4-4abe-bc91-6db27328cd6c admin admin] Validating token access rules against request {{(pid=80377) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:15:58.299255 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 785/1589] 10.222.0.82 () {58 vars in 1122 bytes} [Tue Aug 2 10:15:58 2022] GET /identity/v3/roles => generated 1420 bytes in 52 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 02 10:15:58.300784 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae16b638-9a7d-47b6-a9f9-55e7d181cba6 admin admin] Authenticating user token {{(pid=80378) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:15:58.303807 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e225e49-8e8f-4325-bc04-941b4370b133 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '891005fa5ff74041b75747d16cc66798', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7047912c5acd4adeba5d87c3bf895e39', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80377) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:15:58.304835 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e225e49-8e8f-4325-bc04-941b4370b133 admin admin] REQUEST_METHOD: `POST` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:15:58.305302 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e225e49-8e8f-4325-bc04-941b4370b133 admin admin] SCRIPT_NAME: `/identity` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:15:58.305571 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e225e49-8e8f-4325-bc04-941b4370b133 admin admin] PATH_INFO: `/v3/users` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:15:58.306745 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e225e49-8e8f-4325-bc04-941b4370b133 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:15:58.309851 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e225e49-8e8f-4325-bc04-941b4370b133 admin admin] RBAC: Authorization granted {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:15:58.315255 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae16b638-9a7d-47b6-a9f9-55e7d181cba6 admin admin] Validating token access rules against request {{(pid=80378) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:15:58.320794 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58af72ab-bb01-4414-853d-7f6c562b9679 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '891005fa5ff74041b75747d16cc66798', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7047912c5acd4adeba5d87c3bf895e39', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80378) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:15:58.322492 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58af72ab-bb01-4414-853d-7f6c562b9679 admin admin] REQUEST_METHOD: `GET` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:15:58.322882 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58af72ab-bb01-4414-853d-7f6c562b9679 admin admin] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:15:58.323349 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58af72ab-bb01-4414-853d-7f6c562b9679 admin admin] PATH_INFO: `/v3/roles` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:15:58.324192 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58af72ab-bb01-4414-853d-7f6c562b9679 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:15:58.326750 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58af72ab-bb01-4414-853d-7f6c562b9679 admin admin] RBAC: Authorization granted {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:15:58.343303 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: [pid: 80377|app: 0|req: 805/1590] 10.222.0.82 () {60 vars in 1144 bytes} [Tue Aug 2 10:15:58 2022] POST /identity/v3/users => generated 281 bytes in 73 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 02 10:15:58.343523 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 786/1591] 10.222.0.82 () {58 vars in 1122 bytes} [Tue Aug 2 10:15:58 2022] GET /identity/v3/roles => generated 1420 bytes in 41 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 02 10:15:58.343523 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58af72ab-bb01-4414-853d-7f6c562b9679 admin admin] Authenticating user token {{(pid=80378) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:15:58.345168 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e225e49-8e8f-4325-bc04-941b4370b133 admin admin] Authenticating user token {{(pid=80377) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:15:58.355747 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58af72ab-bb01-4414-853d-7f6c562b9679 admin admin] Validating token access rules against request {{(pid=80378) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:15:58.356175 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e225e49-8e8f-4325-bc04-941b4370b133 admin admin] Validating token access rules against request {{(pid=80377) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:15:58.361622 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c133124-d849-41fb-96bf-b29e4581935b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '891005fa5ff74041b75747d16cc66798', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7047912c5acd4adeba5d87c3bf895e39', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80377) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:15:58.362419 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c133124-d849-41fb-96bf-b29e4581935b admin admin] REQUEST_METHOD: `GET` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:15:58.362573 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c133124-d849-41fb-96bf-b29e4581935b admin admin] SCRIPT_NAME: `/identity` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:15:58.362855 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c133124-d849-41fb-96bf-b29e4581935b admin admin] PATH_INFO: `/v3/roles` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:15:58.363005 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3249a1aa-5a0b-4e2c-9387-49666a01225f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '891005fa5ff74041b75747d16cc66798', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7047912c5acd4adeba5d87c3bf895e39', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80378) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:15:58.363566 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c133124-d849-41fb-96bf-b29e4581935b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:15:58.364356 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3249a1aa-5a0b-4e2c-9387-49666a01225f admin admin] REQUEST_METHOD: `PUT` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:15:58.364723 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3249a1aa-5a0b-4e2c-9387-49666a01225f admin admin] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:15:58.365101 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3249a1aa-5a0b-4e2c-9387-49666a01225f admin admin] PATH_INFO: `/v3/projects/9bab6ba334404db0b6709c91b165e0bc/users/b4624993d8e94d3ba204d1f8b71f216e/roles/e1c04b19fc4543b8b54f58bbcffbc598` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:15:58.365234 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c133124-d849-41fb-96bf-b29e4581935b admin admin] RBAC: Authorization granted {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:15:58.381919 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: [pid: 80377|app: 0|req: 806/1592] 10.222.0.82 () {58 vars in 1122 bytes} [Tue Aug 2 10:15:58 2022] GET /identity/v3/roles => generated 1420 bytes in 37 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 02 10:15:58.381919 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c133124-d849-41fb-96bf-b29e4581935b admin admin] Authenticating user token {{(pid=80377) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:15:58.388720 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3249a1aa-5a0b-4e2c-9387-49666a01225f admin admin] RBAC: Authorizing `identity:create_grant(project_id=9bab6ba334404db0b6709c91b165e0bc, user_id=b4624993d8e94d3ba204d1f8b71f216e, role_id=e1c04b19fc4543b8b54f58bbcffbc598)` {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:15:58.391497 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3249a1aa-5a0b-4e2c-9387-49666a01225f admin admin] RBAC: Authorization granted {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:15:58.394223 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c133124-d849-41fb-96bf-b29e4581935b admin admin] Validating token access rules against request {{(pid=80377) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:15:58.399589 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4128302-f987-478e-99fc-48064dda3748 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '891005fa5ff74041b75747d16cc66798', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7047912c5acd4adeba5d87c3bf895e39', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80377) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:15:58.400409 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4128302-f987-478e-99fc-48064dda3748 admin admin] REQUEST_METHOD: `GET` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:15:58.400675 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4128302-f987-478e-99fc-48064dda3748 admin admin] SCRIPT_NAME: `/identity` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:15:58.400985 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4128302-f987-478e-99fc-48064dda3748 admin admin] PATH_INFO: `/v3/roles` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:15:58.401944 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4128302-f987-478e-99fc-48064dda3748 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:15:58.404119 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4128302-f987-478e-99fc-48064dda3748 admin admin] RBAC: Authorization granted {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:15:58.409334 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 787/1593] 10.222.0.82 () {60 vars in 1483 bytes} [Tue Aug 2 10:15:58 2022] PUT /identity/v3/projects/9bab6ba334404db0b6709c91b165e0bc/users/b4624993d8e94d3ba204d1f8b71f216e/roles/e1c04b19fc4543b8b54f58bbcffbc598 => generated 0 bytes in 68 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 02 10:15:58.410566 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3249a1aa-5a0b-4e2c-9387-49666a01225f admin admin] Authenticating user token {{(pid=80378) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:15:58.415598 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: [pid: 80377|app: 0|req: 807/1594] 10.222.0.82 () {58 vars in 1122 bytes} [Tue Aug 2 10:15:58 2022] GET /identity/v3/roles => generated 1420 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 02 10:15:58.416215 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4128302-f987-478e-99fc-48064dda3748 admin admin] Authenticating user token {{(pid=80377) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:15:58.434026 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4128302-f987-478e-99fc-48064dda3748 admin admin] Validating token access rules against request {{(pid=80377) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:15:58.495982 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3249a1aa-5a0b-4e2c-9387-49666a01225f admin admin] Validating token access rules against request {{(pid=80378) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:15:58.497357 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8192fb5-a6c0-4e30-828b-db4a37a3ed17 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '891005fa5ff74041b75747d16cc66798', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7047912c5acd4adeba5d87c3bf895e39', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80377) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:15:58.498316 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8192fb5-a6c0-4e30-828b-db4a37a3ed17 admin admin] REQUEST_METHOD: `PUT` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:15:58.498665 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8192fb5-a6c0-4e30-828b-db4a37a3ed17 admin admin] SCRIPT_NAME: `/identity` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:15:58.499056 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8192fb5-a6c0-4e30-828b-db4a37a3ed17 admin admin] PATH_INFO: `/v3/projects/4bc7ee5e1e7f4f5c8d9de8fa1e48991b/users/e881ae7c60254c12860fff0a40480641/roles/e1c04b19fc4543b8b54f58bbcffbc598` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:15:58.501349 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e8063d4-177a-46c8-9168-17abefbce429 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '891005fa5ff74041b75747d16cc66798', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7047912c5acd4adeba5d87c3bf895e39', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80378) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:15:58.502076 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e8063d4-177a-46c8-9168-17abefbce429 admin admin] REQUEST_METHOD: `GET` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:15:58.502437 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e8063d4-177a-46c8-9168-17abefbce429 admin admin] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:15:58.502819 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e8063d4-177a-46c8-9168-17abefbce429 admin admin] PATH_INFO: `/v3/roles` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:15:58.503639 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e8063d4-177a-46c8-9168-17abefbce429 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:15:58.505358 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e8063d4-177a-46c8-9168-17abefbce429 admin admin] RBAC: Authorization granted {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:15:58.513050 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG dbcounter [-] [80377] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=80377) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 10:15:58.517269 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8192fb5-a6c0-4e30-828b-db4a37a3ed17 admin admin] RBAC: Authorizing `identity:create_grant(project_id=4bc7ee5e1e7f4f5c8d9de8fa1e48991b, user_id=e881ae7c60254c12860fff0a40480641, role_id=e1c04b19fc4543b8b54f58bbcffbc598)` {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:15:58.517892 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 788/1595] 10.222.0.82 () {58 vars in 1122 bytes} [Tue Aug 2 10:15:58 2022] GET /identity/v3/roles => generated 1420 bytes in 108 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 02 10:15:58.519778 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8192fb5-a6c0-4e30-828b-db4a37a3ed17 admin admin] RBAC: Authorization granted {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:15:58.521921 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e8063d4-177a-46c8-9168-17abefbce429 admin admin] Authenticating user token {{(pid=80378) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:15:58.533523 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e8063d4-177a-46c8-9168-17abefbce429 admin admin] Validating token access rules against request {{(pid=80378) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:15:58.536251 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: [pid: 80377|app: 0|req: 808/1596] 10.222.0.82 () {60 vars in 1483 bytes} [Tue Aug 2 10:15:58 2022] PUT /identity/v3/projects/4bc7ee5e1e7f4f5c8d9de8fa1e48991b/users/e881ae7c60254c12860fff0a40480641/roles/e1c04b19fc4543b8b54f58bbcffbc598 => generated 0 bytes in 120 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 02 10:15:58.536836 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8192fb5-a6c0-4e30-828b-db4a37a3ed17 admin admin] Authenticating user token {{(pid=80377) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:15:58.554293 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG dbcounter [-] [80378] Writing DB stats keystone:SELECT=93,keystone:INSERT=7 {{(pid=80378) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 10:15:58.606949 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1aaffdcd-2aa0-4698-9bf4-4e591aaf59f5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '891005fa5ff74041b75747d16cc66798', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7047912c5acd4adeba5d87c3bf895e39', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80378) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:15:58.607615 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1aaffdcd-2aa0-4698-9bf4-4e591aaf59f5 admin admin] REQUEST_METHOD: `GET` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:15:58.607902 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1aaffdcd-2aa0-4698-9bf4-4e591aaf59f5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:15:58.608185 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1aaffdcd-2aa0-4698-9bf4-4e591aaf59f5 admin admin] PATH_INFO: `/v3/roles` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:15:58.608925 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1aaffdcd-2aa0-4698-9bf4-4e591aaf59f5 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:15:58.610684 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1aaffdcd-2aa0-4698-9bf4-4e591aaf59f5 admin admin] RBAC: Authorization granted {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:15:58.612345 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8192fb5-a6c0-4e30-828b-db4a37a3ed17 admin admin] Validating token access rules against request {{(pid=80377) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:15:58.618748 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e625c0fb-f9c8-43e5-a2c2-1b17c6925e77 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '891005fa5ff74041b75747d16cc66798', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7047912c5acd4adeba5d87c3bf895e39', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80377) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:15:58.620170 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e625c0fb-f9c8-43e5-a2c2-1b17c6925e77 admin admin] REQUEST_METHOD: `GET` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:15:58.620617 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e625c0fb-f9c8-43e5-a2c2-1b17c6925e77 admin admin] SCRIPT_NAME: `/identity` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:15:58.621042 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e625c0fb-f9c8-43e5-a2c2-1b17c6925e77 admin admin] PATH_INFO: `/v3/roles` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:15:58.622505 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 789/1597] 10.222.0.82 () {58 vars in 1122 bytes} [Tue Aug 2 10:15:58 2022] GET /identity/v3/roles => generated 1420 bytes in 100 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 02 10:15:58.622889 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e625c0fb-f9c8-43e5-a2c2-1b17c6925e77 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:15:58.623279 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1aaffdcd-2aa0-4698-9bf4-4e591aaf59f5 admin admin] Authenticating user token {{(pid=80378) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:15:58.624848 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e625c0fb-f9c8-43e5-a2c2-1b17c6925e77 admin admin] RBAC: Authorization granted {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:15:58.635781 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1aaffdcd-2aa0-4698-9bf4-4e591aaf59f5 admin admin] Validating token access rules against request {{(pid=80378) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:15:58.636868 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: [pid: 80377|app: 0|req: 809/1598] 10.222.0.82 () {58 vars in 1122 bytes} [Tue Aug 2 10:15:58 2022] GET /identity/v3/roles => generated 1420 bytes in 100 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 02 10:15:58.638126 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de5f6bbf-7769-4063-90e2-8b242816c0bc None None] REQUEST_METHOD: `POST` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:15:58.638417 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de5f6bbf-7769-4063-90e2-8b242816c0bc None None] SCRIPT_NAME: `/identity` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:15:58.638692 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de5f6bbf-7769-4063-90e2-8b242816c0bc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:15:58.643793 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42c51a02-60d5-4456-9af5-e2426d7d8fbf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '891005fa5ff74041b75747d16cc66798', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7047912c5acd4adeba5d87c3bf895e39', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80378) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:15:58.644926 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42c51a02-60d5-4456-9af5-e2426d7d8fbf admin admin] REQUEST_METHOD: `PUT` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:15:58.645365 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42c51a02-60d5-4456-9af5-e2426d7d8fbf admin admin] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:15:58.645961 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42c51a02-60d5-4456-9af5-e2426d7d8fbf admin admin] PATH_INFO: `/v3/projects/719a14aab1034164a033ddcb985695c1/users/8debb5e1e38a40159eb1713169f93cc0/roles/e1c04b19fc4543b8b54f58bbcffbc598` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:15:58.667323 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42c51a02-60d5-4456-9af5-e2426d7d8fbf admin admin] RBAC: Authorizing `identity:create_grant(project_id=719a14aab1034164a033ddcb985695c1, user_id=8debb5e1e38a40159eb1713169f93cc0, role_id=e1c04b19fc4543b8b54f58bbcffbc598)` {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:15:58.671156 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42c51a02-60d5-4456-9af5-e2426d7d8fbf admin admin] RBAC: Authorization granted {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:15:58.672100 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.auth.core [None req-de5f6bbf-7769-4063-90e2-8b242816c0bc None None] MFA Rules not processed for user `b4624993d8e94d3ba204d1f8b71f216e`. Rule list: `[]` (Enabled: `True`). {{(pid=80377) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 02 10:15:58.673734 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.fernet_utils [None req-de5f6bbf-7769-4063-90e2-8b242816c0bc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80377) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 10:15:58.692285 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 790/1599] 10.222.0.82 () {60 vars in 1483 bytes} [Tue Aug 2 10:15:58 2022] PUT /identity/v3/projects/719a14aab1034164a033ddcb985695c1/users/8debb5e1e38a40159eb1713169f93cc0/roles/e1c04b19fc4543b8b54f58bbcffbc598 => generated 0 bytes in 69 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 02 10:15:58.695039 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42c51a02-60d5-4456-9af5-e2426d7d8fbf admin admin] Authenticating user token {{(pid=80378) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:15:58.773952 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: [pid: 80377|app: 0|req: 810/1600] 10.222.0.82 () {58 vars in 958 bytes} [Tue Aug 2 10:15:58 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 137 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 02 10:15:58.775475 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de5f6bbf-7769-4063-90e2-8b242816c0bc None None] Authenticating user token {{(pid=80377) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:15:58.789543 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de5f6bbf-7769-4063-90e2-8b242816c0bc None None] Validating token access rules against request {{(pid=80377) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:15:58.797080 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18e6c7b5-f578-4560-a90e-afe36f12a5f7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '891005fa5ff74041b75747d16cc66798', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7047912c5acd4adeba5d87c3bf895e39', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80377) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:15:58.797699 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18e6c7b5-f578-4560-a90e-afe36f12a5f7 admin admin] REQUEST_METHOD: `PUT` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:15:58.798031 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18e6c7b5-f578-4560-a90e-afe36f12a5f7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:15:58.798309 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18e6c7b5-f578-4560-a90e-afe36f12a5f7 admin admin] PATH_INFO: `/v3/projects/ce6b9eda623740608e0c9998aaa74dab/users/99616b362eac4b7dba445dafd3ca1426/roles/e1c04b19fc4543b8b54f58bbcffbc598` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:15:58.804627 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42c51a02-60d5-4456-9af5-e2426d7d8fbf admin admin] Validating token access rules against request {{(pid=80378) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:15:58.813150 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2513cdad-fc64-40bd-ba3e-23483b9d37ac admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '891005fa5ff74041b75747d16cc66798', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7047912c5acd4adeba5d87c3bf895e39', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80378) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:15:58.813985 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2513cdad-fc64-40bd-ba3e-23483b9d37ac admin admin] REQUEST_METHOD: `GET` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:15:58.814346 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2513cdad-fc64-40bd-ba3e-23483b9d37ac admin admin] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:15:58.814627 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2513cdad-fc64-40bd-ba3e-23483b9d37ac admin admin] PATH_INFO: `/v3/roles` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:15:58.815401 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2513cdad-fc64-40bd-ba3e-23483b9d37ac admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:15:58.817664 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2513cdad-fc64-40bd-ba3e-23483b9d37ac admin admin] RBAC: Authorization granted {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:15:58.831178 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 791/1601] 10.222.0.82 () {58 vars in 1122 bytes} [Tue Aug 2 10:15:58 2022] GET /identity/v3/roles => generated 1420 bytes in 138 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 02 10:15:58.833871 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d2d50f4-d75c-45ed-9ca7-61d781fcfb23 None None] REQUEST_METHOD: `POST` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:15:58.834170 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d2d50f4-d75c-45ed-9ca7-61d781fcfb23 None None] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:15:58.834453 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d2d50f4-d75c-45ed-9ca7-61d781fcfb23 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:15:58.836953 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18e6c7b5-f578-4560-a90e-afe36f12a5f7 admin admin] RBAC: Authorizing `identity:create_grant(project_id=ce6b9eda623740608e0c9998aaa74dab, user_id=99616b362eac4b7dba445dafd3ca1426, role_id=e1c04b19fc4543b8b54f58bbcffbc598)` {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:15:58.840130 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18e6c7b5-f578-4560-a90e-afe36f12a5f7 admin admin] RBAC: Authorization granted {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:15:58.857906 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.auth.core [None req-4d2d50f4-d75c-45ed-9ca7-61d781fcfb23 None None] MFA Rules not processed for user `891005fa5ff74041b75747d16cc66798`. Rule list: `[]` (Enabled: `True`). {{(pid=80378) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 02 10:15:58.859281 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.fernet_utils [None req-4d2d50f4-d75c-45ed-9ca7-61d781fcfb23 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80378) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 10:15:58.869363 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: [pid: 80377|app: 0|req: 811/1602] 10.222.0.82 () {60 vars in 1483 bytes} [Tue Aug 2 10:15:58 2022] PUT /identity/v3/projects/ce6b9eda623740608e0c9998aaa74dab/users/99616b362eac4b7dba445dafd3ca1426/roles/e1c04b19fc4543b8b54f58bbcffbc598 => generated 0 bytes in 94 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 02 10:15:58.870561 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18e6c7b5-f578-4560-a90e-afe36f12a5f7 admin admin] Authenticating user token {{(pid=80377) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:15:58.945548 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 792/1603] 10.222.0.82 () {58 vars in 958 bytes} [Tue Aug 2 10:15:58 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 112 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 02 10:15:58.946259 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG dbcounter [-] [80377] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80377) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 10:15:58.949549 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d2d50f4-d75c-45ed-9ca7-61d781fcfb23 None None] Authenticating user token {{(pid=80378) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:15:58.961270 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d2d50f4-d75c-45ed-9ca7-61d781fcfb23 None None] Validating token access rules against request {{(pid=80378) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:15:58.966766 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18e6c7b5-f578-4560-a90e-afe36f12a5f7 admin admin] Validating token access rules against request {{(pid=80377) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:15:58.969386 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0a00875-b1b3-4e26-96ee-c8959c15be24 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '891005fa5ff74041b75747d16cc66798', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7047912c5acd4adeba5d87c3bf895e39', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80378) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:15:58.970302 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0a00875-b1b3-4e26-96ee-c8959c15be24 admin admin] REQUEST_METHOD: `PUT` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:15:58.971163 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0a00875-b1b3-4e26-96ee-c8959c15be24 admin admin] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:15:58.971763 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0a00875-b1b3-4e26-96ee-c8959c15be24 admin admin] PATH_INFO: `/v3/projects/23714720faaa4562b694d7e3fbda2c33/users/b6890a8127e04092920cb1d31ab56d91/roles/e1c04b19fc4543b8b54f58bbcffbc598` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:15:58.973390 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5308aaf3-9c61-4d9a-b810-fb382b4e5ee9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '891005fa5ff74041b75747d16cc66798', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7047912c5acd4adeba5d87c3bf895e39', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80377) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:15:58.974098 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5308aaf3-9c61-4d9a-b810-fb382b4e5ee9 admin admin] REQUEST_METHOD: `GET` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:15:58.974382 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5308aaf3-9c61-4d9a-b810-fb382b4e5ee9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:15:58.974663 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5308aaf3-9c61-4d9a-b810-fb382b4e5ee9 admin admin] PATH_INFO: `/v3/roles` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:15:58.975375 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5308aaf3-9c61-4d9a-b810-fb382b4e5ee9 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:15:58.977070 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5308aaf3-9c61-4d9a-b810-fb382b4e5ee9 admin admin] RBAC: Authorization granted {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:15:58.988528 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: [pid: 80377|app: 0|req: 812/1604] 10.222.0.82 () {58 vars in 1122 bytes} [Tue Aug 2 10:15:58 2022] GET /identity/v3/roles => generated 1420 bytes in 118 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 02 10:15:58.989981 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbb8b13d-60eb-45f4-ba18-15b9b3f0502a None None] REQUEST_METHOD: `POST` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:15:58.990214 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbb8b13d-60eb-45f4-ba18-15b9b3f0502a None None] SCRIPT_NAME: `/identity` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:15:58.990491 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbb8b13d-60eb-45f4-ba18-15b9b3f0502a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:15:58.995547 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0a00875-b1b3-4e26-96ee-c8959c15be24 admin admin] RBAC: Authorizing `identity:create_grant(project_id=23714720faaa4562b694d7e3fbda2c33, user_id=b6890a8127e04092920cb1d31ab56d91, role_id=e1c04b19fc4543b8b54f58bbcffbc598)` {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:15:59.000480 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0a00875-b1b3-4e26-96ee-c8959c15be24 admin admin] RBAC: Authorization granted {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:15:59.014447 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.auth.core [None req-dbb8b13d-60eb-45f4-ba18-15b9b3f0502a None None] MFA Rules not processed for user `891005fa5ff74041b75747d16cc66798`. Rule list: `[]` (Enabled: `True`). {{(pid=80377) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 02 10:15:59.015684 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.fernet_utils [None req-dbb8b13d-60eb-45f4-ba18-15b9b3f0502a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80377) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 10:15:59.023168 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 793/1605] 10.222.0.82 () {60 vars in 1483 bytes} [Tue Aug 2 10:15:58 2022] PUT /identity/v3/projects/23714720faaa4562b694d7e3fbda2c33/users/b6890a8127e04092920cb1d31ab56d91/roles/e1c04b19fc4543b8b54f58bbcffbc598 => generated 0 bytes in 77 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 02 10:15:59.024890 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24cfdce0-a160-4733-b563-7984c92f6c9a None None] REQUEST_METHOD: `GET` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:15:59.025640 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24cfdce0-a160-4733-b563-7984c92f6c9a None None] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:15:59.025958 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24cfdce0-a160-4733-b563-7984c92f6c9a None None] PATH_INFO: `/` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:15:59.027707 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 794/1606] 10.222.0.82 () {54 vars in 962 bytes} [Tue Aug 2 10:15:59 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 02 10:15:59.029172 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24cfdce0-a160-4733-b563-7984c92f6c9a None None] Authenticating user token {{(pid=80378) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:15:59.041861 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG dbcounter [-] [80378] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80378) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 10:15:59.133439 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24cfdce0-a160-4733-b563-7984c92f6c9a None None] Validating token access rules against request {{(pid=80378) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:15:59.140356 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a876e13-b2dc-44dc-a848-28305121559e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '891005fa5ff74041b75747d16cc66798', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7047912c5acd4adeba5d87c3bf895e39', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80378) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:15:59.141996 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a876e13-b2dc-44dc-a848-28305121559e admin admin] REQUEST_METHOD: `PUT` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:15:59.142289 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a876e13-b2dc-44dc-a848-28305121559e admin admin] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:15:59.142568 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a876e13-b2dc-44dc-a848-28305121559e admin admin] PATH_INFO: `/v3/projects/b349cf4ab586491097572ca9556b268f/users/4db0cf73ea0944e98d94446749e28b44/roles/e1c04b19fc4543b8b54f58bbcffbc598` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:15:59.147527 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: [pid: 80377|app: 0|req: 813/1607] 10.222.0.82 () {58 vars in 958 bytes} [Tue Aug 2 10:15:58 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 158 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 02 10:15:59.149542 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed6de262-3a87-4b8e-8910-77d84369268e None None] REQUEST_METHOD: `POST` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:15:59.150016 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed6de262-3a87-4b8e-8910-77d84369268e None None] SCRIPT_NAME: `/identity` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:15:59.151389 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed6de262-3a87-4b8e-8910-77d84369268e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:15:59.174470 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a876e13-b2dc-44dc-a848-28305121559e admin admin] RBAC: Authorizing `identity:create_grant(project_id=b349cf4ab586491097572ca9556b268f, user_id=4db0cf73ea0944e98d94446749e28b44, role_id=e1c04b19fc4543b8b54f58bbcffbc598)` {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:15:59.177681 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a876e13-b2dc-44dc-a848-28305121559e admin admin] RBAC: Authorization granted {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:15:59.178202 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.auth.core [None req-ed6de262-3a87-4b8e-8910-77d84369268e None None] MFA Rules not processed for user `891005fa5ff74041b75747d16cc66798`. Rule list: `[]` (Enabled: `True`). {{(pid=80377) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 02 10:15:59.179422 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.fernet_utils [None req-ed6de262-3a87-4b8e-8910-77d84369268e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80377) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 10:15:59.193298 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: [pid: 80377|app: 0|req: 814/1608] 10.222.0.82 () {58 vars in 958 bytes} [Tue Aug 2 10:15:59 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 45 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 02 10:15:59.194365 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed6de262-3a87-4b8e-8910-77d84369268e None None] Authenticating user token {{(pid=80377) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:15:59.215104 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 795/1609] 10.222.0.82 () {60 vars in 1483 bytes} [Tue Aug 2 10:15:59 2022] PUT /identity/v3/projects/b349cf4ab586491097572ca9556b268f/users/4db0cf73ea0944e98d94446749e28b44/roles/e1c04b19fc4543b8b54f58bbcffbc598 => generated 0 bytes in 176 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 02 10:15:59.215104 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a876e13-b2dc-44dc-a848-28305121559e admin admin] Authenticating user token {{(pid=80378) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:15:59.278383 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed6de262-3a87-4b8e-8910-77d84369268e None None] Validating token access rules against request {{(pid=80377) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:15:59.296201 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a876e13-b2dc-44dc-a848-28305121559e admin admin] Validating token access rules against request {{(pid=80378) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:15:59.306401 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-722cbc54-50f2-4a0a-b562-1847bd9d88ff admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '891005fa5ff74041b75747d16cc66798', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7047912c5acd4adeba5d87c3bf895e39', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80378) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:15:59.308226 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-722cbc54-50f2-4a0a-b562-1847bd9d88ff admin admin] REQUEST_METHOD: `PUT` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:15:59.308685 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-722cbc54-50f2-4a0a-b562-1847bd9d88ff admin admin] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:15:59.310069 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-722cbc54-50f2-4a0a-b562-1847bd9d88ff admin admin] PATH_INFO: `/v3/projects/18813e93dc5249d3a0fa816213f95375/users/c547f60c1d234f0eb2f2159c81368bba/roles/e1c04b19fc4543b8b54f58bbcffbc598` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:15:59.332859 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-722cbc54-50f2-4a0a-b562-1847bd9d88ff admin admin] RBAC: Authorizing `identity:create_grant(project_id=18813e93dc5249d3a0fa816213f95375, user_id=c547f60c1d234f0eb2f2159c81368bba, role_id=e1c04b19fc4543b8b54f58bbcffbc598)` {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:15:59.337409 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-722cbc54-50f2-4a0a-b562-1847bd9d88ff admin admin] RBAC: Authorization granted {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:15:59.345567 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3aa17dbc-8a83-4172-98fc-7379d461526b service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '33b87170f0634b67bce60750847b5cbe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f1b38ebb804c41c69772c998aae72c41', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80377) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:15:59.346426 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3aa17dbc-8a83-4172-98fc-7379d461526b service neutron] REQUEST_METHOD: `GET` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:15:59.346730 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3aa17dbc-8a83-4172-98fc-7379d461526b service neutron] SCRIPT_NAME: `/identity` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:15:59.347076 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3aa17dbc-8a83-4172-98fc-7379d461526b service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:15:59.360566 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 796/1610] 10.222.0.82 () {60 vars in 1483 bytes} [Tue Aug 2 10:15:59 2022] PUT /identity/v3/projects/18813e93dc5249d3a0fa816213f95375/users/c547f60c1d234f0eb2f2159c81368bba/roles/e1c04b19fc4543b8b54f58bbcffbc598 => generated 0 bytes in 155 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 02 10:15:59.362894 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-494d179b-db91-462b-ab94-125c8f72b554 None None] REQUEST_METHOD: `POST` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:15:59.363493 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-494d179b-db91-462b-ab94-125c8f72b554 None None] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:15:59.363959 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-494d179b-db91-462b-ab94-125c8f72b554 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:15:59.375451 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3aa17dbc-8a83-4172-98fc-7379d461526b service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:15:59.377380 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3aa17dbc-8a83-4172-98fc-7379d461526b service neutron] RBAC: Authorization granted {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:15:59.390147 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: [pid: 80377|app: 0|req: 815/1611] 10.222.0.82 () {60 vars in 1365 bytes} [Tue Aug 2 10:15:59 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 196 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 02 10:15:59.397338 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.auth.core [None req-494d179b-db91-462b-ab94-125c8f72b554 None None] MFA Rules not processed for user `891005fa5ff74041b75747d16cc66798`. Rule list: `[]` (Enabled: `True`). {{(pid=80378) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 02 10:15:59.398661 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.fernet_utils [None req-494d179b-db91-462b-ab94-125c8f72b554 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80378) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 10:15:59.403832 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f7e5520-d49e-47b3-bd8a-56fbb6e5af64 None None] REQUEST_METHOD: `POST` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:15:59.406804 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f7e5520-d49e-47b3-bd8a-56fbb6e5af64 None None] SCRIPT_NAME: `/identity` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:15:59.407167 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f7e5520-d49e-47b3-bd8a-56fbb6e5af64 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:15:59.509560 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.auth.core [None req-8f7e5520-d49e-47b3-bd8a-56fbb6e5af64 None None] MFA Rules not processed for user `88af518196674fe593ad88d30111fb7d`. Rule list: `[]` (Enabled: `True`). {{(pid=80377) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 02 10:15:59.510355 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 797/1612] 10.222.0.82 () {58 vars in 958 bytes} [Tue Aug 2 10:15:59 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 149 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 02 10:15:59.512149 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-494d179b-db91-462b-ab94-125c8f72b554 None None] Authenticating user token {{(pid=80378) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:15:59.513316 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.fernet_utils [None req-8f7e5520-d49e-47b3-bd8a-56fbb6e5af64 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80377) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 10:15:59.530836 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG dbcounter [-] [80378] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80378) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 10:15:59.569490 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG dbcounter [-] [80377] Writing DB stats keystone:SELECT=100 {{(pid=80377) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 10:15:59.652742 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-494d179b-db91-462b-ab94-125c8f72b554 None None] Validating token access rules against request {{(pid=80378) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:15:59.663890 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9891cd2f-9cf6-48f6-8ed0-1dbf53dd71c3 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '33b87170f0634b67bce60750847b5cbe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f1b38ebb804c41c69772c998aae72c41', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80378) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:15:59.663890 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9891cd2f-9cf6-48f6-8ed0-1dbf53dd71c3 service neutron] REQUEST_METHOD: `GET` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:15:59.663890 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9891cd2f-9cf6-48f6-8ed0-1dbf53dd71c3 service neutron] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:15:59.663890 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9891cd2f-9cf6-48f6-8ed0-1dbf53dd71c3 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:15:59.670617 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: [pid: 80377|app: 0|req: 816/1613] 10.222.0.82 () {58 vars in 1059 bytes} [Tue Aug 2 10:15:59 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 269 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 02 10:15:59.672079 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f7e5520-d49e-47b3-bd8a-56fbb6e5af64 None None] Authenticating user token {{(pid=80377) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:15:59.684980 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f7e5520-d49e-47b3-bd8a-56fbb6e5af64 None None] Validating token access rules against request {{(pid=80377) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:15:59.690610 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6a79362-67d8-48c9-96fa-33048988fc99 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '33b87170f0634b67bce60750847b5cbe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f1b38ebb804c41c69772c998aae72c41', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80377) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:15:59.692082 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6a79362-67d8-48c9-96fa-33048988fc99 service neutron] REQUEST_METHOD: `GET` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:15:59.692532 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6a79362-67d8-48c9-96fa-33048988fc99 service neutron] SCRIPT_NAME: `/identity` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:15:59.693495 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9891cd2f-9cf6-48f6-8ed0-1dbf53dd71c3 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:15:59.693910 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6a79362-67d8-48c9-96fa-33048988fc99 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:15:59.695435 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9891cd2f-9cf6-48f6-8ed0-1dbf53dd71c3 service neutron] RBAC: Authorization granted {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:15:59.709231 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6a79362-67d8-48c9-96fa-33048988fc99 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:15:59.711183 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6a79362-67d8-48c9-96fa-33048988fc99 service neutron] RBAC: Authorization granted {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:15:59.712653 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 798/1614] 10.222.0.82 () {60 vars in 1365 bytes} [Tue Aug 2 10:15:59 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 202 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 02 10:15:59.714034 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28ba8035-ad1c-4a64-af22-0cc571e15626 None None] REQUEST_METHOD: `POST` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:15:59.714328 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28ba8035-ad1c-4a64-af22-0cc571e15626 None None] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:15:59.714666 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28ba8035-ad1c-4a64-af22-0cc571e15626 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:15:59.733163 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: [pid: 80377|app: 0|req: 817/1615] 10.222.0.82 () {60 vars in 1365 bytes} [Tue Aug 2 10:15:59 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 62 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 02 10:15:59.734501 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-658d8dcf-0c9c-4f29-8197-939dc785662a None None] REQUEST_METHOD: `POST` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:15:59.734791 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-658d8dcf-0c9c-4f29-8197-939dc785662a None None] SCRIPT_NAME: `/identity` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:15:59.735065 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-658d8dcf-0c9c-4f29-8197-939dc785662a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:15:59.760284 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.auth.core [None req-28ba8035-ad1c-4a64-af22-0cc571e15626 None None] MFA Rules not processed for user `4db0cf73ea0944e98d94446749e28b44`. Rule list: `[]` (Enabled: `True`). {{(pid=80378) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 02 10:15:59.764043 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.fernet_utils [None req-28ba8035-ad1c-4a64-af22-0cc571e15626 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80378) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 10:15:59.807876 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.auth.core [None req-658d8dcf-0c9c-4f29-8197-939dc785662a None None] MFA Rules not processed for user `c547f60c1d234f0eb2f2159c81368bba`. Rule list: `[]` (Enabled: `True`). {{(pid=80377) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 02 10:15:59.809430 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.fernet_utils [None req-658d8dcf-0c9c-4f29-8197-939dc785662a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80377) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 10:15:59.960185 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 799/1616] 10.222.0.82 () {58 vars in 958 bytes} [Tue Aug 2 10:15:59 2022] POST /identity/v3/auth/tokens => generated 3244 bytes in 236 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 02 10:15:59.960185 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28ba8035-ad1c-4a64-af22-0cc571e15626 None None] Authenticating user token {{(pid=80378) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:15:59.981061 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: [pid: 80377|app: 0|req: 818/1617] 10.222.0.82 () {58 vars in 958 bytes} [Tue Aug 2 10:15:59 2022] POST /identity/v3/auth/tokens => generated 3238 bytes in 247 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 02 10:15:59.982570 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-658d8dcf-0c9c-4f29-8197-939dc785662a None None] Authenticating user token {{(pid=80377) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:15:59.987969 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28ba8035-ad1c-4a64-af22-0cc571e15626 None None] Validating token access rules against request {{(pid=80378) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:15:59.992784 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24f08f4d-0ad3-43d8-9050-8e4ee3481db5 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '33b87170f0634b67bce60750847b5cbe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f1b38ebb804c41c69772c998aae72c41', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80378) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:15:59.993951 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24f08f4d-0ad3-43d8-9050-8e4ee3481db5 service neutron] REQUEST_METHOD: `GET` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:15:59.994427 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24f08f4d-0ad3-43d8-9050-8e4ee3481db5 service neutron] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:15:59.994895 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24f08f4d-0ad3-43d8-9050-8e4ee3481db5 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:16:00.009622 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-658d8dcf-0c9c-4f29-8197-939dc785662a None None] Validating token access rules against request {{(pid=80377) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:16:00.018622 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24f08f4d-0ad3-43d8-9050-8e4ee3481db5 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:16:00.019977 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e7d5d9f-cc4c-4de8-b567-c247b86d540d service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88af518196674fe593ad88d30111fb7d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f1b38ebb804c41c69772c998aae72c41', 'project_domain_id': 'default', 'roles': ['admin', 'ResellerAdmin', 'reader', 'service', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80377) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:16:00.020675 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e7d5d9f-cc4c-4de8-b567-c247b86d540d service nova] REQUEST_METHOD: `GET` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:16:00.021316 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24f08f4d-0ad3-43d8-9050-8e4ee3481db5 service neutron] RBAC: Authorization granted {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:16:00.021440 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e7d5d9f-cc4c-4de8-b567-c247b86d540d service nova] SCRIPT_NAME: `/identity` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:16:00.021837 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e7d5d9f-cc4c-4de8-b567-c247b86d540d service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:16:00.035370 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 800/1618] 10.222.0.82 () {60 vars in 1365 bytes} [Tue Aug 2 10:15:59 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 85 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 02 10:16:00.036831 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24f08f4d-0ad3-43d8-9050-8e4ee3481db5 service neutron] Authenticating user token {{(pid=80378) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:16:00.145527 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e7d5d9f-cc4c-4de8-b567-c247b86d540d service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:16:00.149489 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e7d5d9f-cc4c-4de8-b567-c247b86d540d service nova] RBAC: Authorization granted {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:16:00.155718 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24f08f4d-0ad3-43d8-9050-8e4ee3481db5 service neutron] Validating token access rules against request {{(pid=80378) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:16:00.163746 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: [pid: 80377|app: 0|req: 819/1619] 10.222.0.82 () {60 vars in 1365 bytes} [Tue Aug 2 10:15:59 2022] GET /identity/v3/auth/tokens => generated 3250 bytes in 182 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 02 10:16:00.165106 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e7d5d9f-cc4c-4de8-b567-c247b86d540d service nova] Authenticating user token {{(pid=80377) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:16:00.174750 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6137b9b-cf12-4cfa-b7fb-826dd8896764 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1023de281ae74e9d8a6d6c312c1c97a7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f1b38ebb804c41c69772c998aae72c41', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80378) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:16:00.177542 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6137b9b-cf12-4cfa-b7fb-826dd8896764 service cinder] REQUEST_METHOD: `GET` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:16:00.178196 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6137b9b-cf12-4cfa-b7fb-826dd8896764 service cinder] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:16:00.178794 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6137b9b-cf12-4cfa-b7fb-826dd8896764 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:16:00.183866 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e7d5d9f-cc4c-4de8-b567-c247b86d540d service nova] Validating token access rules against request {{(pid=80377) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:16:00.190346 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4047b505-baf3-4b79-a46f-1a8a64815985 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '891005fa5ff74041b75747d16cc66798', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7047912c5acd4adeba5d87c3bf895e39', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80377) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:16:00.190768 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4047b505-baf3-4b79-a46f-1a8a64815985 admin admin] REQUEST_METHOD: `POST` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:16:00.191144 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4047b505-baf3-4b79-a46f-1a8a64815985 admin admin] SCRIPT_NAME: `/identity` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:16:00.191521 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4047b505-baf3-4b79-a46f-1a8a64815985 admin admin] PATH_INFO: `/v3/projects` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:16:00.192658 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4047b505-baf3-4b79-a46f-1a8a64815985 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:16:00.194415 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6137b9b-cf12-4cfa-b7fb-826dd8896764 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:16:00.196522 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4047b505-baf3-4b79-a46f-1a8a64815985 admin admin] RBAC: Authorization granted {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:16:00.196687 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6137b9b-cf12-4cfa-b7fb-826dd8896764 service cinder] RBAC: Authorization granted {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:16:00.229148 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 801/1620] 10.222.0.82 () {60 vars in 1365 bytes} [Tue Aug 2 10:16:00 2022] GET /identity/v3/auth/tokens => generated 3244 bytes in 193 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 02 10:16:00.241174 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: [pid: 80377|app: 0|req: 820/1621] 10.222.0.82 () {60 vars in 1153 bytes} [Tue Aug 2 10:16:00 2022] POST /identity/v3/projects => generated 361 bytes in 77 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 02 10:16:00.245245 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6137b9b-cf12-4cfa-b7fb-826dd8896764 service cinder] Authenticating user token {{(pid=80378) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:16:00.265544 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6137b9b-cf12-4cfa-b7fb-826dd8896764 service cinder] Validating token access rules against request {{(pid=80378) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:16:00.314946 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG dbcounter [-] [80378] Writing DB stats keystone:SELECT=100 {{(pid=80378) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 10:16:00.355795 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cec450e6-bd4d-4267-a307-4a5a6e06f9d5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '891005fa5ff74041b75747d16cc66798', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7047912c5acd4adeba5d87c3bf895e39', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80378) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:16:00.356841 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cec450e6-bd4d-4267-a307-4a5a6e06f9d5 admin admin] REQUEST_METHOD: `POST` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:16:00.357342 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cec450e6-bd4d-4267-a307-4a5a6e06f9d5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:16:00.357864 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cec450e6-bd4d-4267-a307-4a5a6e06f9d5 admin admin] PATH_INFO: `/v3/users` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:16:00.358824 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cec450e6-bd4d-4267-a307-4a5a6e06f9d5 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:16:00.362189 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cec450e6-bd4d-4267-a307-4a5a6e06f9d5 admin admin] RBAC: Authorization granted {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:16:00.398270 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 802/1622] 10.222.0.82 () {60 vars in 1144 bytes} [Tue Aug 2 10:16:00 2022] POST /identity/v3/users => generated 287 bytes in 153 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 02 10:16:00.401869 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4047b505-baf3-4b79-a46f-1a8a64815985 admin admin] Authenticating user token {{(pid=80377) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:16:00.436189 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4047b505-baf3-4b79-a46f-1a8a64815985 admin admin] Validating token access rules against request {{(pid=80377) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:16:00.447607 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdcf158b-d60a-4df2-a94b-67a7ea1dfd04 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '891005fa5ff74041b75747d16cc66798', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7047912c5acd4adeba5d87c3bf895e39', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80377) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:16:00.449556 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdcf158b-d60a-4df2-a94b-67a7ea1dfd04 admin admin] REQUEST_METHOD: `GET` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:16:00.450381 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdcf158b-d60a-4df2-a94b-67a7ea1dfd04 admin admin] SCRIPT_NAME: `/identity` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:16:00.450889 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdcf158b-d60a-4df2-a94b-67a7ea1dfd04 admin admin] PATH_INFO: `/v3/roles` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:16:00.454023 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bdcf158b-d60a-4df2-a94b-67a7ea1dfd04 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:16:00.455914 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bdcf158b-d60a-4df2-a94b-67a7ea1dfd04 admin admin] RBAC: Authorization granted {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:16:00.483302 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: [pid: 80377|app: 0|req: 821/1623] 10.222.0.82 () {58 vars in 1122 bytes} [Tue Aug 2 10:16:00 2022] GET /identity/v3/roles => generated 1420 bytes in 82 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 02 10:16:00.488007 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cec450e6-bd4d-4267-a307-4a5a6e06f9d5 admin admin] Authenticating user token {{(pid=80378) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:16:00.507848 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cec450e6-bd4d-4267-a307-4a5a6e06f9d5 admin admin] Validating token access rules against request {{(pid=80378) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:16:00.515412 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b5ed240-afec-43cd-85a4-9d31e4bd1c00 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '891005fa5ff74041b75747d16cc66798', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7047912c5acd4adeba5d87c3bf895e39', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80378) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:16:00.516618 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b5ed240-afec-43cd-85a4-9d31e4bd1c00 admin admin] REQUEST_METHOD: `PUT` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:16:00.517377 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b5ed240-afec-43cd-85a4-9d31e4bd1c00 admin admin] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:16:00.517377 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b5ed240-afec-43cd-85a4-9d31e4bd1c00 admin admin] PATH_INFO: `/v3/projects/a5aafa32be16445a98f61c66fd099973/users/6e6ba698dd2940ae9d3c4e416986aed9/roles/237966442f05411191105bf336bc14f3` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:16:00.536947 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b5ed240-afec-43cd-85a4-9d31e4bd1c00 admin admin] RBAC: Authorizing `identity:create_grant(project_id=a5aafa32be16445a98f61c66fd099973, user_id=6e6ba698dd2940ae9d3c4e416986aed9, role_id=237966442f05411191105bf336bc14f3)` {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:16:00.540053 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b5ed240-afec-43cd-85a4-9d31e4bd1c00 admin admin] RBAC: Authorization granted {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:16:00.571116 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 803/1624] 10.222.0.82 () {60 vars in 1483 bytes} [Tue Aug 2 10:16:00 2022] PUT /identity/v3/projects/a5aafa32be16445a98f61c66fd099973/users/6e6ba698dd2940ae9d3c4e416986aed9/roles/237966442f05411191105bf336bc14f3 => generated 0 bytes in 81 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 02 10:16:00.572702 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdcf158b-d60a-4df2-a94b-67a7ea1dfd04 admin admin] Authenticating user token {{(pid=80377) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:16:00.580512 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG dbcounter [-] [80377] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80377) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 10:16:00.593391 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdcf158b-d60a-4df2-a94b-67a7ea1dfd04 admin admin] Validating token access rules against request {{(pid=80377) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:16:00.723215 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5fdc595-dfef-4111-9cf9-2e8af0483330 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '891005fa5ff74041b75747d16cc66798', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7047912c5acd4adeba5d87c3bf895e39', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80377) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:16:00.724064 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5fdc595-dfef-4111-9cf9-2e8af0483330 admin admin] REQUEST_METHOD: `GET` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:16:00.724451 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5fdc595-dfef-4111-9cf9-2e8af0483330 admin admin] SCRIPT_NAME: `/identity` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:16:00.725473 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5fdc595-dfef-4111-9cf9-2e8af0483330 admin admin] PATH_INFO: `/v3/roles` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:16:00.726336 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5fdc595-dfef-4111-9cf9-2e8af0483330 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:16:00.738820 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5fdc595-dfef-4111-9cf9-2e8af0483330 admin admin] RBAC: Authorization granted {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:16:00.757513 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: [pid: 80377|app: 0|req: 822/1625] 10.222.0.82 () {58 vars in 1122 bytes} [Tue Aug 2 10:16:00 2022] GET /identity/v3/roles => generated 1420 bytes in 183 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 02 10:16:00.758965 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b5ed240-afec-43cd-85a4-9d31e4bd1c00 admin admin] Authenticating user token {{(pid=80378) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:16:00.771263 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b5ed240-afec-43cd-85a4-9d31e4bd1c00 admin admin] Validating token access rules against request {{(pid=80378) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:16:00.780464 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0c6fbcd-62f0-4150-8205-470b5b5e9488 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '891005fa5ff74041b75747d16cc66798', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7047912c5acd4adeba5d87c3bf895e39', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80378) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:16:00.781273 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0c6fbcd-62f0-4150-8205-470b5b5e9488 admin admin] REQUEST_METHOD: `PUT` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:16:00.781662 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0c6fbcd-62f0-4150-8205-470b5b5e9488 admin admin] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:16:00.782148 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0c6fbcd-62f0-4150-8205-470b5b5e9488 admin admin] PATH_INFO: `/v3/projects/a5aafa32be16445a98f61c66fd099973/users/6e6ba698dd2940ae9d3c4e416986aed9/roles/237966442f05411191105bf336bc14f3` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:16:00.784749 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0c6fbcd-62f0-4150-8205-470b5b5e9488 admin admin] RBAC: Authorizing `identity:create_grant(project_id=a5aafa32be16445a98f61c66fd099973, user_id=6e6ba698dd2940ae9d3c4e416986aed9, role_id=237966442f05411191105bf336bc14f3)` {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:16:00.788896 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0c6fbcd-62f0-4150-8205-470b5b5e9488 admin admin] RBAC: Authorization granted {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:16:00.816524 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 804/1626] 10.222.0.82 () {60 vars in 1483 bytes} [Tue Aug 2 10:16:00 2022] PUT /identity/v3/projects/a5aafa32be16445a98f61c66fd099973/users/6e6ba698dd2940ae9d3c4e416986aed9/roles/237966442f05411191105bf336bc14f3 => generated 0 bytes in 58 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 02 10:16:00.829153 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcd66673-5c0b-4687-b3a9-c102043a88a8 None None] REQUEST_METHOD: `POST` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:16:00.829625 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcd66673-5c0b-4687-b3a9-c102043a88a8 None None] SCRIPT_NAME: `/identity` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:16:00.830889 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcd66673-5c0b-4687-b3a9-c102043a88a8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:16:00.883435 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.auth.core [None req-bcd66673-5c0b-4687-b3a9-c102043a88a8 None None] MFA Rules not processed for user `6e6ba698dd2940ae9d3c4e416986aed9`. Rule list: `[]` (Enabled: `True`). {{(pid=80377) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 02 10:16:00.884874 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.fernet_utils [None req-bcd66673-5c0b-4687-b3a9-c102043a88a8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80377) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 10:16:01.012967 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: [pid: 80377|app: 0|req: 823/1627] 10.222.0.82 () {58 vars in 958 bytes} [Tue Aug 2 10:16:00 2022] POST /identity/v3/auth/tokens => generated 3305 bytes in 184 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 02 10:16:01.026588 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0c6fbcd-62f0-4150-8205-470b5b5e9488 admin admin] Authenticating user token {{(pid=80378) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:16:01.156446 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0c6fbcd-62f0-4150-8205-470b5b5e9488 admin admin] Validating token access rules against request {{(pid=80378) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:16:01.168063 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aac1dac0-77ea-4629-a13d-4ab6bd6f5a89 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1023de281ae74e9d8a6d6c312c1c97a7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f1b38ebb804c41c69772c998aae72c41', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80378) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:16:01.168789 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aac1dac0-77ea-4629-a13d-4ab6bd6f5a89 service cinder] REQUEST_METHOD: `GET` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:16:01.169135 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aac1dac0-77ea-4629-a13d-4ab6bd6f5a89 service cinder] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:16:01.169519 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aac1dac0-77ea-4629-a13d-4ab6bd6f5a89 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:16:01.190642 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aac1dac0-77ea-4629-a13d-4ab6bd6f5a89 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:16:01.192121 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aac1dac0-77ea-4629-a13d-4ab6bd6f5a89 service cinder] RBAC: Authorization granted {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:16:01.207190 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c542ee91-d081-4861-8557-b11072f3c347 None None] REQUEST_METHOD: `GET` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:16:01.207477 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c542ee91-d081-4861-8557-b11072f3c347 None None] SCRIPT_NAME: `/identity` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:16:01.207750 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c542ee91-d081-4861-8557-b11072f3c347 None None] PATH_INFO: `/` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:16:01.209261 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: [pid: 80377|app: 0|req: 824/1628] 10.222.0.82 () {54 vars in 962 bytes} [Tue Aug 2 10:16:01 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 02 10:16:01.217610 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5695e32b-ded5-404d-8366-80b6d8d2a5da None None] REQUEST_METHOD: `POST` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:16:01.218012 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5695e32b-ded5-404d-8366-80b6d8d2a5da None None] SCRIPT_NAME: `/identity` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:16:01.218515 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5695e32b-ded5-404d-8366-80b6d8d2a5da None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:16:01.222012 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 805/1629] 10.222.0.82 () {60 vars in 1365 bytes} [Tue Aug 2 10:16:01 2022] GET /identity/v3/auth/tokens => generated 3305 bytes in 196 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 02 10:16:01.274891 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.auth.core [None req-5695e32b-ded5-404d-8366-80b6d8d2a5da None None] MFA Rules not processed for user `88af518196674fe593ad88d30111fb7d`. Rule list: `[]` (Enabled: `True`). {{(pid=80377) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 02 10:16:01.276643 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.fernet_utils [None req-5695e32b-ded5-404d-8366-80b6d8d2a5da None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80377) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 10:16:01.462513 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: [pid: 80377|app: 0|req: 825/1630] 10.222.0.82 () {58 vars in 1059 bytes} [Tue Aug 2 10:16:01 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 248 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 02 10:16:01.474910 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20419295-c5f5-4ee7-9caf-71aebb255fe4 None None] REQUEST_METHOD: `GET` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:16:01.477317 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20419295-c5f5-4ee7-9caf-71aebb255fe4 None None] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:16:01.477317 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20419295-c5f5-4ee7-9caf-71aebb255fe4 None None] PATH_INFO: `/` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:16:01.477317 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 806/1631] 10.222.0.82 () {54 vars in 912 bytes} [Tue Aug 2 10:16:01 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 02 10:16:01.485614 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-681ce19c-31d0-498f-b010-8966012b70f8 None None] REQUEST_METHOD: `POST` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:16:01.486159 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-681ce19c-31d0-498f-b010-8966012b70f8 None None] SCRIPT_NAME: `/identity` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:16:01.486822 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-681ce19c-31d0-498f-b010-8966012b70f8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:16:01.539355 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.auth.core [None req-681ce19c-31d0-498f-b010-8966012b70f8 None None] MFA Rules not processed for user `88af518196674fe593ad88d30111fb7d`. Rule list: `[]` (Enabled: `True`). {{(pid=80377) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 02 10:16:01.540686 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.fernet_utils [None req-681ce19c-31d0-498f-b010-8966012b70f8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80377) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 10:16:01.574643 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: [pid: 80377|app: 0|req: 826/1632] 10.222.0.82 () {58 vars in 1009 bytes} [Tue Aug 2 10:16:01 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 89 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 02 10:16:01.597404 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20419295-c5f5-4ee7-9caf-71aebb255fe4 None None] Authenticating user token {{(pid=80378) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:16:01.618837 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20419295-c5f5-4ee7-9caf-71aebb255fe4 None None] Validating token access rules against request {{(pid=80378) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:16:01.635993 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-028e7a82-468d-408e-94c5-bb7d07b7715f service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1023de281ae74e9d8a6d6c312c1c97a7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f1b38ebb804c41c69772c998aae72c41', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80378) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:16:01.636635 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-028e7a82-468d-408e-94c5-bb7d07b7715f service cinder] REQUEST_METHOD: `GET` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:16:01.638008 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-028e7a82-468d-408e-94c5-bb7d07b7715f service cinder] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:16:01.639592 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-028e7a82-468d-408e-94c5-bb7d07b7715f service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:16:01.657357 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-028e7a82-468d-408e-94c5-bb7d07b7715f service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:16:01.658588 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-028e7a82-468d-408e-94c5-bb7d07b7715f service cinder] RBAC: Authorization granted {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:16:01.675609 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 807/1633] 10.222.0.82 () {60 vars in 1365 bytes} [Tue Aug 2 10:16:01 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 78 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 02 10:16:01.824267 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afddad9b-c774-4650-bdd9-6aff713fe2ad None None] REQUEST_METHOD: `GET` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:16:01.824932 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afddad9b-c774-4650-bdd9-6aff713fe2ad None None] SCRIPT_NAME: `/identity` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:16:01.825229 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afddad9b-c774-4650-bdd9-6aff713fe2ad None None] PATH_INFO: `/` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:16:01.826776 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: [pid: 80377|app: 0|req: 827/1634] 10.222.0.82 () {54 vars in 912 bytes} [Tue Aug 2 10:16:01 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 02 10:16:01.833937 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94e05627-a311-4918-9307-d876763e544e None None] REQUEST_METHOD: `POST` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:16:01.833937 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94e05627-a311-4918-9307-d876763e544e None None] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:16:01.834220 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94e05627-a311-4918-9307-d876763e544e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:16:01.872405 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.auth.core [None req-94e05627-a311-4918-9307-d876763e544e None None] MFA Rules not processed for user `88af518196674fe593ad88d30111fb7d`. Rule list: `[]` (Enabled: `True`). {{(pid=80378) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 02 10:16:01.878067 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.fernet_utils [None req-94e05627-a311-4918-9307-d876763e544e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80378) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 10:16:01.898619 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 808/1635] 10.222.0.82 () {58 vars in 1009 bytes} [Tue Aug 2 10:16:01 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 66 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 02 10:16:01.919846 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afddad9b-c774-4650-bdd9-6aff713fe2ad None None] Authenticating user token {{(pid=80377) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:16:01.938163 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afddad9b-c774-4650-bdd9-6aff713fe2ad None None] Validating token access rules against request {{(pid=80377) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:16:01.948648 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90858596-517a-460c-93fa-da432c32539f service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1023de281ae74e9d8a6d6c312c1c97a7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f1b38ebb804c41c69772c998aae72c41', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80377) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:16:01.949751 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90858596-517a-460c-93fa-da432c32539f service cinder] REQUEST_METHOD: `GET` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:16:01.951848 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90858596-517a-460c-93fa-da432c32539f service cinder] SCRIPT_NAME: `/identity` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:16:01.952134 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90858596-517a-460c-93fa-da432c32539f service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:16:01.979804 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG dbcounter [-] [80377] Writing DB stats keystone:SELECT=100 {{(pid=80377) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 10:16:01.986822 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90858596-517a-460c-93fa-da432c32539f service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:16:01.989660 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90858596-517a-460c-93fa-da432c32539f service cinder] RBAC: Authorization granted {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:16:02.019271 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: [pid: 80377|app: 0|req: 828/1636] 10.222.0.82 () {60 vars in 1365 bytes} [Tue Aug 2 10:16:01 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 102 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 02 10:16:02.225089 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94e05627-a311-4918-9307-d876763e544e None None] Authenticating user token {{(pid=80378) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:16:02.320205 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90858596-517a-460c-93fa-da432c32539f service cinder] Authenticating user token {{(pid=80377) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:16:02.433193 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94e05627-a311-4918-9307-d876763e544e None None] Validating token access rules against request {{(pid=80378) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:16:02.446886 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88971016-b97c-4f0e-852c-b51b5efa1399 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '891005fa5ff74041b75747d16cc66798', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7047912c5acd4adeba5d87c3bf895e39', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80378) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:16:02.448658 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88971016-b97c-4f0e-852c-b51b5efa1399 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:16:02.448995 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88971016-b97c-4f0e-852c-b51b5efa1399 admin admin] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:16:02.449561 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88971016-b97c-4f0e-852c-b51b5efa1399 admin admin] PATH_INFO: `/v3/users/b4624993d8e94d3ba204d1f8b71f216e` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:16:02.454330 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88971016-b97c-4f0e-852c-b51b5efa1399 admin admin] RBAC: Authorizing `identity:delete_user(user_id=b4624993d8e94d3ba204d1f8b71f216e)` {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:16:02.456701 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88971016-b97c-4f0e-852c-b51b5efa1399 admin admin] RBAC: Authorization granted {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:16:02.482324 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG dbcounter [-] [80378] Writing DB stats keystone:SELECT=94,keystone:INSERT=5,keystone:DELETE=1 {{(pid=80378) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 10:16:02.519723 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90858596-517a-460c-93fa-da432c32539f service cinder] Validating token access rules against request {{(pid=80377) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:16:02.529541 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec49917e-9f26-44df-8423-c070a5b30cee admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '891005fa5ff74041b75747d16cc66798', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7047912c5acd4adeba5d87c3bf895e39', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80377) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:16:02.530463 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec49917e-9f26-44df-8423-c070a5b30cee admin admin] REQUEST_METHOD: `DELETE` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:16:02.530901 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec49917e-9f26-44df-8423-c070a5b30cee admin admin] SCRIPT_NAME: `/identity` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:16:02.531308 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec49917e-9f26-44df-8423-c070a5b30cee admin admin] PATH_INFO: `/v3/users/c547f60c1d234f0eb2f2159c81368bba` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:16:02.534537 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec49917e-9f26-44df-8423-c070a5b30cee admin admin] RBAC: Authorizing `identity:delete_user(user_id=c547f60c1d234f0eb2f2159c81368bba)` {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:16:02.539672 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: WARNING py.warnings [None req-ec49917e-9f26-44df-8423-c070a5b30cee admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:delete_user": "(role:admin and system_scope:all) or (role:admin and token.domain.id:%(target.user.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Aug 02 10:16:02.539672 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: warnings.warn(msg) Aug 02 10:16:02.539672 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: Aug 02 10:16:02.541207 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec49917e-9f26-44df-8423-c070a5b30cee admin admin] RBAC: Authorization granted {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:16:02.559022 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.notifications [None req-88971016-b97c-4f0e-852c-b51b5efa1399 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'b4624993d8e94d3ba204d1f8b71f216e'} {{(pid=80378) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 10:16:02.577011 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.notifications [None req-88971016-b97c-4f0e-852c-b51b5efa1399 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'b4624993d8e94d3ba204d1f8b71f216e'} {{(pid=80378) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 10:16:02.618950 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.notifications [None req-88971016-b97c-4f0e-852c-b51b5efa1399 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'b4624993d8e94d3ba204d1f8b71f216e'} {{(pid=80378) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 10:16:02.620238 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.notifications [None req-88971016-b97c-4f0e-852c-b51b5efa1399 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'b4624993d8e94d3ba204d1f8b71f216e'} {{(pid=80378) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 10:16:02.670111 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.notifications [None req-88971016-b97c-4f0e-852c-b51b5efa1399 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'b4624993d8e94d3ba204d1f8b71f216e'} {{(pid=80378) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 10:16:02.688616 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 809/1637] 10.222.0.82 () {58 vars in 1224 bytes} [Tue Aug 2 10:16:02 2022] DELETE /identity/v3/users/b4624993d8e94d3ba204d1f8b71f216e => generated 0 bytes in 464 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 02 10:16:02.693336 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4d2ff2a-c373-4276-a55d-49769226ffd8 None None] REQUEST_METHOD: `POST` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:16:02.696877 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4d2ff2a-c373-4276-a55d-49769226ffd8 None None] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:16:02.698685 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4d2ff2a-c373-4276-a55d-49769226ffd8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:16:02.724537 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.notifications [None req-ec49917e-9f26-44df-8423-c070a5b30cee admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'c547f60c1d234f0eb2f2159c81368bba'} {{(pid=80377) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 10:16:02.751645 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.notifications [None req-ec49917e-9f26-44df-8423-c070a5b30cee admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'c547f60c1d234f0eb2f2159c81368bba'} {{(pid=80377) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 10:16:02.774340 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.notifications [None req-ec49917e-9f26-44df-8423-c070a5b30cee admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'c547f60c1d234f0eb2f2159c81368bba'} {{(pid=80377) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 10:16:02.775374 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.notifications [None req-ec49917e-9f26-44df-8423-c070a5b30cee admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'c547f60c1d234f0eb2f2159c81368bba'} {{(pid=80377) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 10:16:02.776678 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.auth.core [None req-e4d2ff2a-c373-4276-a55d-49769226ffd8 None None] MFA Rules not processed for user `8debb5e1e38a40159eb1713169f93cc0`. Rule list: `[]` (Enabled: `True`). {{(pid=80378) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 02 10:16:02.780861 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.fernet_utils [None req-e4d2ff2a-c373-4276-a55d-49769226ffd8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80378) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 10:16:02.824671 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.notifications [None req-ec49917e-9f26-44df-8423-c070a5b30cee admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'c547f60c1d234f0eb2f2159c81368bba'} {{(pid=80377) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 10:16:02.854278 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: [pid: 80377|app: 0|req: 829/1638] 10.222.0.82 () {58 vars in 1224 bytes} [Tue Aug 2 10:16:02 2022] DELETE /identity/v3/users/c547f60c1d234f0eb2f2159c81368bba => generated 0 bytes in 523 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 02 10:16:02.855162 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad36662c-9ec6-462d-8d62-474fc2118432 None None] REQUEST_METHOD: `POST` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:16:02.855527 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad36662c-9ec6-462d-8d62-474fc2118432 None None] SCRIPT_NAME: `/identity` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:16:02.855866 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad36662c-9ec6-462d-8d62-474fc2118432 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:16:02.891242 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.auth.core [None req-ad36662c-9ec6-462d-8d62-474fc2118432 None None] MFA Rules not processed for user `891005fa5ff74041b75747d16cc66798`. Rule list: `[]` (Enabled: `True`). {{(pid=80377) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 02 10:16:02.892497 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.fernet_utils [None req-ad36662c-9ec6-462d-8d62-474fc2118432 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80377) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 10:16:02.942773 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 810/1639] 10.222.0.82 () {58 vars in 958 bytes} [Tue Aug 2 10:16:02 2022] POST /identity/v3/auth/tokens => generated 3280 bytes in 252 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 02 10:16:02.946795 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9ccee97-5b50-4c71-a7ee-f1c02a0474dc None None] REQUEST_METHOD: `POST` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:16:02.947093 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9ccee97-5b50-4c71-a7ee-f1c02a0474dc None None] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:16:02.948700 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9ccee97-5b50-4c71-a7ee-f1c02a0474dc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:16:02.991682 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.auth.core [None req-e9ccee97-5b50-4c71-a7ee-f1c02a0474dc None None] MFA Rules not processed for user `891005fa5ff74041b75747d16cc66798`. Rule list: `[]` (Enabled: `True`). {{(pid=80378) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 02 10:16:02.994379 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.fernet_utils [None req-e9ccee97-5b50-4c71-a7ee-f1c02a0474dc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80378) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 10:16:03.050154 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: [pid: 80377|app: 0|req: 830/1640] 10.222.0.82 () {58 vars in 958 bytes} [Tue Aug 2 10:16:02 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 196 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 02 10:16:03.051675 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad36662c-9ec6-462d-8d62-474fc2118432 None None] Authenticating user token {{(pid=80377) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:16:03.054755 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.fernet_utils [None req-ad36662c-9ec6-462d-8d62-474fc2118432 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80377) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 10:16:03.080320 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad36662c-9ec6-462d-8d62-474fc2118432 None None] Validating token access rules against request {{(pid=80377) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:16:03.092624 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4dcade8b-b2e8-4295-80f9-017ff03f3a60 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '891005fa5ff74041b75747d16cc66798', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7047912c5acd4adeba5d87c3bf895e39', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80377) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:16:03.093294 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4dcade8b-b2e8-4295-80f9-017ff03f3a60 admin admin] REQUEST_METHOD: `POST` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:16:03.093574 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4dcade8b-b2e8-4295-80f9-017ff03f3a60 admin admin] SCRIPT_NAME: `/identity` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:16:03.093870 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4dcade8b-b2e8-4295-80f9-017ff03f3a60 admin admin] PATH_INFO: `/v3/projects` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:16:03.095335 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4dcade8b-b2e8-4295-80f9-017ff03f3a60 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:16:03.101361 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4dcade8b-b2e8-4295-80f9-017ff03f3a60 admin admin] RBAC: Authorization granted {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:16:03.152098 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 811/1641] 10.222.0.82 () {58 vars in 958 bytes} [Tue Aug 2 10:16:02 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 207 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 02 10:16:03.154343 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9ccee97-5b50-4c71-a7ee-f1c02a0474dc None None] Authenticating user token {{(pid=80378) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:16:03.155175 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG dbcounter [-] [80377] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80377) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 10:16:03.156128 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.fernet_utils [None req-e9ccee97-5b50-4c71-a7ee-f1c02a0474dc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80378) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 10:16:03.195263 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: [pid: 80377|app: 0|req: 831/1642] 10.222.0.82 () {60 vars in 1153 bytes} [Tue Aug 2 10:16:03 2022] POST /identity/v3/projects => generated 405 bytes in 144 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 02 10:16:03.196291 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4dcade8b-b2e8-4295-80f9-017ff03f3a60 admin admin] Authenticating user token {{(pid=80377) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:16:03.227305 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG dbcounter [-] [80378] Writing DB stats keystone:SELECT=90,keystone:DELETE=9,keystone:INSERT=1 {{(pid=80378) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 10:16:03.329218 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9ccee97-5b50-4c71-a7ee-f1c02a0474dc None None] Validating token access rules against request {{(pid=80378) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:16:03.368659 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4dcade8b-b2e8-4295-80f9-017ff03f3a60 admin admin] Validating token access rules against request {{(pid=80377) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:16:03.379180 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-104abe91-4503-4ad9-89b9-7c943dbbfe01 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '891005fa5ff74041b75747d16cc66798', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7047912c5acd4adeba5d87c3bf895e39', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80377) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:16:03.380052 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-104abe91-4503-4ad9-89b9-7c943dbbfe01 admin admin] REQUEST_METHOD: `POST` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:16:03.380498 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-104abe91-4503-4ad9-89b9-7c943dbbfe01 admin admin] SCRIPT_NAME: `/identity` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:16:03.381213 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-104abe91-4503-4ad9-89b9-7c943dbbfe01 admin admin] PATH_INFO: `/v3/users` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:16:03.388499 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-104abe91-4503-4ad9-89b9-7c943dbbfe01 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:16:03.390799 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-104abe91-4503-4ad9-89b9-7c943dbbfe01 admin admin] RBAC: Authorization granted {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:16:03.442344 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: [pid: 80377|app: 0|req: 832/1643] 10.222.0.82 () {60 vars in 1144 bytes} [Tue Aug 2 10:16:03 2022] POST /identity/v3/users => generated 309 bytes in 244 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 02 10:16:03.444936 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-104abe91-4503-4ad9-89b9-7c943dbbfe01 admin admin] Authenticating user token {{(pid=80377) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:16:03.468329 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b36149b-51f2-47ae-87fd-3311a20ef55f service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '33b87170f0634b67bce60750847b5cbe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f1b38ebb804c41c69772c998aae72c41', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80378) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:16:03.470475 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b36149b-51f2-47ae-87fd-3311a20ef55f service neutron] REQUEST_METHOD: `GET` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:16:03.470944 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b36149b-51f2-47ae-87fd-3311a20ef55f service neutron] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:16:03.471460 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b36149b-51f2-47ae-87fd-3311a20ef55f service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:16:03.498755 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b36149b-51f2-47ae-87fd-3311a20ef55f service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:16:03.503082 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b36149b-51f2-47ae-87fd-3311a20ef55f service neutron] RBAC: Authorization granted {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:16:03.527407 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 812/1644] 10.222.0.82 () {60 vars in 1365 bytes} [Tue Aug 2 10:16:03 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 375 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 02 10:16:03.533064 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b36149b-51f2-47ae-87fd-3311a20ef55f service neutron] Authenticating user token {{(pid=80378) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:16:03.557156 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b36149b-51f2-47ae-87fd-3311a20ef55f service neutron] Validating token access rules against request {{(pid=80378) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:16:03.566386 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b04e2ee-6c7e-48ee-b73b-fafb6beecfad admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '891005fa5ff74041b75747d16cc66798', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7047912c5acd4adeba5d87c3bf895e39', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80378) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:16:03.568144 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b04e2ee-6c7e-48ee-b73b-fafb6beecfad admin admin] REQUEST_METHOD: `GET` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:16:03.570180 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b04e2ee-6c7e-48ee-b73b-fafb6beecfad admin admin] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:16:03.570540 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b04e2ee-6c7e-48ee-b73b-fafb6beecfad admin admin] PATH_INFO: `/v3/roles` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:16:03.571758 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b04e2ee-6c7e-48ee-b73b-fafb6beecfad admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:16:03.577455 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b04e2ee-6c7e-48ee-b73b-fafb6beecfad admin admin] RBAC: Authorization granted {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:16:03.636392 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-104abe91-4503-4ad9-89b9-7c943dbbfe01 admin admin] Validating token access rules against request {{(pid=80377) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:16:03.644876 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 813/1645] 10.222.0.82 () {58 vars in 1122 bytes} [Tue Aug 2 10:16:03 2022] GET /identity/v3/roles => generated 1420 bytes in 113 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 02 10:16:03.649495 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b04e2ee-6c7e-48ee-b73b-fafb6beecfad admin admin] Authenticating user token {{(pid=80378) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:16:03.664120 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e97aae2-a068-4e9f-8fee-cde045f3aa1b service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '33b87170f0634b67bce60750847b5cbe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f1b38ebb804c41c69772c998aae72c41', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80377) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:16:03.665272 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e97aae2-a068-4e9f-8fee-cde045f3aa1b service neutron] REQUEST_METHOD: `GET` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:16:03.665570 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e97aae2-a068-4e9f-8fee-cde045f3aa1b service neutron] SCRIPT_NAME: `/identity` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:16:03.665990 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e97aae2-a068-4e9f-8fee-cde045f3aa1b service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:16:03.691940 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b04e2ee-6c7e-48ee-b73b-fafb6beecfad admin admin] Validating token access rules against request {{(pid=80378) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:16:03.695503 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e97aae2-a068-4e9f-8fee-cde045f3aa1b service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:16:03.700338 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e97aae2-a068-4e9f-8fee-cde045f3aa1b service neutron] RBAC: Authorization granted {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:16:03.725403 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-757e2fd9-387a-4e49-b255-9bbdc9c80e4b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '891005fa5ff74041b75747d16cc66798', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7047912c5acd4adeba5d87c3bf895e39', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80378) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:16:03.726375 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-757e2fd9-387a-4e49-b255-9bbdc9c80e4b admin admin] REQUEST_METHOD: `PUT` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:16:03.729943 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-757e2fd9-387a-4e49-b255-9bbdc9c80e4b admin admin] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:16:03.732365 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-757e2fd9-387a-4e49-b255-9bbdc9c80e4b admin admin] PATH_INFO: `/v3/projects/df2bc25ad42743939bd44e1f31049208/users/4d76a9dc026047038d14dadedb3df1fb/roles/237966442f05411191105bf336bc14f3` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:16:03.735779 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: [pid: 80377|app: 0|req: 833/1646] 10.222.0.82 () {60 vars in 1365 bytes} [Tue Aug 2 10:16:03 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 291 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 02 10:16:03.803417 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-757e2fd9-387a-4e49-b255-9bbdc9c80e4b admin admin] RBAC: Authorizing `identity:create_grant(project_id=df2bc25ad42743939bd44e1f31049208, user_id=4d76a9dc026047038d14dadedb3df1fb, role_id=237966442f05411191105bf336bc14f3)` {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:16:03.807020 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-757e2fd9-387a-4e49-b255-9bbdc9c80e4b admin admin] RBAC: Authorization granted {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:16:03.836006 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 814/1647] 10.222.0.82 () {60 vars in 1483 bytes} [Tue Aug 2 10:16:03 2022] PUT /identity/v3/projects/df2bc25ad42743939bd44e1f31049208/users/4d76a9dc026047038d14dadedb3df1fb/roles/237966442f05411191105bf336bc14f3 => generated 0 bytes in 190 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 02 10:16:03.840338 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e97aae2-a068-4e9f-8fee-cde045f3aa1b service neutron] Authenticating user token {{(pid=80377) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:16:03.860257 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e97aae2-a068-4e9f-8fee-cde045f3aa1b service neutron] Validating token access rules against request {{(pid=80377) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:16:04.117591 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-152e03c8-d10c-4c05-be27-31f72ff4c7fb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '891005fa5ff74041b75747d16cc66798', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7047912c5acd4adeba5d87c3bf895e39', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80377) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:16:04.118274 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-152e03c8-d10c-4c05-be27-31f72ff4c7fb admin admin] REQUEST_METHOD: `GET` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:16:04.118560 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-152e03c8-d10c-4c05-be27-31f72ff4c7fb admin admin] SCRIPT_NAME: `/identity` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:16:04.118836 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-152e03c8-d10c-4c05-be27-31f72ff4c7fb admin admin] PATH_INFO: `/v3/roles` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:16:04.119555 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-152e03c8-d10c-4c05-be27-31f72ff4c7fb admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:16:04.121481 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-152e03c8-d10c-4c05-be27-31f72ff4c7fb admin admin] RBAC: Authorization granted {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:16:04.170893 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-757e2fd9-387a-4e49-b255-9bbdc9c80e4b admin admin] Authenticating user token {{(pid=80378) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:16:04.172471 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: [pid: 80377|app: 0|req: 834/1648] 10.222.0.82 () {58 vars in 1122 bytes} [Tue Aug 2 10:16:03 2022] GET /identity/v3/roles => generated 1420 bytes in 333 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 02 10:16:04.190342 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-757e2fd9-387a-4e49-b255-9bbdc9c80e4b admin admin] Validating token access rules against request {{(pid=80378) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:16:04.205752 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04f62745-e498-43a2-b1af-34f19c5afcf9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '891005fa5ff74041b75747d16cc66798', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7047912c5acd4adeba5d87c3bf895e39', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80378) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:16:04.206735 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04f62745-e498-43a2-b1af-34f19c5afcf9 admin admin] REQUEST_METHOD: `PUT` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:16:04.207158 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04f62745-e498-43a2-b1af-34f19c5afcf9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:16:04.207574 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04f62745-e498-43a2-b1af-34f19c5afcf9 admin admin] PATH_INFO: `/v3/projects/df2bc25ad42743939bd44e1f31049208/users/4d76a9dc026047038d14dadedb3df1fb/roles/237966442f05411191105bf336bc14f3` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:16:04.209901 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04f62745-e498-43a2-b1af-34f19c5afcf9 admin admin] RBAC: Authorizing `identity:create_grant(project_id=df2bc25ad42743939bd44e1f31049208, user_id=4d76a9dc026047038d14dadedb3df1fb, role_id=237966442f05411191105bf336bc14f3)` {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:16:04.213055 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04f62745-e498-43a2-b1af-34f19c5afcf9 admin admin] RBAC: Authorization granted {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:16:04.238395 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 815/1649] 10.222.0.82 () {60 vars in 1483 bytes} [Tue Aug 2 10:16:04 2022] PUT /identity/v3/projects/df2bc25ad42743939bd44e1f31049208/users/4d76a9dc026047038d14dadedb3df1fb/roles/237966442f05411191105bf336bc14f3 => generated 0 bytes in 68 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 02 10:16:05.499557 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-152e03c8-d10c-4c05-be27-31f72ff4c7fb admin admin] Authenticating user token {{(pid=80377) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:16:05.502322 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.fernet_utils [None req-152e03c8-d10c-4c05-be27-31f72ff4c7fb admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80377) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 10:16:05.523263 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-152e03c8-d10c-4c05-be27-31f72ff4c7fb admin admin] Validating token access rules against request {{(pid=80377) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:16:05.574210 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG dbcounter [-] [80377] Writing DB stats keystone:INSERT=4,keystone:SELECT=96 {{(pid=80377) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 10:16:05.685074 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62aae62f-987e-4dfb-b547-4eec66640ee8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '891005fa5ff74041b75747d16cc66798', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7047912c5acd4adeba5d87c3bf895e39', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80377) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:16:05.685748 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62aae62f-987e-4dfb-b547-4eec66640ee8 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:16:05.686064 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62aae62f-987e-4dfb-b547-4eec66640ee8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:16:05.686335 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62aae62f-987e-4dfb-b547-4eec66640ee8 admin admin] PATH_INFO: `/v3/projects/18813e93dc5249d3a0fa816213f95375` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:16:05.707109 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62aae62f-987e-4dfb-b547-4eec66640ee8 admin admin] RBAC: Authorizing `identity:delete_project(project_id=18813e93dc5249d3a0fa816213f95375)` {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:16:05.710750 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62aae62f-987e-4dfb-b547-4eec66640ee8 admin admin] RBAC: Authorization granted {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:16:05.824326 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.notifications [None req-62aae62f-987e-4dfb-b547-4eec66640ee8 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '18813e93dc5249d3a0fa816213f95375', 'request_id': 'req-62aae62f-987e-4dfb-b547-4eec66640ee8'} {{(pid=80377) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 10:16:05.844080 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.notifications [None req-62aae62f-987e-4dfb-b547-4eec66640ee8 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '18813e93dc5249d3a0fa816213f95375', 'request_id': 'req-62aae62f-987e-4dfb-b547-4eec66640ee8'} {{(pid=80377) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 10:16:05.866495 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.notifications [None req-62aae62f-987e-4dfb-b547-4eec66640ee8 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '18813e93dc5249d3a0fa816213f95375', 'request_id': 'req-62aae62f-987e-4dfb-b547-4eec66640ee8'} {{(pid=80377) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 10:16:05.890169 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.notifications [None req-62aae62f-987e-4dfb-b547-4eec66640ee8 admin admin] The token cache is being invalidate because project 18813e93dc5249d3a0fa816213f95375 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80377) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 02 10:16:05.890907 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.notifications [None req-62aae62f-987e-4dfb-b547-4eec66640ee8 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80377) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 10:16:05.891805 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.notifications [None req-62aae62f-987e-4dfb-b547-4eec66640ee8 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80377) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 10:16:05.903464 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: [pid: 80377|app: 0|req: 835/1650] 10.222.0.82 () {58 vars in 1233 bytes} [Tue Aug 2 10:16:05 2022] DELETE /identity/v3/projects/18813e93dc5249d3a0fa816213f95375 => generated 0 bytes in 399 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 02 10:16:05.904165 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04f62745-e498-43a2-b1af-34f19c5afcf9 admin admin] Authenticating user token {{(pid=80378) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:16:05.908651 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.fernet_utils [None req-04f62745-e498-43a2-b1af-34f19c5afcf9 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80378) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 10:16:05.922952 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04f62745-e498-43a2-b1af-34f19c5afcf9 admin admin] Validating token access rules against request {{(pid=80378) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:16:06.055921 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-539a6bb7-22b5-41d2-8677-1973cd49f9b7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '891005fa5ff74041b75747d16cc66798', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7047912c5acd4adeba5d87c3bf895e39', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80378) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:16:06.056748 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-539a6bb7-22b5-41d2-8677-1973cd49f9b7 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:16:06.057320 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-539a6bb7-22b5-41d2-8677-1973cd49f9b7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:16:06.057890 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-539a6bb7-22b5-41d2-8677-1973cd49f9b7 admin admin] PATH_INFO: `/v3/users/6e6ba698dd2940ae9d3c4e416986aed9` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:16:06.060902 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-539a6bb7-22b5-41d2-8677-1973cd49f9b7 admin admin] RBAC: Authorizing `identity:delete_user(user_id=6e6ba698dd2940ae9d3c4e416986aed9)` {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:16:06.064427 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-539a6bb7-22b5-41d2-8677-1973cd49f9b7 admin admin] RBAC: Authorization granted {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:16:06.151507 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG dbcounter [-] [80378] Writing DB stats keystone:SELECT=93,keystone:INSERT=2,keystone:DELETE=5 {{(pid=80378) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 10:16:06.160587 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62aae62f-987e-4dfb-b547-4eec66640ee8 admin admin] Authenticating user token {{(pid=80377) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:16:06.162315 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.fernet_utils [None req-62aae62f-987e-4dfb-b547-4eec66640ee8 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80377) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 10:16:06.201283 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62aae62f-987e-4dfb-b547-4eec66640ee8 admin admin] Validating token access rules against request {{(pid=80377) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:16:06.207915 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.notifications [None req-539a6bb7-22b5-41d2-8677-1973cd49f9b7 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '6e6ba698dd2940ae9d3c4e416986aed9'} {{(pid=80378) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 10:16:06.231374 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d50fb2f5-d591-43bc-88a7-d720a4d4b076 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '891005fa5ff74041b75747d16cc66798', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7047912c5acd4adeba5d87c3bf895e39', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80377) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:16:06.232360 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d50fb2f5-d591-43bc-88a7-d720a4d4b076 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:16:06.232849 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d50fb2f5-d591-43bc-88a7-d720a4d4b076 admin admin] SCRIPT_NAME: `/identity` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:16:06.233303 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d50fb2f5-d591-43bc-88a7-d720a4d4b076 admin admin] PATH_INFO: `/v3/projects/9bab6ba334404db0b6709c91b165e0bc` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:16:06.234772 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d50fb2f5-d591-43bc-88a7-d720a4d4b076 admin admin] RBAC: Authorizing `identity:delete_project(project_id=9bab6ba334404db0b6709c91b165e0bc)` {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:16:06.236927 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d50fb2f5-d591-43bc-88a7-d720a4d4b076 admin admin] RBAC: Authorization granted {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:16:06.242770 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.notifications [None req-539a6bb7-22b5-41d2-8677-1973cd49f9b7 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '6e6ba698dd2940ae9d3c4e416986aed9'} {{(pid=80378) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 10:16:06.258743 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.notifications [None req-539a6bb7-22b5-41d2-8677-1973cd49f9b7 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '6e6ba698dd2940ae9d3c4e416986aed9'} {{(pid=80378) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 10:16:06.263248 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.notifications [None req-539a6bb7-22b5-41d2-8677-1973cd49f9b7 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '6e6ba698dd2940ae9d3c4e416986aed9'} {{(pid=80378) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 10:16:06.337724 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.notifications [None req-539a6bb7-22b5-41d2-8677-1973cd49f9b7 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '6e6ba698dd2940ae9d3c4e416986aed9'} {{(pid=80378) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 10:16:06.348328 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 816/1651] 10.222.0.82 () {58 vars in 1224 bytes} [Tue Aug 2 10:16:05 2022] DELETE /identity/v3/users/6e6ba698dd2940ae9d3c4e416986aed9 => generated 0 bytes in 445 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 02 10:16:06.359949 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.notifications [None req-d50fb2f5-d591-43bc-88a7-d720a4d4b076 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '9bab6ba334404db0b6709c91b165e0bc', 'request_id': 'req-d50fb2f5-d591-43bc-88a7-d720a4d4b076'} {{(pid=80377) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 10:16:06.377952 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.notifications [None req-d50fb2f5-d591-43bc-88a7-d720a4d4b076 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '9bab6ba334404db0b6709c91b165e0bc', 'request_id': 'req-d50fb2f5-d591-43bc-88a7-d720a4d4b076'} {{(pid=80377) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 10:16:06.387795 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.notifications [None req-d50fb2f5-d591-43bc-88a7-d720a4d4b076 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '9bab6ba334404db0b6709c91b165e0bc', 'request_id': 'req-d50fb2f5-d591-43bc-88a7-d720a4d4b076'} {{(pid=80377) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 10:16:06.446978 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.notifications [None req-d50fb2f5-d591-43bc-88a7-d720a4d4b076 admin admin] The token cache is being invalidate because project 9bab6ba334404db0b6709c91b165e0bc was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80377) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 02 10:16:06.447172 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.notifications [None req-d50fb2f5-d591-43bc-88a7-d720a4d4b076 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80377) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 10:16:06.448466 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.notifications [None req-d50fb2f5-d591-43bc-88a7-d720a4d4b076 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80377) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 10:16:06.450795 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: [pid: 80377|app: 0|req: 836/1652] 10.222.0.82 () {58 vars in 1233 bytes} [Tue Aug 2 10:16:06 2022] DELETE /identity/v3/projects/9bab6ba334404db0b6709c91b165e0bc => generated 0 bytes in 291 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 02 10:16:06.455834 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-342a9093-b9bb-4679-83c8-8afa1214d4a4 None None] REQUEST_METHOD: `POST` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:16:06.456251 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-342a9093-b9bb-4679-83c8-8afa1214d4a4 None None] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:16:06.456639 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-342a9093-b9bb-4679-83c8-8afa1214d4a4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:16:06.501048 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.auth.core [None req-342a9093-b9bb-4679-83c8-8afa1214d4a4 None None] MFA Rules not processed for user `891005fa5ff74041b75747d16cc66798`. Rule list: `[]` (Enabled: `True`). {{(pid=80378) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 02 10:16:06.502736 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.fernet_utils [None req-342a9093-b9bb-4679-83c8-8afa1214d4a4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80378) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 10:16:06.615506 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 817/1653] 10.222.0.82 () {58 vars in 958 bytes} [Tue Aug 2 10:16:06 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 161 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 02 10:16:06.623977 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f91bd4a5-880f-4961-9f0a-ac527489d491 None None] REQUEST_METHOD: `POST` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:16:06.624477 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f91bd4a5-880f-4961-9f0a-ac527489d491 None None] SCRIPT_NAME: `/identity` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:16:06.624552 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f91bd4a5-880f-4961-9f0a-ac527489d491 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:16:06.661738 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.auth.core [None req-f91bd4a5-880f-4961-9f0a-ac527489d491 None None] MFA Rules not processed for user `891005fa5ff74041b75747d16cc66798`. Rule list: `[]` (Enabled: `True`). {{(pid=80377) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 02 10:16:06.663283 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.fernet_utils [None req-f91bd4a5-880f-4961-9f0a-ac527489d491 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80377) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 10:16:06.716996 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: [pid: 80377|app: 0|req: 837/1654] 10.222.0.82 () {58 vars in 958 bytes} [Tue Aug 2 10:16:06 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 02 10:16:06.721187 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-342a9093-b9bb-4679-83c8-8afa1214d4a4 None None] Authenticating user token {{(pid=80378) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:16:06.754943 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-342a9093-b9bb-4679-83c8-8afa1214d4a4 None None] Validating token access rules against request {{(pid=80378) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:16:06.771317 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25b19b17-04ef-45ad-8bdd-ad58cd9d4ce3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '891005fa5ff74041b75747d16cc66798', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7047912c5acd4adeba5d87c3bf895e39', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80378) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:16:06.772486 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25b19b17-04ef-45ad-8bdd-ad58cd9d4ce3 admin admin] REQUEST_METHOD: `GET` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:16:06.773002 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25b19b17-04ef-45ad-8bdd-ad58cd9d4ce3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:16:06.774077 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25b19b17-04ef-45ad-8bdd-ad58cd9d4ce3 admin admin] PATH_INFO: `/v3/domains` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:16:06.778552 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25b19b17-04ef-45ad-8bdd-ad58cd9d4ce3 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80378) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 02 10:16:06.778931 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25b19b17-04ef-45ad-8bdd-ad58cd9d4ce3 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:16:06.780783 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25b19b17-04ef-45ad-8bdd-ad58cd9d4ce3 admin admin] RBAC: Authorization granted {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:16:06.805261 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 818/1655] 10.222.0.82 () {58 vars in 1153 bytes} [Tue Aug 2 10:16:06 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 82 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 02 10:16:06.807330 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f91bd4a5-880f-4961-9f0a-ac527489d491 None None] Authenticating user token {{(pid=80377) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:16:06.824104 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f91bd4a5-880f-4961-9f0a-ac527489d491 None None] Validating token access rules against request {{(pid=80377) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:16:06.834508 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42d74f73-44f5-477e-9d84-a54dd72c2cce admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '891005fa5ff74041b75747d16cc66798', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7047912c5acd4adeba5d87c3bf895e39', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80377) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:16:06.838197 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42d74f73-44f5-477e-9d84-a54dd72c2cce admin admin] REQUEST_METHOD: `POST` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:16:06.838948 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42d74f73-44f5-477e-9d84-a54dd72c2cce admin admin] SCRIPT_NAME: `/identity` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:16:06.839437 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42d74f73-44f5-477e-9d84-a54dd72c2cce admin admin] PATH_INFO: `/v3/projects` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:16:06.840370 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42d74f73-44f5-477e-9d84-a54dd72c2cce admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:16:06.852769 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42d74f73-44f5-477e-9d84-a54dd72c2cce admin admin] RBAC: Authorization granted {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:16:06.932158 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: [pid: 80377|app: 0|req: 838/1656] 10.222.0.82 () {60 vars in 1153 bytes} [Tue Aug 2 10:16:06 2022] POST /identity/v3/projects => generated 405 bytes in 125 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 02 10:16:06.936296 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25b19b17-04ef-45ad-8bdd-ad58cd9d4ce3 admin admin] Authenticating user token {{(pid=80378) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:16:06.953200 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25b19b17-04ef-45ad-8bdd-ad58cd9d4ce3 admin admin] Validating token access rules against request {{(pid=80378) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:16:07.116940 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e900bfd9-3d9f-44f1-a36f-f62111c6e3e1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '891005fa5ff74041b75747d16cc66798', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7047912c5acd4adeba5d87c3bf895e39', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80378) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:16:07.117452 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e900bfd9-3d9f-44f1-a36f-f62111c6e3e1 admin admin] REQUEST_METHOD: `POST` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:16:07.117741 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e900bfd9-3d9f-44f1-a36f-f62111c6e3e1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:16:07.118173 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e900bfd9-3d9f-44f1-a36f-f62111c6e3e1 admin admin] PATH_INFO: `/v3/users` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:16:07.118868 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e900bfd9-3d9f-44f1-a36f-f62111c6e3e1 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:16:07.121050 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e900bfd9-3d9f-44f1-a36f-f62111c6e3e1 admin admin] RBAC: Authorization granted {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:16:07.173385 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 819/1657] 10.222.0.82 () {60 vars in 1144 bytes} [Tue Aug 2 10:16:06 2022] POST /identity/v3/users => generated 303 bytes in 238 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 02 10:16:07.177880 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42d74f73-44f5-477e-9d84-a54dd72c2cce admin admin] Authenticating user token {{(pid=80377) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:16:07.189092 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42d74f73-44f5-477e-9d84-a54dd72c2cce admin admin] Validating token access rules against request {{(pid=80377) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:16:07.195277 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fdacc7a1-5c72-489c-9ce6-302a5601537e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '891005fa5ff74041b75747d16cc66798', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7047912c5acd4adeba5d87c3bf895e39', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80377) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:16:07.196248 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdacc7a1-5c72-489c-9ce6-302a5601537e admin admin] REQUEST_METHOD: `GET` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:16:07.196694 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdacc7a1-5c72-489c-9ce6-302a5601537e admin admin] SCRIPT_NAME: `/identity` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:16:07.197113 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdacc7a1-5c72-489c-9ce6-302a5601537e admin admin] PATH_INFO: `/v3/roles` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:16:07.198473 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fdacc7a1-5c72-489c-9ce6-302a5601537e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:16:07.204994 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fdacc7a1-5c72-489c-9ce6-302a5601537e admin admin] RBAC: Authorization granted {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:16:07.207908 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG dbcounter [-] [80377] Writing DB stats keystone:SELECT=83,keystone:DELETE=14,keystone:INSERT=3 {{(pid=80377) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 10:16:07.225313 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: [pid: 80377|app: 0|req: 839/1658] 10.222.0.82 () {58 vars in 1122 bytes} [Tue Aug 2 10:16:07 2022] GET /identity/v3/roles => generated 1420 bytes in 48 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 02 10:16:07.229442 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e900bfd9-3d9f-44f1-a36f-f62111c6e3e1 admin admin] Authenticating user token {{(pid=80378) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:16:07.268100 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e900bfd9-3d9f-44f1-a36f-f62111c6e3e1 admin admin] Validating token access rules against request {{(pid=80378) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:16:07.297832 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fdc8a54b-8dba-4a79-966e-88bf3b5c7d41 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '891005fa5ff74041b75747d16cc66798', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7047912c5acd4adeba5d87c3bf895e39', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80378) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:16:07.298065 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdc8a54b-8dba-4a79-966e-88bf3b5c7d41 admin admin] REQUEST_METHOD: `GET` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:16:07.298447 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdc8a54b-8dba-4a79-966e-88bf3b5c7d41 admin admin] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:16:07.300049 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdc8a54b-8dba-4a79-966e-88bf3b5c7d41 admin admin] PATH_INFO: `/v3/roles` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:16:07.301223 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fdc8a54b-8dba-4a79-966e-88bf3b5c7d41 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:16:07.304979 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fdc8a54b-8dba-4a79-966e-88bf3b5c7d41 admin admin] RBAC: Authorization granted {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:16:07.360965 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 820/1659] 10.222.0.82 () {58 vars in 1122 bytes} [Tue Aug 2 10:16:07 2022] GET /identity/v3/roles => generated 1420 bytes in 132 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 02 10:16:07.361934 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fdacc7a1-5c72-489c-9ce6-302a5601537e admin admin] Authenticating user token {{(pid=80377) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:16:07.382289 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fdacc7a1-5c72-489c-9ce6-302a5601537e admin admin] Validating token access rules against request {{(pid=80377) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:16:07.400909 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba1004ce-ee93-4624-8da7-0b9965f1f455 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '891005fa5ff74041b75747d16cc66798', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7047912c5acd4adeba5d87c3bf895e39', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80377) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:16:07.402497 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba1004ce-ee93-4624-8da7-0b9965f1f455 admin admin] REQUEST_METHOD: `PUT` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:16:07.403561 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba1004ce-ee93-4624-8da7-0b9965f1f455 admin admin] SCRIPT_NAME: `/identity` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:16:07.403938 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba1004ce-ee93-4624-8da7-0b9965f1f455 admin admin] PATH_INFO: `/v3/projects/4391660601b4468c9998da049edd3440/users/e5a0408eab41497c820665ed58f98e61/roles/e1c04b19fc4543b8b54f58bbcffbc598` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:16:07.442894 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba1004ce-ee93-4624-8da7-0b9965f1f455 admin admin] RBAC: Authorizing `identity:create_grant(project_id=4391660601b4468c9998da049edd3440, user_id=e5a0408eab41497c820665ed58f98e61, role_id=e1c04b19fc4543b8b54f58bbcffbc598)` {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:16:07.452936 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba1004ce-ee93-4624-8da7-0b9965f1f455 admin admin] RBAC: Authorization granted {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:16:07.501828 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: [pid: 80377|app: 0|req: 840/1660] 10.222.0.82 () {60 vars in 1483 bytes} [Tue Aug 2 10:16:07 2022] PUT /identity/v3/projects/4391660601b4468c9998da049edd3440/users/e5a0408eab41497c820665ed58f98e61/roles/e1c04b19fc4543b8b54f58bbcffbc598 => generated 0 bytes in 141 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 02 10:16:07.517363 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01c5db03-db9c-44dd-b6ed-14a0eabf3dd3 None None] REQUEST_METHOD: `POST` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:16:07.519954 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01c5db03-db9c-44dd-b6ed-14a0eabf3dd3 None None] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:16:07.520403 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01c5db03-db9c-44dd-b6ed-14a0eabf3dd3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:16:07.612084 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.auth.core [None req-01c5db03-db9c-44dd-b6ed-14a0eabf3dd3 None None] MFA Rules not processed for user `e5a0408eab41497c820665ed58f98e61`. Rule list: `[]` (Enabled: `True`). {{(pid=80378) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 02 10:16:07.613352 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.fernet_utils [None req-01c5db03-db9c-44dd-b6ed-14a0eabf3dd3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80378) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 10:16:07.654877 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG dbcounter [-] [80378] Writing DB stats keystone:SELECT=91,keystone:DELETE=5,keystone:INSERT=4 {{(pid=80378) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 10:16:07.778832 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c82d9d8-b924-4185-92c0-c0917c84b037 None None] REQUEST_METHOD: `POST` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:16:07.779658 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c82d9d8-b924-4185-92c0-c0917c84b037 None None] SCRIPT_NAME: `/identity` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:16:07.780100 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c82d9d8-b924-4185-92c0-c0917c84b037 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:16:07.874722 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 821/1661] 10.222.0.82 () {58 vars in 958 bytes} [Tue Aug 2 10:16:07 2022] POST /identity/v3/auth/tokens => generated 3282 bytes in 359 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 02 10:16:07.881065 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01c5db03-db9c-44dd-b6ed-14a0eabf3dd3 None None] Authenticating user token {{(pid=80378) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:16:07.888935 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.auth.core [None req-4c82d9d8-b924-4185-92c0-c0917c84b037 None None] MFA Rules not processed for user `f690a9da98b548e9aee69426281ae428`. Rule list: `[]` (Enabled: `True`). {{(pid=80377) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 02 10:16:07.895589 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.fernet_utils [None req-4c82d9d8-b924-4185-92c0-c0917c84b037 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80377) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 10:16:08.108338 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01c5db03-db9c-44dd-b6ed-14a0eabf3dd3 None None] Validating token access rules against request {{(pid=80378) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:16:08.126567 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e954d4b-e55f-4cf6-b73e-d863dfc8cb7b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '891005fa5ff74041b75747d16cc66798', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7047912c5acd4adeba5d87c3bf895e39', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80378) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:16:08.127272 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e954d4b-e55f-4cf6-b73e-d863dfc8cb7b admin admin] REQUEST_METHOD: `POST` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:16:08.127622 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e954d4b-e55f-4cf6-b73e-d863dfc8cb7b admin admin] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:16:08.127909 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e954d4b-e55f-4cf6-b73e-d863dfc8cb7b admin admin] PATH_INFO: `/v3/projects` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:16:08.128596 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e954d4b-e55f-4cf6-b73e-d863dfc8cb7b admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:16:08.132453 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e954d4b-e55f-4cf6-b73e-d863dfc8cb7b admin admin] RBAC: Authorization granted {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:16:08.201978 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 822/1662] 10.222.0.82 () {60 vars in 1153 bytes} [Tue Aug 2 10:16:07 2022] POST /identity/v3/projects => generated 405 bytes in 321 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 02 10:16:08.203891 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: [pid: 80377|app: 0|req: 841/1663] 10.222.0.82 () {58 vars in 958 bytes} [Tue Aug 2 10:16:07 2022] POST /identity/v3/auth/tokens => generated 3254 bytes in 427 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 02 10:16:08.219410 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c82d9d8-b924-4185-92c0-c0917c84b037 None None] Authenticating user token {{(pid=80377) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:16:08.228490 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-526b0c0d-f19d-4c9b-b50f-fdb9eb20b94d None None] REQUEST_METHOD: `POST` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:16:08.228702 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-526b0c0d-f19d-4c9b-b50f-fdb9eb20b94d None None] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:16:08.228958 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-526b0c0d-f19d-4c9b-b50f-fdb9eb20b94d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:16:08.238841 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c82d9d8-b924-4185-92c0-c0917c84b037 None None] Validating token access rules against request {{(pid=80377) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:16:08.324363 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.auth.core [None req-526b0c0d-f19d-4c9b-b50f-fdb9eb20b94d None None] MFA Rules not processed for user `4d76a9dc026047038d14dadedb3df1fb`. Rule list: `[]` (Enabled: `True`). {{(pid=80378) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 02 10:16:08.330005 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.fernet_utils [None req-526b0c0d-f19d-4c9b-b50f-fdb9eb20b94d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80378) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 10:16:08.393107 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e4661b3-202c-48d8-b424-19fd3b583bbc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '891005fa5ff74041b75747d16cc66798', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7047912c5acd4adeba5d87c3bf895e39', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80377) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:16:08.395136 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e4661b3-202c-48d8-b424-19fd3b583bbc admin admin] REQUEST_METHOD: `POST` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:16:08.397318 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e4661b3-202c-48d8-b424-19fd3b583bbc admin admin] SCRIPT_NAME: `/identity` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:16:08.397629 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e4661b3-202c-48d8-b424-19fd3b583bbc admin admin] PATH_INFO: `/v3/users` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:16:08.398379 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e4661b3-202c-48d8-b424-19fd3b583bbc admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:16:08.409476 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e4661b3-202c-48d8-b424-19fd3b583bbc admin admin] RBAC: Authorization granted {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:16:08.464162 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: [pid: 80377|app: 0|req: 842/1664] 10.222.0.82 () {60 vars in 1144 bytes} [Tue Aug 2 10:16:08 2022] POST /identity/v3/users => generated 309 bytes in 245 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 02 10:16:08.473192 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d829c3fc-4e34-43e2-afcd-df885d6687ee None None] REQUEST_METHOD: `POST` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:16:08.473752 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d829c3fc-4e34-43e2-afcd-df885d6687ee None None] SCRIPT_NAME: `/identity` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:16:08.474058 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d829c3fc-4e34-43e2-afcd-df885d6687ee None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:16:08.509887 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 823/1665] 10.222.0.82 () {58 vars in 958 bytes} [Tue Aug 2 10:16:08 2022] POST /identity/v3/auth/tokens => generated 3349 bytes in 279 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 02 10:16:08.509887 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-526b0c0d-f19d-4c9b-b50f-fdb9eb20b94d None None] Authenticating user token {{(pid=80378) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:16:08.524313 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-526b0c0d-f19d-4c9b-b50f-fdb9eb20b94d None None] Validating token access rules against request {{(pid=80378) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:16:08.534530 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c8764b7-f4a6-4fe6-8d92-13ddf052fd89 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '891005fa5ff74041b75747d16cc66798', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7047912c5acd4adeba5d87c3bf895e39', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80378) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:16:08.535836 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c8764b7-f4a6-4fe6-8d92-13ddf052fd89 admin admin] REQUEST_METHOD: `GET` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:16:08.535836 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c8764b7-f4a6-4fe6-8d92-13ddf052fd89 admin admin] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:16:08.537732 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c8764b7-f4a6-4fe6-8d92-13ddf052fd89 admin admin] PATH_INFO: `/v3/roles` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:16:08.538796 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c8764b7-f4a6-4fe6-8d92-13ddf052fd89 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:16:08.540678 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c8764b7-f4a6-4fe6-8d92-13ddf052fd89 admin admin] RBAC: Authorization granted {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:16:08.546103 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.auth.core [None req-d829c3fc-4e34-43e2-afcd-df885d6687ee None None] MFA Rules not processed for user `f690a9da98b548e9aee69426281ae428`. Rule list: `[]` (Enabled: `True`). {{(pid=80377) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 02 10:16:08.547591 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.fernet_utils [None req-d829c3fc-4e34-43e2-afcd-df885d6687ee None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80377) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 10:16:08.568170 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 824/1666] 10.222.0.82 () {58 vars in 1122 bytes} [Tue Aug 2 10:16:08 2022] GET /identity/v3/roles => generated 1420 bytes in 63 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 02 10:16:08.573553 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c8764b7-f4a6-4fe6-8d92-13ddf052fd89 admin admin] Authenticating user token {{(pid=80378) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:16:08.591562 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG dbcounter [-] [80378] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80378) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 10:16:08.603464 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c8764b7-f4a6-4fe6-8d92-13ddf052fd89 admin admin] Validating token access rules against request {{(pid=80378) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:16:08.614217 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-163fa497-dca6-497d-8cd9-b1832d48dc4b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '891005fa5ff74041b75747d16cc66798', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7047912c5acd4adeba5d87c3bf895e39', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80378) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:16:08.615646 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-163fa497-dca6-497d-8cd9-b1832d48dc4b admin admin] REQUEST_METHOD: `PUT` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:16:08.617587 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-163fa497-dca6-497d-8cd9-b1832d48dc4b admin admin] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:16:08.618088 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-163fa497-dca6-497d-8cd9-b1832d48dc4b admin admin] PATH_INFO: `/v3/projects/a19cdac12af2479fb179d691e096b8a9/users/e291ef3b0c3347128a3ad0e99914a44b/roles/237966442f05411191105bf336bc14f3` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:16:08.637049 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG dbcounter [-] [80377] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=80377) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 10:16:08.667480 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-163fa497-dca6-497d-8cd9-b1832d48dc4b admin admin] RBAC: Authorizing `identity:create_grant(project_id=a19cdac12af2479fb179d691e096b8a9, user_id=e291ef3b0c3347128a3ad0e99914a44b, role_id=237966442f05411191105bf336bc14f3)` {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:16:08.682092 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: [pid: 80377|app: 0|req: 843/1667] 10.222.0.82 () {58 vars in 958 bytes} [Tue Aug 2 10:16:08 2022] POST /identity/v3/auth/tokens => generated 3254 bytes in 215 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 02 10:16:08.687242 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d829c3fc-4e34-43e2-afcd-df885d6687ee None None] Authenticating user token {{(pid=80377) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:16:08.687376 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-163fa497-dca6-497d-8cd9-b1832d48dc4b admin admin] RBAC: Authorization granted {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:16:08.697546 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.fernet_utils [None req-d829c3fc-4e34-43e2-afcd-df885d6687ee None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80377) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 10:16:08.759990 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 825/1668] 10.222.0.82 () {60 vars in 1483 bytes} [Tue Aug 2 10:16:08 2022] PUT /identity/v3/projects/a19cdac12af2479fb179d691e096b8a9/users/e291ef3b0c3347128a3ad0e99914a44b/roles/237966442f05411191105bf336bc14f3 => generated 0 bytes in 187 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 02 10:16:08.765463 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-163fa497-dca6-497d-8cd9-b1832d48dc4b admin admin] Authenticating user token {{(pid=80378) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:16:08.971120 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d829c3fc-4e34-43e2-afcd-df885d6687ee None None] Validating token access rules against request {{(pid=80377) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:16:09.063647 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-163fa497-dca6-497d-8cd9-b1832d48dc4b admin admin] Validating token access rules against request {{(pid=80378) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:16:09.092028 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-464a570e-3c27-4b57-87c9-64212e17b84b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '891005fa5ff74041b75747d16cc66798', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7047912c5acd4adeba5d87c3bf895e39', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80378) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:16:09.092858 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-464a570e-3c27-4b57-87c9-64212e17b84b admin admin] REQUEST_METHOD: `GET` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:16:09.093468 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-464a570e-3c27-4b57-87c9-64212e17b84b admin admin] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:16:09.093918 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-464a570e-3c27-4b57-87c9-64212e17b84b admin admin] PATH_INFO: `/v3/roles` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:16:09.095086 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-464a570e-3c27-4b57-87c9-64212e17b84b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:16:09.100252 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-464a570e-3c27-4b57-87c9-64212e17b84b admin admin] RBAC: Authorization granted {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:16:09.145027 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 826/1669] 10.222.0.82 () {58 vars in 1122 bytes} [Tue Aug 2 10:16:08 2022] GET /identity/v3/roles => generated 1420 bytes in 380 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 02 10:16:09.148979 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-464a570e-3c27-4b57-87c9-64212e17b84b admin admin] Authenticating user token {{(pid=80378) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:16:09.179044 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-464a570e-3c27-4b57-87c9-64212e17b84b admin admin] Validating token access rules against request {{(pid=80378) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:16:09.199217 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63189eef-0ab8-446a-b8ad-efac998d8b62 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '891005fa5ff74041b75747d16cc66798', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7047912c5acd4adeba5d87c3bf895e39', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80378) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:16:09.200589 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63189eef-0ab8-446a-b8ad-efac998d8b62 admin admin] REQUEST_METHOD: `PUT` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:16:09.200775 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63189eef-0ab8-446a-b8ad-efac998d8b62 admin admin] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:16:09.201032 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63189eef-0ab8-446a-b8ad-efac998d8b62 admin admin] PATH_INFO: `/v3/projects/a19cdac12af2479fb179d691e096b8a9/users/e291ef3b0c3347128a3ad0e99914a44b/roles/237966442f05411191105bf336bc14f3` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:16:09.207924 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63189eef-0ab8-446a-b8ad-efac998d8b62 admin admin] RBAC: Authorizing `identity:create_grant(project_id=a19cdac12af2479fb179d691e096b8a9, user_id=e291ef3b0c3347128a3ad0e99914a44b, role_id=237966442f05411191105bf336bc14f3)` {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:16:09.210695 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63189eef-0ab8-446a-b8ad-efac998d8b62 admin admin] RBAC: Authorization granted {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:16:09.240326 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7b05325-0cb8-4c2a-8890-a04467238268 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88af518196674fe593ad88d30111fb7d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f1b38ebb804c41c69772c998aae72c41', 'project_domain_id': 'default', 'roles': ['admin', 'ResellerAdmin', 'reader', 'service', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80377) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:16:09.241132 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7b05325-0cb8-4c2a-8890-a04467238268 service nova] REQUEST_METHOD: `GET` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:16:09.241411 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7b05325-0cb8-4c2a-8890-a04467238268 service nova] SCRIPT_NAME: `/identity` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:16:09.241682 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7b05325-0cb8-4c2a-8890-a04467238268 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:16:09.269388 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 827/1670] 10.222.0.82 () {60 vars in 1483 bytes} [Tue Aug 2 10:16:09 2022] PUT /identity/v3/projects/a19cdac12af2479fb179d691e096b8a9/users/e291ef3b0c3347128a3ad0e99914a44b/roles/237966442f05411191105bf336bc14f3 => generated 0 bytes in 121 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 02 10:16:09.271920 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63189eef-0ab8-446a-b8ad-efac998d8b62 admin admin] Authenticating user token {{(pid=80378) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:16:09.277759 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.fernet_utils [None req-63189eef-0ab8-446a-b8ad-efac998d8b62 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80378) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 10:16:09.425130 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7b05325-0cb8-4c2a-8890-a04467238268 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:16:09.432314 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7b05325-0cb8-4c2a-8890-a04467238268 service nova] RBAC: Authorization granted {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:16:09.457198 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: [pid: 80377|app: 0|req: 844/1671] 10.222.0.82 () {60 vars in 1365 bytes} [Tue Aug 2 10:16:08 2022] GET /identity/v3/auth/tokens => generated 3254 bytes in 770 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 02 10:16:09.458920 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-205d0cd6-7fbc-4ede-aeb4-3e198c370e86 None None] REQUEST_METHOD: `POST` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:16:09.462949 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-205d0cd6-7fbc-4ede-aeb4-3e198c370e86 None None] SCRIPT_NAME: `/identity` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:16:09.463503 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-205d0cd6-7fbc-4ede-aeb4-3e198c370e86 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:16:09.489473 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63189eef-0ab8-446a-b8ad-efac998d8b62 admin admin] Validating token access rules against request {{(pid=80378) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:16:09.511617 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c272afd-71fd-4df5-b415-4c7af1382c76 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '891005fa5ff74041b75747d16cc66798', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7047912c5acd4adeba5d87c3bf895e39', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80378) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:16:09.512310 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c272afd-71fd-4df5-b415-4c7af1382c76 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:16:09.512630 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c272afd-71fd-4df5-b415-4c7af1382c76 admin admin] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:16:09.512911 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c272afd-71fd-4df5-b415-4c7af1382c76 admin admin] PATH_INFO: `/v3/projects/a5aafa32be16445a98f61c66fd099973` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:16:09.518582 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c272afd-71fd-4df5-b415-4c7af1382c76 admin admin] RBAC: Authorizing `identity:delete_project(project_id=a5aafa32be16445a98f61c66fd099973)` {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:16:09.519634 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: WARNING py.warnings [None req-4c272afd-71fd-4df5-b415-4c7af1382c76 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:delete_project": "(role:admin and system_scope:all) or (role:admin and domain_id:%(target.project.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Aug 02 10:16:09.519634 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: warnings.warn(msg) Aug 02 10:16:09.519634 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: Aug 02 10:16:09.521614 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c272afd-71fd-4df5-b415-4c7af1382c76 admin admin] RBAC: Authorization granted {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:16:09.542494 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.auth.core [None req-205d0cd6-7fbc-4ede-aeb4-3e198c370e86 None None] MFA Rules not processed for user `e291ef3b0c3347128a3ad0e99914a44b`. Rule list: `[]` (Enabled: `True`). {{(pid=80377) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 02 10:16:09.546582 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.fernet_utils [None req-205d0cd6-7fbc-4ede-aeb4-3e198c370e86 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80377) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 10:16:09.605521 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG dbcounter [-] [80377] Writing DB stats keystone:SELECT=100 {{(pid=80377) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 10:16:09.695478 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.notifications [None req-4c272afd-71fd-4df5-b415-4c7af1382c76 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'a5aafa32be16445a98f61c66fd099973', 'request_id': 'req-4c272afd-71fd-4df5-b415-4c7af1382c76'} {{(pid=80378) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 10:16:09.753905 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: [pid: 80377|app: 0|req: 845/1672] 10.222.0.82 () {58 vars in 958 bytes} [Tue Aug 2 10:16:09 2022] POST /identity/v3/auth/tokens => generated 3349 bytes in 296 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 02 10:16:09.766993 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.notifications [None req-4c272afd-71fd-4df5-b415-4c7af1382c76 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'a5aafa32be16445a98f61c66fd099973', 'request_id': 'req-4c272afd-71fd-4df5-b415-4c7af1382c76'} {{(pid=80378) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 10:16:09.767491 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-205d0cd6-7fbc-4ede-aeb4-3e198c370e86 None None] Authenticating user token {{(pid=80377) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:16:09.769280 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.fernet_utils [None req-205d0cd6-7fbc-4ede-aeb4-3e198c370e86 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80377) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 10:16:09.793411 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.notifications [None req-4c272afd-71fd-4df5-b415-4c7af1382c76 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'a5aafa32be16445a98f61c66fd099973', 'request_id': 'req-4c272afd-71fd-4df5-b415-4c7af1382c76'} {{(pid=80378) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 10:16:09.798813 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG dbcounter [-] [80378] Writing DB stats keystone:SELECT=91,keystone:INSERT=3,keystone:DELETE=6 {{(pid=80378) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 10:16:09.832179 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.notifications [None req-4c272afd-71fd-4df5-b415-4c7af1382c76 admin admin] The token cache is being invalidate because project a5aafa32be16445a98f61c66fd099973 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80378) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 02 10:16:09.832553 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.notifications [None req-4c272afd-71fd-4df5-b415-4c7af1382c76 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80378) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 10:16:09.833371 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.notifications [None req-4c272afd-71fd-4df5-b415-4c7af1382c76 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80378) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 10:16:09.835434 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 828/1673] 10.222.0.82 () {58 vars in 1233 bytes} [Tue Aug 2 10:16:09 2022] DELETE /identity/v3/projects/a5aafa32be16445a98f61c66fd099973 => generated 0 bytes in 566 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 02 10:16:09.842908 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-141a548b-c980-4c6c-99c3-bc693cfbbb5a None None] REQUEST_METHOD: `POST` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:16:09.843135 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-141a548b-c980-4c6c-99c3-bc693cfbbb5a None None] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:16:09.843726 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-141a548b-c980-4c6c-99c3-bc693cfbbb5a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:16:09.890054 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-205d0cd6-7fbc-4ede-aeb4-3e198c370e86 None None] Validating token access rules against request {{(pid=80377) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:16:09.933538 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.auth.core [None req-141a548b-c980-4c6c-99c3-bc693cfbbb5a None None] MFA Rules not processed for user `891005fa5ff74041b75747d16cc66798`. Rule list: `[]` (Enabled: `True`). {{(pid=80378) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 02 10:16:09.935498 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.fernet_utils [None req-141a548b-c980-4c6c-99c3-bc693cfbbb5a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80378) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 10:16:09.950507 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6427a14-f6ea-4abb-a18b-e7bee605ef0b service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1023de281ae74e9d8a6d6c312c1c97a7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f1b38ebb804c41c69772c998aae72c41', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80377) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:16:09.951197 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6427a14-f6ea-4abb-a18b-e7bee605ef0b service cinder] REQUEST_METHOD: `GET` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:16:09.951482 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6427a14-f6ea-4abb-a18b-e7bee605ef0b service cinder] SCRIPT_NAME: `/identity` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:16:09.951755 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6427a14-f6ea-4abb-a18b-e7bee605ef0b service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:16:09.954010 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.fernet_utils [None req-c6427a14-f6ea-4abb-a18b-e7bee605ef0b service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80377) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 10:16:10.049811 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6427a14-f6ea-4abb-a18b-e7bee605ef0b service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:16:10.051551 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6427a14-f6ea-4abb-a18b-e7bee605ef0b service cinder] RBAC: Authorization granted {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:16:10.067317 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: [pid: 80377|app: 0|req: 846/1674] 10.222.0.82 () {60 vars in 1365 bytes} [Tue Aug 2 10:16:09 2022] GET /identity/v3/auth/tokens => generated 3349 bytes in 300 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 02 10:16:10.134739 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 829/1675] 10.222.0.82 () {58 vars in 958 bytes} [Tue Aug 2 10:16:09 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 287 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 02 10:16:10.139164 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0991dd1-b470-4eae-9056-87f66dba2b2c None None] REQUEST_METHOD: `POST` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:16:10.139512 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0991dd1-b470-4eae-9056-87f66dba2b2c None None] SCRIPT_NAME: `/identity` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:16:10.156240 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0991dd1-b470-4eae-9056-87f66dba2b2c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:16:10.210602 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.auth.core [None req-a0991dd1-b470-4eae-9056-87f66dba2b2c None None] MFA Rules not processed for user `891005fa5ff74041b75747d16cc66798`. Rule list: `[]` (Enabled: `True`). {{(pid=80377) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 02 10:16:10.213451 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.fernet_utils [None req-a0991dd1-b470-4eae-9056-87f66dba2b2c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80377) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 10:16:10.265999 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: [pid: 80377|app: 0|req: 847/1676] 10.222.0.82 () {58 vars in 958 bytes} [Tue Aug 2 10:16:10 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 131 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 02 10:16:10.270391 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-141a548b-c980-4c6c-99c3-bc693cfbbb5a None None] Authenticating user token {{(pid=80378) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:16:10.283827 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-141a548b-c980-4c6c-99c3-bc693cfbbb5a None None] Validating token access rules against request {{(pid=80378) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:16:10.298182 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3826046b-d827-4a94-8aca-d9607cf693e4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '891005fa5ff74041b75747d16cc66798', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7047912c5acd4adeba5d87c3bf895e39', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80378) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:16:10.298839 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3826046b-d827-4a94-8aca-d9607cf693e4 admin admin] REQUEST_METHOD: `GET` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:16:10.302568 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3826046b-d827-4a94-8aca-d9607cf693e4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:16:10.302856 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3826046b-d827-4a94-8aca-d9607cf693e4 admin admin] PATH_INFO: `/v3/domains` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:16:10.304642 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3826046b-d827-4a94-8aca-d9607cf693e4 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80378) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 02 10:16:10.304961 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3826046b-d827-4a94-8aca-d9607cf693e4 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:16:10.311030 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3826046b-d827-4a94-8aca-d9607cf693e4 admin admin] RBAC: Authorization granted {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:16:10.355233 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 830/1677] 10.222.0.82 () {58 vars in 1153 bytes} [Tue Aug 2 10:16:10 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 85 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 02 10:16:10.358212 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0991dd1-b470-4eae-9056-87f66dba2b2c None None] Authenticating user token {{(pid=80377) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:16:10.374086 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0991dd1-b470-4eae-9056-87f66dba2b2c None None] Validating token access rules against request {{(pid=80377) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:16:10.396800 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d95e85c-29d6-4a94-9d17-521ade968b7b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '891005fa5ff74041b75747d16cc66798', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7047912c5acd4adeba5d87c3bf895e39', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80377) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:16:10.398084 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d95e85c-29d6-4a94-9d17-521ade968b7b admin admin] REQUEST_METHOD: `POST` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:16:10.398084 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d95e85c-29d6-4a94-9d17-521ade968b7b admin admin] SCRIPT_NAME: `/identity` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:16:10.398084 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d95e85c-29d6-4a94-9d17-521ade968b7b admin admin] PATH_INFO: `/v3/projects` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:16:10.398710 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d95e85c-29d6-4a94-9d17-521ade968b7b admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:16:10.400922 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d95e85c-29d6-4a94-9d17-521ade968b7b admin admin] RBAC: Authorization granted {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:16:10.416579 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3826046b-d827-4a94-8aca-d9607cf693e4 admin admin] Authenticating user token {{(pid=80378) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:16:10.418847 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.fernet_utils [None req-3826046b-d827-4a94-8aca-d9607cf693e4 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80378) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 10:16:10.448674 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3826046b-d827-4a94-8aca-d9607cf693e4 admin admin] Validating token access rules against request {{(pid=80378) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:16:10.464586 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-882177e8-8e45-4b61-ab25-d36f7984c001 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '891005fa5ff74041b75747d16cc66798', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7047912c5acd4adeba5d87c3bf895e39', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80378) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:16:10.465942 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-882177e8-8e45-4b61-ab25-d36f7984c001 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:16:10.466379 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-882177e8-8e45-4b61-ab25-d36f7984c001 admin admin] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:16:10.466791 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-882177e8-8e45-4b61-ab25-d36f7984c001 admin admin] PATH_INFO: `/v3/users/e5a0408eab41497c820665ed58f98e61` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:16:10.470105 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-882177e8-8e45-4b61-ab25-d36f7984c001 admin admin] RBAC: Authorizing `identity:delete_user(user_id=e5a0408eab41497c820665ed58f98e61)` {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:16:10.472276 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-882177e8-8e45-4b61-ab25-d36f7984c001 admin admin] RBAC: Authorization granted {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:16:10.476777 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG dbcounter [-] [80377] Writing DB stats keystone:SELECT=100 {{(pid=80377) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 10:16:10.514562 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: [pid: 80377|app: 0|req: 848/1678] 10.222.0.82 () {60 vars in 1153 bytes} [Tue Aug 2 10:16:10 2022] POST /identity/v3/projects => generated 409 bytes in 157 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 02 10:16:10.518367 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d95e85c-29d6-4a94-9d17-521ade968b7b admin admin] Authenticating user token {{(pid=80377) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:16:10.617767 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.notifications [None req-882177e8-8e45-4b61-ab25-d36f7984c001 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'e5a0408eab41497c820665ed58f98e61'} {{(pid=80378) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 10:16:10.636931 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.notifications [None req-882177e8-8e45-4b61-ab25-d36f7984c001 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'e5a0408eab41497c820665ed58f98e61'} {{(pid=80378) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 10:16:10.651851 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.notifications [None req-882177e8-8e45-4b61-ab25-d36f7984c001 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'e5a0408eab41497c820665ed58f98e61'} {{(pid=80378) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 10:16:10.654134 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.notifications [None req-882177e8-8e45-4b61-ab25-d36f7984c001 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'e5a0408eab41497c820665ed58f98e61'} {{(pid=80378) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 10:16:10.658136 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d95e85c-29d6-4a94-9d17-521ade968b7b admin admin] Validating token access rules against request {{(pid=80377) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:16:10.670660 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c055839e-0bb7-4de8-94d4-d7d7c5dc191e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '891005fa5ff74041b75747d16cc66798', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7047912c5acd4adeba5d87c3bf895e39', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80377) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:16:10.671440 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c055839e-0bb7-4de8-94d4-d7d7c5dc191e admin admin] REQUEST_METHOD: `POST` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:16:10.671831 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c055839e-0bb7-4de8-94d4-d7d7c5dc191e admin admin] SCRIPT_NAME: `/identity` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:16:10.672236 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c055839e-0bb7-4de8-94d4-d7d7c5dc191e admin admin] PATH_INFO: `/v3/users` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:16:10.673027 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c055839e-0bb7-4de8-94d4-d7d7c5dc191e admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:16:10.679269 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c055839e-0bb7-4de8-94d4-d7d7c5dc191e admin admin] RBAC: Authorization granted {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:16:10.690714 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.notifications [None req-882177e8-8e45-4b61-ab25-d36f7984c001 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'e5a0408eab41497c820665ed58f98e61'} {{(pid=80378) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 10:16:10.714707 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 831/1679] 10.222.0.82 () {58 vars in 1224 bytes} [Tue Aug 2 10:16:10 2022] DELETE /identity/v3/users/e5a0408eab41497c820665ed58f98e61 => generated 0 bytes in 294 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 02 10:16:10.719276 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-191597ae-2b4b-4ba3-b0ca-aa69652a28d0 None None] REQUEST_METHOD: `POST` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:16:10.719569 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-191597ae-2b4b-4ba3-b0ca-aa69652a28d0 None None] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:16:10.719846 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-191597ae-2b4b-4ba3-b0ca-aa69652a28d0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:16:10.743760 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: [pid: 80377|app: 0|req: 849/1680] 10.222.0.82 () {60 vars in 1144 bytes} [Tue Aug 2 10:16:10 2022] POST /identity/v3/users => generated 305 bytes in 223 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 02 10:16:10.745353 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c055839e-0bb7-4de8-94d4-d7d7c5dc191e admin admin] Authenticating user token {{(pid=80377) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:16:10.751920 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.fernet_utils [None req-c055839e-0bb7-4de8-94d4-d7d7c5dc191e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80377) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 10:16:10.769537 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.auth.core [None req-191597ae-2b4b-4ba3-b0ca-aa69652a28d0 None None] MFA Rules not processed for user `891005fa5ff74041b75747d16cc66798`. Rule list: `[]` (Enabled: `True`). {{(pid=80378) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 02 10:16:10.772952 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.fernet_utils [None req-191597ae-2b4b-4ba3-b0ca-aa69652a28d0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80378) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 10:16:10.785358 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c055839e-0bb7-4de8-94d4-d7d7c5dc191e admin admin] Validating token access rules against request {{(pid=80377) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:16:10.920277 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG dbcounter [-] [80378] Writing DB stats keystone:DELETE=11,keystone:SELECT=88,keystone:INSERT=1 {{(pid=80378) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 10:16:10.944502 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ee835ea-f292-40d5-9bba-763ee4542ab1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '891005fa5ff74041b75747d16cc66798', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7047912c5acd4adeba5d87c3bf895e39', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80377) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:16:10.946579 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ee835ea-f292-40d5-9bba-763ee4542ab1 admin admin] REQUEST_METHOD: `GET` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:16:10.946827 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ee835ea-f292-40d5-9bba-763ee4542ab1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:16:10.947195 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ee835ea-f292-40d5-9bba-763ee4542ab1 admin admin] PATH_INFO: `/v3/roles` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:16:10.947917 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ee835ea-f292-40d5-9bba-763ee4542ab1 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:16:10.957627 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 832/1681] 10.222.0.82 () {58 vars in 958 bytes} [Tue Aug 2 10:16:10 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 242 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 02 10:16:10.959637 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ee835ea-f292-40d5-9bba-763ee4542ab1 admin admin] RBAC: Authorization granted {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:16:10.980803 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: [pid: 80377|app: 0|req: 850/1682] 10.222.0.82 () {58 vars in 1122 bytes} [Tue Aug 2 10:16:10 2022] GET /identity/v3/roles => generated 1420 bytes in 236 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 02 10:16:10.985463 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-191597ae-2b4b-4ba3-b0ca-aa69652a28d0 None None] Authenticating user token {{(pid=80378) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:16:10.999437 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-191597ae-2b4b-4ba3-b0ca-aa69652a28d0 None None] Validating token access rules against request {{(pid=80378) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:16:11.016760 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ee835ea-f292-40d5-9bba-763ee4542ab1 admin admin] Authenticating user token {{(pid=80377) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:16:11.018640 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f58240e-5a49-4211-aa67-f46c1ad2e01c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '891005fa5ff74041b75747d16cc66798', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7047912c5acd4adeba5d87c3bf895e39', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80378) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:16:11.019486 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f58240e-5a49-4211-aa67-f46c1ad2e01c admin admin] REQUEST_METHOD: `GET` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:16:11.019910 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f58240e-5a49-4211-aa67-f46c1ad2e01c admin admin] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:16:11.020442 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f58240e-5a49-4211-aa67-f46c1ad2e01c admin admin] PATH_INFO: `/v3/roles` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:16:11.021327 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f58240e-5a49-4211-aa67-f46c1ad2e01c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:16:11.023168 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f58240e-5a49-4211-aa67-f46c1ad2e01c admin admin] RBAC: Authorization granted {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:16:11.023973 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.fernet_utils [None req-0ee835ea-f292-40d5-9bba-763ee4542ab1 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80377) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 10:16:11.049590 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 833/1683] 10.222.0.82 () {58 vars in 1122 bytes} [Tue Aug 2 10:16:10 2022] GET /identity/v3/roles => generated 1420 bytes in 65 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 02 10:16:11.056227 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f58240e-5a49-4211-aa67-f46c1ad2e01c admin admin] Authenticating user token {{(pid=80378) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:16:11.079009 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f58240e-5a49-4211-aa67-f46c1ad2e01c admin admin] Validating token access rules against request {{(pid=80378) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:16:11.094878 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67337b1f-90d8-4791-b36c-98a8e2f3ddd2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '891005fa5ff74041b75747d16cc66798', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7047912c5acd4adeba5d87c3bf895e39', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80378) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:16:11.095897 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67337b1f-90d8-4791-b36c-98a8e2f3ddd2 admin admin] REQUEST_METHOD: `PUT` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:16:11.096351 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67337b1f-90d8-4791-b36c-98a8e2f3ddd2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:16:11.096792 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67337b1f-90d8-4791-b36c-98a8e2f3ddd2 admin admin] PATH_INFO: `/v3/projects/4b921cb3c8e941c58ac781f641e2830c/users/cda272cef98e4ed6bcf00540dfdd2625/roles/e1c04b19fc4543b8b54f58bbcffbc598` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:16:11.129977 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67337b1f-90d8-4791-b36c-98a8e2f3ddd2 admin admin] RBAC: Authorizing `identity:create_grant(project_id=4b921cb3c8e941c58ac781f641e2830c, user_id=cda272cef98e4ed6bcf00540dfdd2625, role_id=e1c04b19fc4543b8b54f58bbcffbc598)` {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:16:11.139001 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67337b1f-90d8-4791-b36c-98a8e2f3ddd2 admin admin] RBAC: Authorization granted {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:16:11.168227 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 834/1684] 10.222.0.82 () {60 vars in 1483 bytes} [Tue Aug 2 10:16:11 2022] PUT /identity/v3/projects/4b921cb3c8e941c58ac781f641e2830c/users/cda272cef98e4ed6bcf00540dfdd2625/roles/e1c04b19fc4543b8b54f58bbcffbc598 => generated 0 bytes in 113 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 02 10:16:11.172981 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c493aed-a4bb-4546-947a-d2b42adf69bd None None] REQUEST_METHOD: `GET` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:16:11.173510 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c493aed-a4bb-4546-947a-d2b42adf69bd None None] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:16:11.173995 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c493aed-a4bb-4546-947a-d2b42adf69bd None None] PATH_INFO: `/` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:16:11.176274 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 835/1685] 10.222.0.82 () {54 vars in 912 bytes} [Tue Aug 2 10:16:11 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 02 10:16:11.179213 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad7cd4ba-2229-4ae9-a669-4cf7dfcd8772 None None] REQUEST_METHOD: `POST` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:16:11.180956 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad7cd4ba-2229-4ae9-a669-4cf7dfcd8772 None None] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:16:11.181531 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad7cd4ba-2229-4ae9-a669-4cf7dfcd8772 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:16:11.195735 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ee835ea-f292-40d5-9bba-763ee4542ab1 admin admin] Validating token access rules against request {{(pid=80377) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:16:11.267679 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.auth.core [None req-ad7cd4ba-2229-4ae9-a669-4cf7dfcd8772 None None] MFA Rules not processed for user `cda272cef98e4ed6bcf00540dfdd2625`. Rule list: `[]` (Enabled: `True`). {{(pid=80378) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 02 10:16:11.270787 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.fernet_utils [None req-ad7cd4ba-2229-4ae9-a669-4cf7dfcd8772 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80378) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 10:16:11.279571 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG dbcounter [-] [80377] Writing DB stats keystone:INSERT=4,keystone:SELECT=96 {{(pid=80377) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 10:16:11.324481 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9819083b-e809-4e46-be07-cabf3866495f service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '33b87170f0634b67bce60750847b5cbe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f1b38ebb804c41c69772c998aae72c41', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80377) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:16:11.325704 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9819083b-e809-4e46-be07-cabf3866495f service neutron] REQUEST_METHOD: `GET` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:16:11.328177 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9819083b-e809-4e46-be07-cabf3866495f service neutron] SCRIPT_NAME: `/identity` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:16:11.328475 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9819083b-e809-4e46-be07-cabf3866495f service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:16:11.475144 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 836/1686] 10.222.0.82 () {58 vars in 958 bytes} [Tue Aug 2 10:16:11 2022] POST /identity/v3/auth/tokens => generated 3286 bytes in 297 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 02 10:16:11.476379 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d477ca91-b521-4ff1-b4ef-328439c019c1 None None] REQUEST_METHOD: `POST` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:16:11.476673 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d477ca91-b521-4ff1-b4ef-328439c019c1 None None] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:16:11.476953 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d477ca91-b521-4ff1-b4ef-328439c019c1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:16:11.493976 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9819083b-e809-4e46-be07-cabf3866495f service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:16:11.495429 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9819083b-e809-4e46-be07-cabf3866495f service neutron] RBAC: Authorization granted {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:16:11.531314 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: [pid: 80377|app: 0|req: 851/1687] 10.222.0.82 () {60 vars in 1365 bytes} [Tue Aug 2 10:16:11 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 518 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 02 10:16:11.532192 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9819083b-e809-4e46-be07-cabf3866495f service neutron] Authenticating user token {{(pid=80377) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:16:11.546237 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.auth.core [None req-d477ca91-b521-4ff1-b4ef-328439c019c1 None None] MFA Rules not processed for user `33b87170f0634b67bce60750847b5cbe`. Rule list: `[]` (Enabled: `True`). {{(pid=80378) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 02 10:16:11.548904 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.fernet_utils [None req-d477ca91-b521-4ff1-b4ef-328439c019c1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80378) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 10:16:11.551932 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9819083b-e809-4e46-be07-cabf3866495f service neutron] Validating token access rules against request {{(pid=80377) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:16:11.569612 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 837/1688] 10.222.0.82 () {58 vars in 1009 bytes} [Tue Aug 2 10:16:11 2022] POST /identity/v3/auth/tokens => generated 3121 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 02 10:16:11.571310 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec113aeb-15b7-499b-9edc-9c721d8198e5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '891005fa5ff74041b75747d16cc66798', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7047912c5acd4adeba5d87c3bf895e39', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80377) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:16:11.572690 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec113aeb-15b7-499b-9edc-9c721d8198e5 admin admin] REQUEST_METHOD: `POST` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:16:11.573163 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec113aeb-15b7-499b-9edc-9c721d8198e5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:16:11.573380 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec113aeb-15b7-499b-9edc-9c721d8198e5 admin admin] PATH_INFO: `/v3/projects` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:16:11.577520 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec113aeb-15b7-499b-9edc-9c721d8198e5 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:16:11.580421 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec113aeb-15b7-499b-9edc-9c721d8198e5 admin admin] RBAC: Authorization granted {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:16:11.643171 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: [pid: 80377|app: 0|req: 852/1689] 10.222.0.82 () {60 vars in 1153 bytes} [Tue Aug 2 10:16:11 2022] POST /identity/v3/projects => generated 407 bytes in 111 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 02 10:16:11.647055 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d477ca91-b521-4ff1-b4ef-328439c019c1 None None] Authenticating user token {{(pid=80378) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:16:11.664912 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d477ca91-b521-4ff1-b4ef-328439c019c1 None None] Validating token access rules against request {{(pid=80378) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:16:11.816171 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fbeba84-cdc0-4b82-8d87-de64f603b7a8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '891005fa5ff74041b75747d16cc66798', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7047912c5acd4adeba5d87c3bf895e39', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80378) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:16:11.817004 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fbeba84-cdc0-4b82-8d87-de64f603b7a8 admin admin] REQUEST_METHOD: `POST` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:16:11.817366 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fbeba84-cdc0-4b82-8d87-de64f603b7a8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:16:11.817715 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fbeba84-cdc0-4b82-8d87-de64f603b7a8 admin admin] PATH_INFO: `/v3/users` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:16:11.818500 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4fbeba84-cdc0-4b82-8d87-de64f603b7a8 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:16:11.820532 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4fbeba84-cdc0-4b82-8d87-de64f603b7a8 admin admin] RBAC: Authorization granted {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:16:11.888342 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 838/1690] 10.222.0.82 () {60 vars in 1144 bytes} [Tue Aug 2 10:16:11 2022] POST /identity/v3/users => generated 310 bytes in 241 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 02 10:16:11.892158 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec113aeb-15b7-499b-9edc-9c721d8198e5 admin admin] Authenticating user token {{(pid=80377) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:16:11.904399 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec113aeb-15b7-499b-9edc-9c721d8198e5 admin admin] Validating token access rules against request {{(pid=80377) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:16:11.924192 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e45e959f-9925-4bac-90c5-45772cf68df1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '891005fa5ff74041b75747d16cc66798', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7047912c5acd4adeba5d87c3bf895e39', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80377) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:16:11.925070 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e45e959f-9925-4bac-90c5-45772cf68df1 admin admin] REQUEST_METHOD: `GET` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:16:11.926182 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e45e959f-9925-4bac-90c5-45772cf68df1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:16:11.926681 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e45e959f-9925-4bac-90c5-45772cf68df1 admin admin] PATH_INFO: `/v3/roles` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:16:11.927582 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e45e959f-9925-4bac-90c5-45772cf68df1 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:16:11.929409 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e45e959f-9925-4bac-90c5-45772cf68df1 admin admin] RBAC: Authorization granted {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:16:11.966407 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: [pid: 80377|app: 0|req: 853/1691] 10.222.0.82 () {58 vars in 1122 bytes} [Tue Aug 2 10:16:11 2022] GET /identity/v3/roles => generated 1420 bytes in 73 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 02 10:16:11.969029 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fbeba84-cdc0-4b82-8d87-de64f603b7a8 admin admin] Authenticating user token {{(pid=80378) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:16:11.986301 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fbeba84-cdc0-4b82-8d87-de64f603b7a8 admin admin] Validating token access rules against request {{(pid=80378) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:16:12.016758 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc1d8052-4ffe-45ec-9302-960f88e3cd30 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '891005fa5ff74041b75747d16cc66798', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7047912c5acd4adeba5d87c3bf895e39', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80378) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:16:12.017494 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc1d8052-4ffe-45ec-9302-960f88e3cd30 admin admin] REQUEST_METHOD: `PUT` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:16:12.017897 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc1d8052-4ffe-45ec-9302-960f88e3cd30 admin admin] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:16:12.018429 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc1d8052-4ffe-45ec-9302-960f88e3cd30 admin admin] PATH_INFO: `/v3/projects/01874b4ed3844636ad2cef0be4876db7/users/2159c8b1594e482d9e1955b6491737b3/roles/237966442f05411191105bf336bc14f3` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:16:12.058749 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc1d8052-4ffe-45ec-9302-960f88e3cd30 admin admin] RBAC: Authorizing `identity:create_grant(project_id=01874b4ed3844636ad2cef0be4876db7, user_id=2159c8b1594e482d9e1955b6491737b3, role_id=237966442f05411191105bf336bc14f3)` {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:16:12.064146 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc1d8052-4ffe-45ec-9302-960f88e3cd30 admin admin] RBAC: Authorization granted {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:16:12.080524 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG dbcounter [-] [80378] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=80378) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 10:16:12.085031 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e45e959f-9925-4bac-90c5-45772cf68df1 admin admin] Authenticating user token {{(pid=80377) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:16:12.087545 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.fernet_utils [None req-e45e959f-9925-4bac-90c5-45772cf68df1 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80377) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 10:16:12.101935 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 839/1692] 10.222.0.82 () {60 vars in 1483 bytes} [Tue Aug 2 10:16:11 2022] PUT /identity/v3/projects/01874b4ed3844636ad2cef0be4876db7/users/2159c8b1594e482d9e1955b6491737b3/roles/237966442f05411191105bf336bc14f3 => generated 0 bytes in 132 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 02 10:16:12.112253 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc1d8052-4ffe-45ec-9302-960f88e3cd30 admin admin] Authenticating user token {{(pid=80378) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:16:12.230095 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e45e959f-9925-4bac-90c5-45772cf68df1 admin admin] Validating token access rules against request {{(pid=80377) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:16:12.263031 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc1d8052-4ffe-45ec-9302-960f88e3cd30 admin admin] Validating token access rules against request {{(pid=80378) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:16:12.268105 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a33da1df-7aec-4bba-ba48-4a7ab7a0ecac admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '891005fa5ff74041b75747d16cc66798', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7047912c5acd4adeba5d87c3bf895e39', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80378) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:16:12.268895 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a33da1df-7aec-4bba-ba48-4a7ab7a0ecac admin admin] REQUEST_METHOD: `GET` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:16:12.269192 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a33da1df-7aec-4bba-ba48-4a7ab7a0ecac admin admin] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:16:12.269467 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a33da1df-7aec-4bba-ba48-4a7ab7a0ecac admin admin] PATH_INFO: `/v3/roles` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:16:12.270291 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a33da1df-7aec-4bba-ba48-4a7ab7a0ecac admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:16:12.273353 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a33da1df-7aec-4bba-ba48-4a7ab7a0ecac admin admin] RBAC: Authorization granted {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:16:12.292280 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 840/1693] 10.222.0.82 () {58 vars in 1122 bytes} [Tue Aug 2 10:16:12 2022] GET /identity/v3/roles => generated 1420 bytes in 181 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 02 10:16:12.293424 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a33da1df-7aec-4bba-ba48-4a7ab7a0ecac admin admin] Authenticating user token {{(pid=80378) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:16:12.308559 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a33da1df-7aec-4bba-ba48-4a7ab7a0ecac admin admin] Validating token access rules against request {{(pid=80378) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:16:12.316512 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c511c964-3a50-4ee3-adf9-156dd06bcf99 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '891005fa5ff74041b75747d16cc66798', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7047912c5acd4adeba5d87c3bf895e39', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80378) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:16:12.317287 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c511c964-3a50-4ee3-adf9-156dd06bcf99 admin admin] REQUEST_METHOD: `PUT` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:16:12.317580 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c511c964-3a50-4ee3-adf9-156dd06bcf99 admin admin] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:16:12.317948 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c511c964-3a50-4ee3-adf9-156dd06bcf99 admin admin] PATH_INFO: `/v3/projects/01874b4ed3844636ad2cef0be4876db7/users/2159c8b1594e482d9e1955b6491737b3/roles/237966442f05411191105bf336bc14f3` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:16:12.320379 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c511c964-3a50-4ee3-adf9-156dd06bcf99 admin admin] RBAC: Authorizing `identity:create_grant(project_id=01874b4ed3844636ad2cef0be4876db7, user_id=2159c8b1594e482d9e1955b6491737b3, role_id=237966442f05411191105bf336bc14f3)` {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:16:12.325335 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c511c964-3a50-4ee3-adf9-156dd06bcf99 admin admin] RBAC: Authorization granted {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:16:12.349027 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 841/1694] 10.222.0.82 () {60 vars in 1483 bytes} [Tue Aug 2 10:16:12 2022] PUT /identity/v3/projects/01874b4ed3844636ad2cef0be4876db7/users/2159c8b1594e482d9e1955b6491737b3/roles/237966442f05411191105bf336bc14f3 => generated 0 bytes in 46 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 02 10:16:12.350578 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-148558b4-6bd0-4c55-9186-753143ccbbea None None] REQUEST_METHOD: `POST` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:16:12.350879 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-148558b4-6bd0-4c55-9186-753143ccbbea None None] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:16:12.351161 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-148558b4-6bd0-4c55-9186-753143ccbbea None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:16:12.376901 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18d73137-d295-4138-ac82-8e5014581bd2 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '33b87170f0634b67bce60750847b5cbe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f1b38ebb804c41c69772c998aae72c41', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80377) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:16:12.377530 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18d73137-d295-4138-ac82-8e5014581bd2 service neutron] REQUEST_METHOD: `GET` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:16:12.377827 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18d73137-d295-4138-ac82-8e5014581bd2 service neutron] SCRIPT_NAME: `/identity` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:16:12.378112 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18d73137-d295-4138-ac82-8e5014581bd2 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:16:12.392984 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.auth.core [None req-148558b4-6bd0-4c55-9186-753143ccbbea None None] MFA Rules not processed for user `2159c8b1594e482d9e1955b6491737b3`. Rule list: `[]` (Enabled: `True`). {{(pid=80378) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 02 10:16:12.395373 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.fernet_utils [None req-148558b4-6bd0-4c55-9186-753143ccbbea None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80378) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 10:16:12.403437 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18d73137-d295-4138-ac82-8e5014581bd2 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:16:12.404816 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18d73137-d295-4138-ac82-8e5014581bd2 service neutron] RBAC: Authorization granted {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:16:12.433823 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: [pid: 80377|app: 0|req: 854/1695] 10.222.0.82 () {60 vars in 1365 bytes} [Tue Aug 2 10:16:12 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 350 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 02 10:16:12.527985 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 842/1696] 10.222.0.82 () {58 vars in 958 bytes} [Tue Aug 2 10:16:12 2022] POST /identity/v3/auth/tokens => generated 3351 bytes in 178 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 02 10:16:12.543134 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18d73137-d295-4138-ac82-8e5014581bd2 service neutron] Authenticating user token {{(pid=80377) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:16:12.544871 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.fernet_utils [None req-18d73137-d295-4138-ac82-8e5014581bd2 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80377) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 10:16:12.556202 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-148558b4-6bd0-4c55-9186-753143ccbbea None None] Authenticating user token {{(pid=80378) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:16:12.561771 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.fernet_utils [None req-148558b4-6bd0-4c55-9186-753143ccbbea None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80378) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 10:16:12.573711 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG dbcounter [-] [80377] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80377) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 10:16:12.686343 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18d73137-d295-4138-ac82-8e5014581bd2 service neutron] Validating token access rules against request {{(pid=80377) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:16:12.748131 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-148558b4-6bd0-4c55-9186-753143ccbbea None None] Validating token access rules against request {{(pid=80378) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:16:12.760379 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88302e8b-22c7-480e-be43-730492210a37 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '891005fa5ff74041b75747d16cc66798', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7047912c5acd4adeba5d87c3bf895e39', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80378) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:16:12.762177 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88302e8b-22c7-480e-be43-730492210a37 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:16:12.763427 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88302e8b-22c7-480e-be43-730492210a37 admin admin] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:16:12.764316 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88302e8b-22c7-480e-be43-730492210a37 admin admin] PATH_INFO: `/v3/projects/4391660601b4468c9998da049edd3440` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:16:12.768303 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88302e8b-22c7-480e-be43-730492210a37 admin admin] RBAC: Authorizing `identity:delete_project(project_id=4391660601b4468c9998da049edd3440)` {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:16:12.771784 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88302e8b-22c7-480e-be43-730492210a37 admin admin] RBAC: Authorization granted {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:16:12.779859 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG dbcounter [-] [80378] Writing DB stats keystone:INSERT=2,keystone:SELECT=98 {{(pid=80378) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 10:16:12.815064 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b406730-4248-4737-84d0-9216ac92d134 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1023de281ae74e9d8a6d6c312c1c97a7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f1b38ebb804c41c69772c998aae72c41', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80377) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:16:12.816480 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b406730-4248-4737-84d0-9216ac92d134 service cinder] REQUEST_METHOD: `GET` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:16:12.817685 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b406730-4248-4737-84d0-9216ac92d134 service cinder] SCRIPT_NAME: `/identity` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:16:12.818081 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b406730-4248-4737-84d0-9216ac92d134 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:16:12.840460 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b406730-4248-4737-84d0-9216ac92d134 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:16:12.849827 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b406730-4248-4737-84d0-9216ac92d134 service cinder] RBAC: Authorization granted {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:16:12.878274 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: [pid: 80377|app: 0|req: 855/1697] 10.222.0.82 () {60 vars in 1365 bytes} [Tue Aug 2 10:16:12 2022] GET /identity/v3/auth/tokens => generated 3351 bytes in 336 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 02 10:16:12.882313 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0af5d6e-be90-44cb-9d31-e6fd6102fe3f None None] REQUEST_METHOD: `POST` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:16:12.883056 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0af5d6e-be90-44cb-9d31-e6fd6102fe3f None None] SCRIPT_NAME: `/identity` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:16:12.883339 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0af5d6e-be90-44cb-9d31-e6fd6102fe3f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:16:12.886047 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.notifications [None req-88302e8b-22c7-480e-be43-730492210a37 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '4391660601b4468c9998da049edd3440', 'request_id': 'req-88302e8b-22c7-480e-be43-730492210a37'} {{(pid=80378) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 10:16:12.911433 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.notifications [None req-88302e8b-22c7-480e-be43-730492210a37 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '4391660601b4468c9998da049edd3440', 'request_id': 'req-88302e8b-22c7-480e-be43-730492210a37'} {{(pid=80378) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 10:16:12.938178 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.notifications [None req-88302e8b-22c7-480e-be43-730492210a37 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '4391660601b4468c9998da049edd3440', 'request_id': 'req-88302e8b-22c7-480e-be43-730492210a37'} {{(pid=80378) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 10:16:12.959227 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.notifications [None req-88302e8b-22c7-480e-be43-730492210a37 admin admin] The token cache is being invalidate because project 4391660601b4468c9998da049edd3440 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80378) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 02 10:16:12.960257 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.notifications [None req-88302e8b-22c7-480e-be43-730492210a37 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80378) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 10:16:12.961003 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.auth.core [None req-f0af5d6e-be90-44cb-9d31-e6fd6102fe3f None None] MFA Rules not processed for user `e881ae7c60254c12860fff0a40480641`. Rule list: `[]` (Enabled: `True`). {{(pid=80377) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 02 10:16:12.961174 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.notifications [None req-88302e8b-22c7-480e-be43-730492210a37 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80378) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 10:16:12.962381 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.fernet_utils [None req-f0af5d6e-be90-44cb-9d31-e6fd6102fe3f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80377) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 10:16:12.963643 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 843/1698] 10.222.0.82 () {58 vars in 1233 bytes} [Tue Aug 2 10:16:12 2022] DELETE /identity/v3/projects/4391660601b4468c9998da049edd3440 => generated 0 bytes in 408 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 02 10:16:12.968571 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88302e8b-22c7-480e-be43-730492210a37 admin admin] Authenticating user token {{(pid=80378) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:16:12.972596 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.fernet_utils [None req-88302e8b-22c7-480e-be43-730492210a37 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80378) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 10:16:13.115389 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: [pid: 80377|app: 0|req: 856/1699] 10.222.0.82 () {58 vars in 958 bytes} [Tue Aug 2 10:16:12 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 236 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 02 10:16:13.116728 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e189d132-bc13-4286-9aff-42fac819a812 None None] REQUEST_METHOD: `GET` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:16:13.117016 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e189d132-bc13-4286-9aff-42fac819a812 None None] SCRIPT_NAME: `/identity` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:16:13.117294 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e189d132-bc13-4286-9aff-42fac819a812 None None] PATH_INFO: `/` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:16:13.119101 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: [pid: 80377|app: 0|req: 857/1700] 10.222.0.82 () {54 vars in 921 bytes} [Tue Aug 2 10:16:13 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 02 10:16:13.139651 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e189d132-bc13-4286-9aff-42fac819a812 None None] Authenticating user token {{(pid=80377) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:16:13.141916 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.fernet_utils [None req-e189d132-bc13-4286-9aff-42fac819a812 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80377) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 10:16:13.173810 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88302e8b-22c7-480e-be43-730492210a37 admin admin] Validating token access rules against request {{(pid=80378) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:16:13.181102 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9258eb36-210b-4fcd-a652-46e6333d1493 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '891005fa5ff74041b75747d16cc66798', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7047912c5acd4adeba5d87c3bf895e39', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80378) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:16:13.182552 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9258eb36-210b-4fcd-a652-46e6333d1493 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:16:13.183310 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9258eb36-210b-4fcd-a652-46e6333d1493 admin admin] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:16:13.183924 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9258eb36-210b-4fcd-a652-46e6333d1493 admin admin] PATH_INFO: `/v3/users/e291ef3b0c3347128a3ad0e99914a44b` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:16:13.187534 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9258eb36-210b-4fcd-a652-46e6333d1493 admin admin] RBAC: Authorizing `identity:delete_user(user_id=e291ef3b0c3347128a3ad0e99914a44b)` {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:16:13.190270 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9258eb36-210b-4fcd-a652-46e6333d1493 admin admin] RBAC: Authorization granted {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:16:13.272633 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG dbcounter [-] [80377] Writing DB stats keystone:SELECT=100 {{(pid=80377) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 10:16:13.312182 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e189d132-bc13-4286-9aff-42fac819a812 None None] Validating token access rules against request {{(pid=80377) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:16:13.321716 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67b115c7-0b6c-43c0-8b14-363636840125 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88af518196674fe593ad88d30111fb7d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f1b38ebb804c41c69772c998aae72c41', 'project_domain_id': 'default', 'roles': ['admin', 'ResellerAdmin', 'reader', 'service', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80377) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:16:13.323498 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67b115c7-0b6c-43c0-8b14-363636840125 service nova] REQUEST_METHOD: `GET` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:16:13.323905 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67b115c7-0b6c-43c0-8b14-363636840125 service nova] SCRIPT_NAME: `/identity` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:16:13.324234 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67b115c7-0b6c-43c0-8b14-363636840125 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:16:13.337453 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67b115c7-0b6c-43c0-8b14-363636840125 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:16:13.339825 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67b115c7-0b6c-43c0-8b14-363636840125 service nova] RBAC: Authorization granted {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:16:13.340405 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.notifications [None req-9258eb36-210b-4fcd-a652-46e6333d1493 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'e291ef3b0c3347128a3ad0e99914a44b'} {{(pid=80378) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 10:16:13.355056 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: [pid: 80377|app: 0|req: 858/1701] 10.222.0.82 () {60 vars in 1365 bytes} [Tue Aug 2 10:16:13 2022] GET /identity/v3/auth/tokens => generated 3252 bytes in 216 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 02 10:16:13.358385 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bc428e8-e9cf-4eb6-8f86-82e9eac945ef None None] REQUEST_METHOD: `POST` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:16:13.358886 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bc428e8-e9cf-4eb6-8f86-82e9eac945ef None None] SCRIPT_NAME: `/identity` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:16:13.359283 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bc428e8-e9cf-4eb6-8f86-82e9eac945ef None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:16:13.364955 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.notifications [None req-9258eb36-210b-4fcd-a652-46e6333d1493 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'e291ef3b0c3347128a3ad0e99914a44b'} {{(pid=80378) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 10:16:13.386002 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.auth.core [None req-9bc428e8-e9cf-4eb6-8f86-82e9eac945ef None None] MFA Rules not processed for user `ae6e910e50d74f29bfb89bf18d95a908`. Rule list: `[]` (Enabled: `True`). {{(pid=80377) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 02 10:16:13.387507 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.fernet_utils [None req-9bc428e8-e9cf-4eb6-8f86-82e9eac945ef None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80377) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 10:16:13.392301 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.notifications [None req-9258eb36-210b-4fcd-a652-46e6333d1493 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'e291ef3b0c3347128a3ad0e99914a44b'} {{(pid=80378) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 10:16:13.396085 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.notifications [None req-9258eb36-210b-4fcd-a652-46e6333d1493 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'e291ef3b0c3347128a3ad0e99914a44b'} {{(pid=80378) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 10:16:13.442448 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.notifications [None req-9258eb36-210b-4fcd-a652-46e6333d1493 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'e291ef3b0c3347128a3ad0e99914a44b'} {{(pid=80378) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 10:16:13.457067 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 844/1702] 10.222.0.82 () {58 vars in 1224 bytes} [Tue Aug 2 10:16:12 2022] DELETE /identity/v3/users/e291ef3b0c3347128a3ad0e99914a44b => generated 0 bytes in 489 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 02 10:16:13.458274 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9258eb36-210b-4fcd-a652-46e6333d1493 admin admin] Authenticating user token {{(pid=80378) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:16:13.471257 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.fernet_utils [None req-9258eb36-210b-4fcd-a652-46e6333d1493 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80378) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 10:16:13.503419 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG dbcounter [-] [80378] Writing DB stats keystone:SELECT=81,keystone:DELETE=17,keystone:INSERT=2 {{(pid=80378) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 10:16:13.574167 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: [pid: 80377|app: 0|req: 859/1703] 10.222.0.82 () {58 vars in 1018 bytes} [Tue Aug 2 10:16:13 2022] POST /identity/v3/auth/tokens => generated 3308 bytes in 212 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 02 10:16:13.592256 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9258eb36-210b-4fcd-a652-46e6333d1493 admin admin] Validating token access rules against request {{(pid=80378) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:16:13.600724 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79484f5a-05fe-4aaa-ba26-06dea32a33a7 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1023de281ae74e9d8a6d6c312c1c97a7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f1b38ebb804c41c69772c998aae72c41', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80378) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:16:13.601654 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79484f5a-05fe-4aaa-ba26-06dea32a33a7 service cinder] REQUEST_METHOD: `GET` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:16:13.602246 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79484f5a-05fe-4aaa-ba26-06dea32a33a7 service cinder] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:16:13.602588 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79484f5a-05fe-4aaa-ba26-06dea32a33a7 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:16:13.605960 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.fernet_utils [None req-79484f5a-05fe-4aaa-ba26-06dea32a33a7 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80378) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 10:16:13.628740 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54aac83f-3fed-4baf-813a-9724bbfdcc0c None None] REQUEST_METHOD: `GET` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:16:13.628974 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54aac83f-3fed-4baf-813a-9724bbfdcc0c None None] SCRIPT_NAME: `/identity` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:16:13.629223 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54aac83f-3fed-4baf-813a-9724bbfdcc0c None None] PATH_INFO: `/` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:16:13.631070 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: [pid: 80377|app: 0|req: 860/1704] 10.222.0.82 () {54 vars in 961 bytes} [Tue Aug 2 10:16:13 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 02 10:16:13.638841 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b294b23-6b6f-46ac-b067-0d19fd9ac205 None None] REQUEST_METHOD: `POST` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:16:13.639596 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b294b23-6b6f-46ac-b067-0d19fd9ac205 None None] SCRIPT_NAME: `/identity` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:16:13.639888 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b294b23-6b6f-46ac-b067-0d19fd9ac205 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:16:13.682794 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.auth.core [None req-3b294b23-6b6f-46ac-b067-0d19fd9ac205 None None] MFA Rules not processed for user `ae6e910e50d74f29bfb89bf18d95a908`. Rule list: `[]` (Enabled: `True`). {{(pid=80377) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 02 10:16:13.684468 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.fernet_utils [None req-3b294b23-6b6f-46ac-b067-0d19fd9ac205 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80377) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 10:16:13.738485 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79484f5a-05fe-4aaa-ba26-06dea32a33a7 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:16:13.741355 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79484f5a-05fe-4aaa-ba26-06dea32a33a7 service cinder] RBAC: Authorization granted {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:16:13.774159 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 845/1705] 10.222.0.82 () {60 vars in 1365 bytes} [Tue Aug 2 10:16:13 2022] GET /identity/v3/auth/tokens => generated 3286 bytes in 316 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 02 10:16:13.837711 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: [pid: 80377|app: 0|req: 861/1706] 10.222.0.82 () {58 vars in 1058 bytes} [Tue Aug 2 10:16:13 2022] POST /identity/v3/auth/tokens => generated 3308 bytes in 200 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 02 10:16:13.848722 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79484f5a-05fe-4aaa-ba26-06dea32a33a7 service cinder] Authenticating user token {{(pid=80378) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:16:13.864847 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79484f5a-05fe-4aaa-ba26-06dea32a33a7 service cinder] Validating token access rules against request {{(pid=80378) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:16:13.877131 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88bc84d1-8ba7-4ddf-8784-522ef149a2f7 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ae6e910e50d74f29bfb89bf18d95a908', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f1b38ebb804c41c69772c998aae72c41', 'project_domain_id': 'default', 'roles': ['admin', 'service', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80378) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:16:13.880791 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88bc84d1-8ba7-4ddf-8784-522ef149a2f7 service placement] REQUEST_METHOD: `GET` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:16:13.880791 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88bc84d1-8ba7-4ddf-8784-522ef149a2f7 service placement] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:16:13.881110 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88bc84d1-8ba7-4ddf-8784-522ef149a2f7 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:16:13.902830 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88bc84d1-8ba7-4ddf-8784-522ef149a2f7 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:16:13.905348 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88bc84d1-8ba7-4ddf-8784-522ef149a2f7 service placement] RBAC: Authorization granted {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:16:13.921921 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 846/1707] 10.222.0.82 () {60 vars in 1365 bytes} [Tue Aug 2 10:16:13 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 74 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 02 10:16:14.315719 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b294b23-6b6f-46ac-b067-0d19fd9ac205 None None] Authenticating user token {{(pid=80377) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:16:14.317430 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.fernet_utils [None req-3b294b23-6b6f-46ac-b067-0d19fd9ac205 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80377) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 10:16:14.446635 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88bc84d1-8ba7-4ddf-8784-522ef149a2f7 service placement] Authenticating user token {{(pid=80378) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:16:14.465355 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88bc84d1-8ba7-4ddf-8784-522ef149a2f7 service placement] Validating token access rules against request {{(pid=80378) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:16:14.472640 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e14097e8-c02f-4c1b-aa73-7aa371a87282 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ae6e910e50d74f29bfb89bf18d95a908', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f1b38ebb804c41c69772c998aae72c41', 'project_domain_id': 'default', 'roles': ['admin', 'service', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80378) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:16:14.473598 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e14097e8-c02f-4c1b-aa73-7aa371a87282 service placement] REQUEST_METHOD: `GET` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:16:14.475299 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e14097e8-c02f-4c1b-aa73-7aa371a87282 service placement] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:16:14.475778 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e14097e8-c02f-4c1b-aa73-7aa371a87282 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:16:14.478498 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.fernet_utils [None req-e14097e8-c02f-4c1b-aa73-7aa371a87282 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80378) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 10:16:14.494733 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e14097e8-c02f-4c1b-aa73-7aa371a87282 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:16:14.497361 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e14097e8-c02f-4c1b-aa73-7aa371a87282 service placement] RBAC: Authorization granted {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:16:14.541612 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b294b23-6b6f-46ac-b067-0d19fd9ac205 None None] Validating token access rules against request {{(pid=80377) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:16:14.547081 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 847/1708] 10.222.0.82 () {60 vars in 1365 bytes} [Tue Aug 2 10:16:14 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 101 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 02 10:16:14.556553 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96945b2a-b3dc-437a-b40b-5653a4c95580 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '891005fa5ff74041b75747d16cc66798', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7047912c5acd4adeba5d87c3bf895e39', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80377) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:16:14.558048 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96945b2a-b3dc-437a-b40b-5653a4c95580 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:16:14.558258 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96945b2a-b3dc-437a-b40b-5653a4c95580 admin admin] SCRIPT_NAME: `/identity` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:16:14.558721 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96945b2a-b3dc-437a-b40b-5653a4c95580 admin admin] PATH_INFO: `/v3/users/4db0cf73ea0944e98d94446749e28b44` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:16:14.560933 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96945b2a-b3dc-437a-b40b-5653a4c95580 admin admin] RBAC: Authorizing `identity:delete_user(user_id=4db0cf73ea0944e98d94446749e28b44)` {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:16:14.564239 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96945b2a-b3dc-437a-b40b-5653a4c95580 admin admin] RBAC: Authorization granted {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:16:14.627174 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG dbcounter [-] [80377] Writing DB stats keystone:SELECT=98,keystone:DELETE=2 {{(pid=80377) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 10:16:14.686571 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.notifications [None req-96945b2a-b3dc-437a-b40b-5653a4c95580 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '4db0cf73ea0944e98d94446749e28b44'} {{(pid=80377) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 10:16:14.702772 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.notifications [None req-96945b2a-b3dc-437a-b40b-5653a4c95580 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '4db0cf73ea0944e98d94446749e28b44'} {{(pid=80377) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 10:16:14.720533 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.notifications [None req-96945b2a-b3dc-437a-b40b-5653a4c95580 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '4db0cf73ea0944e98d94446749e28b44'} {{(pid=80377) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 10:16:14.722362 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.notifications [None req-96945b2a-b3dc-437a-b40b-5653a4c95580 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '4db0cf73ea0944e98d94446749e28b44'} {{(pid=80377) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 10:16:14.764675 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.notifications [None req-96945b2a-b3dc-437a-b40b-5653a4c95580 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '4db0cf73ea0944e98d94446749e28b44'} {{(pid=80377) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 10:16:14.784842 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: [pid: 80377|app: 0|req: 862/1709] 10.222.0.82 () {58 vars in 1224 bytes} [Tue Aug 2 10:16:14 2022] DELETE /identity/v3/users/4db0cf73ea0944e98d94446749e28b44 => generated 0 bytes in 467 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 02 10:16:14.787815 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f816b2ad-4c6a-483b-acd1-ff8560ecd294 None None] REQUEST_METHOD: `POST` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:16:14.788141 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f816b2ad-4c6a-483b-acd1-ff8560ecd294 None None] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:16:14.788418 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f816b2ad-4c6a-483b-acd1-ff8560ecd294 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:16:14.839502 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.auth.core [None req-f816b2ad-4c6a-483b-acd1-ff8560ecd294 None None] MFA Rules not processed for user `891005fa5ff74041b75747d16cc66798`. Rule list: `[]` (Enabled: `True`). {{(pid=80378) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 02 10:16:14.843600 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.fernet_utils [None req-f816b2ad-4c6a-483b-acd1-ff8560ecd294 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80378) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 10:16:14.944860 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 848/1710] 10.222.0.82 () {58 vars in 958 bytes} [Tue Aug 2 10:16:14 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 158 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 02 10:16:15.007201 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96945b2a-b3dc-437a-b40b-5653a4c95580 admin admin] Authenticating user token {{(pid=80377) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:16:15.008752 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.fernet_utils [None req-96945b2a-b3dc-437a-b40b-5653a4c95580 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80377) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 10:16:15.129672 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96945b2a-b3dc-437a-b40b-5653a4c95580 admin admin] Validating token access rules against request {{(pid=80377) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:16:15.143316 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95abe9d7-749a-47a1-9543-7babc6d2ca82 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '33b87170f0634b67bce60750847b5cbe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f1b38ebb804c41c69772c998aae72c41', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80377) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:16:15.144709 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95abe9d7-749a-47a1-9543-7babc6d2ca82 service neutron] REQUEST_METHOD: `GET` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:16:15.145153 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95abe9d7-749a-47a1-9543-7babc6d2ca82 service neutron] SCRIPT_NAME: `/identity` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:16:15.145579 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95abe9d7-749a-47a1-9543-7babc6d2ca82 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:16:15.166431 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95abe9d7-749a-47a1-9543-7babc6d2ca82 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:16:15.168042 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95abe9d7-749a-47a1-9543-7babc6d2ca82 service neutron] RBAC: Authorization granted {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:16:15.195747 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: [pid: 80377|app: 0|req: 863/1711] 10.222.0.82 () {60 vars in 1365 bytes} [Tue Aug 2 10:16:15 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 189 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 02 10:16:15.468762 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72e6892f-8a80-49d9-ad50-e9334d685241 None None] REQUEST_METHOD: `GET` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:16:15.469261 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72e6892f-8a80-49d9-ad50-e9334d685241 None None] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:16:15.469343 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72e6892f-8a80-49d9-ad50-e9334d685241 None None] PATH_INFO: `/` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:16:15.471490 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 849/1712] 10.222.0.82 () {54 vars in 921 bytes} [Tue Aug 2 10:16:15 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 02 10:16:15.532517 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95abe9d7-749a-47a1-9543-7babc6d2ca82 service neutron] Authenticating user token {{(pid=80377) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:16:15.534052 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.fernet_utils [None req-95abe9d7-749a-47a1-9543-7babc6d2ca82 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80377) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 10:16:15.568532 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95abe9d7-749a-47a1-9543-7babc6d2ca82 service neutron] Validating token access rules against request {{(pid=80377) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:16:15.599716 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e0d43fb-08c9-486e-a3bd-783d459e1a9c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '891005fa5ff74041b75747d16cc66798', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7047912c5acd4adeba5d87c3bf895e39', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80377) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:16:15.600453 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e0d43fb-08c9-486e-a3bd-783d459e1a9c admin admin] REQUEST_METHOD: `DELETE` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:16:15.600637 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e0d43fb-08c9-486e-a3bd-783d459e1a9c admin admin] SCRIPT_NAME: `/identity` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:16:15.600913 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e0d43fb-08c9-486e-a3bd-783d459e1a9c admin admin] PATH_INFO: `/v3/projects/a19cdac12af2479fb179d691e096b8a9` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:16:15.602374 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e0d43fb-08c9-486e-a3bd-783d459e1a9c admin admin] RBAC: Authorizing `identity:delete_project(project_id=a19cdac12af2479fb179d691e096b8a9)` {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:16:15.604665 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e0d43fb-08c9-486e-a3bd-783d459e1a9c admin admin] RBAC: Authorization granted {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:16:15.619479 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9329386a-769a-47c6-b374-15abbe77c00c None None] REQUEST_METHOD: `POST` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:16:15.620060 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9329386a-769a-47c6-b374-15abbe77c00c None None] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:16:15.621771 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9329386a-769a-47c6-b374-15abbe77c00c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:16:15.766945 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.auth.core [None req-9329386a-769a-47c6-b374-15abbe77c00c None None] MFA Rules not processed for user `ae6e910e50d74f29bfb89bf18d95a908`. Rule list: `[]` (Enabled: `True`). {{(pid=80378) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 02 10:16:15.783676 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.fernet_utils [None req-9329386a-769a-47c6-b374-15abbe77c00c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80378) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 10:16:15.858863 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.notifications [None req-6e0d43fb-08c9-486e-a3bd-783d459e1a9c admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'a19cdac12af2479fb179d691e096b8a9', 'request_id': 'req-6e0d43fb-08c9-486e-a3bd-783d459e1a9c'} {{(pid=80377) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 10:16:15.883163 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.notifications [None req-6e0d43fb-08c9-486e-a3bd-783d459e1a9c admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'a19cdac12af2479fb179d691e096b8a9', 'request_id': 'req-6e0d43fb-08c9-486e-a3bd-783d459e1a9c'} {{(pid=80377) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 10:16:15.904529 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.notifications [None req-6e0d43fb-08c9-486e-a3bd-783d459e1a9c admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'a19cdac12af2479fb179d691e096b8a9', 'request_id': 'req-6e0d43fb-08c9-486e-a3bd-783d459e1a9c'} {{(pid=80377) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 10:16:15.926847 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG dbcounter [-] [80378] Writing DB stats keystone:SELECT=100 {{(pid=80378) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 10:16:15.956439 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.notifications [None req-6e0d43fb-08c9-486e-a3bd-783d459e1a9c admin admin] The token cache is being invalidate because project a19cdac12af2479fb179d691e096b8a9 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80377) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 02 10:16:15.956838 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.notifications [None req-6e0d43fb-08c9-486e-a3bd-783d459e1a9c admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80377) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 10:16:15.958574 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.notifications [None req-6e0d43fb-08c9-486e-a3bd-783d459e1a9c admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80377) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 10:16:15.961276 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: [pid: 80377|app: 0|req: 864/1713] 10.222.0.82 () {58 vars in 1233 bytes} [Tue Aug 2 10:16:15 2022] DELETE /identity/v3/projects/a19cdac12af2479fb179d691e096b8a9 => generated 0 bytes in 429 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 02 10:16:15.962333 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e0d43fb-08c9-486e-a3bd-783d459e1a9c admin admin] Authenticating user token {{(pid=80377) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:16:15.970705 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.fernet_utils [None req-6e0d43fb-08c9-486e-a3bd-783d459e1a9c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80377) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 10:16:16.007589 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 850/1714] 10.222.0.82 () {58 vars in 1018 bytes} [Tue Aug 2 10:16:15 2022] POST /identity/v3/auth/tokens => generated 3308 bytes in 389 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 02 10:16:16.009139 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4de68f45-eb98-412c-abe2-3aaabf40acc4 None None] REQUEST_METHOD: `POST` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:16:16.012385 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4de68f45-eb98-412c-abe2-3aaabf40acc4 None None] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:16:16.013737 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4de68f45-eb98-412c-abe2-3aaabf40acc4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:16:16.058510 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG dbcounter [-] [80377] Writing DB stats keystone:DELETE=15,keystone:SELECT=83,keystone:INSERT=2 {{(pid=80377) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 10:16:16.082256 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.auth.core [None req-4de68f45-eb98-412c-abe2-3aaabf40acc4 None None] MFA Rules not processed for user `891005fa5ff74041b75747d16cc66798`. Rule list: `[]` (Enabled: `True`). {{(pid=80378) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 02 10:16:16.083906 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.fernet_utils [None req-4de68f45-eb98-412c-abe2-3aaabf40acc4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80378) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 10:16:16.313868 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e0d43fb-08c9-486e-a3bd-783d459e1a9c admin admin] Validating token access rules against request {{(pid=80377) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:16:16.336116 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-824065ae-e3eb-480d-aeaf-369afd2d7f1c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '891005fa5ff74041b75747d16cc66798', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7047912c5acd4adeba5d87c3bf895e39', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80377) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:16:16.337558 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-824065ae-e3eb-480d-aeaf-369afd2d7f1c admin admin] REQUEST_METHOD: `DELETE` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:16:16.337558 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-824065ae-e3eb-480d-aeaf-369afd2d7f1c admin admin] SCRIPT_NAME: `/identity` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:16:16.339968 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-824065ae-e3eb-480d-aeaf-369afd2d7f1c admin admin] PATH_INFO: `/v3/users/8debb5e1e38a40159eb1713169f93cc0` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:16:16.353835 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 851/1715] 10.222.0.82 () {58 vars in 958 bytes} [Tue Aug 2 10:16:16 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 345 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 02 10:16:16.354075 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-824065ae-e3eb-480d-aeaf-369afd2d7f1c admin admin] RBAC: Authorizing `identity:delete_user(user_id=8debb5e1e38a40159eb1713169f93cc0)` {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:16:16.355202 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4de68f45-eb98-412c-abe2-3aaabf40acc4 None None] Authenticating user token {{(pid=80378) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:16:16.361547 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-824065ae-e3eb-480d-aeaf-369afd2d7f1c admin admin] RBAC: Authorization granted {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:16:16.364279 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.fernet_utils [None req-4de68f45-eb98-412c-abe2-3aaabf40acc4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80378) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 10:16:16.475599 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4de68f45-eb98-412c-abe2-3aaabf40acc4 None None] Validating token access rules against request {{(pid=80378) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:16:16.483905 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbb0a8f8-7d7e-449b-a230-de53186c50aa service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ae6e910e50d74f29bfb89bf18d95a908', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f1b38ebb804c41c69772c998aae72c41', 'project_domain_id': 'default', 'roles': ['admin', 'service', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80378) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:16:16.484776 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbb0a8f8-7d7e-449b-a230-de53186c50aa service placement] REQUEST_METHOD: `GET` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:16:16.485133 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbb0a8f8-7d7e-449b-a230-de53186c50aa service placement] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:16:16.485481 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbb0a8f8-7d7e-449b-a230-de53186c50aa service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:16:16.501437 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cbb0a8f8-7d7e-449b-a230-de53186c50aa service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:16:16.503453 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cbb0a8f8-7d7e-449b-a230-de53186c50aa service placement] RBAC: Authorization granted {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:16:16.518765 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 852/1716] 10.222.0.82 () {60 vars in 1365 bytes} [Tue Aug 2 10:16:16 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 165 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 02 10:16:16.521723 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12cd1d47-5582-4b0e-aa36-8086e9495401 None None] REQUEST_METHOD: `POST` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:16:16.521723 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12cd1d47-5582-4b0e-aa36-8086e9495401 None None] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:16:16.521723 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12cd1d47-5582-4b0e-aa36-8086e9495401 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:16:16.565365 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.auth.core [None req-12cd1d47-5582-4b0e-aa36-8086e9495401 None None] MFA Rules not processed for user `891005fa5ff74041b75747d16cc66798`. Rule list: `[]` (Enabled: `True`). {{(pid=80378) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 02 10:16:16.566734 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.notifications [None req-824065ae-e3eb-480d-aeaf-369afd2d7f1c admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '8debb5e1e38a40159eb1713169f93cc0'} {{(pid=80377) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 10:16:16.567197 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.fernet_utils [None req-12cd1d47-5582-4b0e-aa36-8086e9495401 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80378) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 10:16:16.590725 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 853/1717] 10.222.0.82 () {58 vars in 958 bytes} [Tue Aug 2 10:16:16 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 72 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 02 10:16:16.596760 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12cd1d47-5582-4b0e-aa36-8086e9495401 None None] Authenticating user token {{(pid=80378) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:16:16.604302 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.notifications [None req-824065ae-e3eb-480d-aeaf-369afd2d7f1c admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '8debb5e1e38a40159eb1713169f93cc0'} {{(pid=80377) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 10:16:16.616085 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12cd1d47-5582-4b0e-aa36-8086e9495401 None None] Validating token access rules against request {{(pid=80378) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:16:16.631548 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e94c229-faaf-4875-8fb6-2ce3abc91413 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '891005fa5ff74041b75747d16cc66798', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7047912c5acd4adeba5d87c3bf895e39', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80378) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:16:16.632481 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e94c229-faaf-4875-8fb6-2ce3abc91413 admin admin] REQUEST_METHOD: `GET` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:16:16.633311 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e94c229-faaf-4875-8fb6-2ce3abc91413 admin admin] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:16:16.633590 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e94c229-faaf-4875-8fb6-2ce3abc91413 admin admin] PATH_INFO: `/v3/domains` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:16:16.634343 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e94c229-faaf-4875-8fb6-2ce3abc91413 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80378) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 02 10:16:16.634790 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e94c229-faaf-4875-8fb6-2ce3abc91413 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:16:16.643924 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e94c229-faaf-4875-8fb6-2ce3abc91413 admin admin] RBAC: Authorization granted {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:16:16.653453 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.notifications [None req-824065ae-e3eb-480d-aeaf-369afd2d7f1c admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '8debb5e1e38a40159eb1713169f93cc0'} {{(pid=80377) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 10:16:16.662685 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.notifications [None req-824065ae-e3eb-480d-aeaf-369afd2d7f1c admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '8debb5e1e38a40159eb1713169f93cc0'} {{(pid=80377) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 10:16:16.678430 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 854/1718] 10.222.0.82 () {58 vars in 1153 bytes} [Tue Aug 2 10:16:16 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 82 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 02 10:16:16.681704 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e94c229-faaf-4875-8fb6-2ce3abc91413 admin admin] Authenticating user token {{(pid=80378) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:16:16.692401 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.fernet_utils [None req-7e94c229-faaf-4875-8fb6-2ce3abc91413 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80378) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 10:16:16.714892 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e94c229-faaf-4875-8fb6-2ce3abc91413 admin admin] Validating token access rules against request {{(pid=80378) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:16:16.742649 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bef0d0db-da70-4cea-b7c7-098a2e74d0b9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '891005fa5ff74041b75747d16cc66798', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7047912c5acd4adeba5d87c3bf895e39', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80378) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:16:16.743355 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bef0d0db-da70-4cea-b7c7-098a2e74d0b9 admin admin] REQUEST_METHOD: `POST` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:16:16.743821 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bef0d0db-da70-4cea-b7c7-098a2e74d0b9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:16:16.747524 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bef0d0db-da70-4cea-b7c7-098a2e74d0b9 admin admin] PATH_INFO: `/v3/projects` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:16:16.750006 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bef0d0db-da70-4cea-b7c7-098a2e74d0b9 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:16:16.757162 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bef0d0db-da70-4cea-b7c7-098a2e74d0b9 admin admin] RBAC: Authorization granted {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:16:16.785596 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.notifications [None req-824065ae-e3eb-480d-aeaf-369afd2d7f1c admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '8debb5e1e38a40159eb1713169f93cc0'} {{(pid=80377) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 10:16:16.854566 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: [pid: 80377|app: 0|req: 865/1719] 10.222.0.82 () {58 vars in 1224 bytes} [Tue Aug 2 10:16:15 2022] DELETE /identity/v3/users/8debb5e1e38a40159eb1713169f93cc0 => generated 0 bytes in 891 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 02 10:16:16.858476 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-824065ae-e3eb-480d-aeaf-369afd2d7f1c admin admin] Authenticating user token {{(pid=80377) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:16:16.863336 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.fernet_utils [None req-824065ae-e3eb-480d-aeaf-369afd2d7f1c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80377) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 10:16:16.868278 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 855/1720] 10.222.0.82 () {60 vars in 1153 bytes} [Tue Aug 2 10:16:16 2022] POST /identity/v3/projects => generated 357 bytes in 182 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 02 10:16:16.869431 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bef0d0db-da70-4cea-b7c7-098a2e74d0b9 admin admin] Authenticating user token {{(pid=80378) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:16:16.945973 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG dbcounter [-] [80378] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80378) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 10:16:17.028273 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bef0d0db-da70-4cea-b7c7-098a2e74d0b9 admin admin] Validating token access rules against request {{(pid=80378) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:16:17.040249 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ccd8063-50af-445c-b26b-f971ccb02aea admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '891005fa5ff74041b75747d16cc66798', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7047912c5acd4adeba5d87c3bf895e39', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80378) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:16:17.040968 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ccd8063-50af-445c-b26b-f971ccb02aea admin admin] REQUEST_METHOD: `POST` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:16:17.041249 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ccd8063-50af-445c-b26b-f971ccb02aea admin admin] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:16:17.041528 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ccd8063-50af-445c-b26b-f971ccb02aea admin admin] PATH_INFO: `/v3/users` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:16:17.042426 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ccd8063-50af-445c-b26b-f971ccb02aea admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:16:17.044255 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ccd8063-50af-445c-b26b-f971ccb02aea admin admin] RBAC: Authorization granted {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:16:17.111945 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 856/1721] 10.222.0.82 () {60 vars in 1144 bytes} [Tue Aug 2 10:16:16 2022] POST /identity/v3/users => generated 279 bytes in 239 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 02 10:16:17.123809 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ccd8063-50af-445c-b26b-f971ccb02aea admin admin] Authenticating user token {{(pid=80378) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:16:17.157910 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ccd8063-50af-445c-b26b-f971ccb02aea admin admin] Validating token access rules against request {{(pid=80378) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:16:17.174936 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7dd66f99-c934-435f-8fd4-e07a2d1aae26 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '891005fa5ff74041b75747d16cc66798', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7047912c5acd4adeba5d87c3bf895e39', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80378) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:16:17.178067 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7dd66f99-c934-435f-8fd4-e07a2d1aae26 admin admin] REQUEST_METHOD: `GET` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:16:17.178067 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7dd66f99-c934-435f-8fd4-e07a2d1aae26 admin admin] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:16:17.178067 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7dd66f99-c934-435f-8fd4-e07a2d1aae26 admin admin] PATH_INFO: `/v3/roles` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:16:17.178067 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7dd66f99-c934-435f-8fd4-e07a2d1aae26 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:16:17.191570 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7dd66f99-c934-435f-8fd4-e07a2d1aae26 admin admin] RBAC: Authorization granted {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:16:17.219513 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-824065ae-e3eb-480d-aeaf-369afd2d7f1c admin admin] Validating token access rules against request {{(pid=80377) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:16:17.229479 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 857/1722] 10.222.0.82 () {58 vars in 1122 bytes} [Tue Aug 2 10:16:17 2022] GET /identity/v3/roles => generated 1420 bytes in 104 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 02 10:16:17.230894 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7dd66f99-c934-435f-8fd4-e07a2d1aae26 admin admin] Authenticating user token {{(pid=80378) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:16:17.234503 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bea79cb-99c1-4aea-9392-e16a790112c4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '891005fa5ff74041b75747d16cc66798', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7047912c5acd4adeba5d87c3bf895e39', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80377) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:16:17.235190 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bea79cb-99c1-4aea-9392-e16a790112c4 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:16:17.235476 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bea79cb-99c1-4aea-9392-e16a790112c4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:16:17.235760 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bea79cb-99c1-4aea-9392-e16a790112c4 admin admin] PATH_INFO: `/v3/projects/b349cf4ab586491097572ca9556b268f` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:16:17.241642 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7bea79cb-99c1-4aea-9392-e16a790112c4 admin admin] RBAC: Authorizing `identity:delete_project(project_id=b349cf4ab586491097572ca9556b268f)` {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:16:17.243671 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7bea79cb-99c1-4aea-9392-e16a790112c4 admin admin] RBAC: Authorization granted {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:16:17.259370 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7dd66f99-c934-435f-8fd4-e07a2d1aae26 admin admin] Validating token access rules against request {{(pid=80378) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:16:17.284154 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63ca0e33-8d41-4c27-aa09-85d12f42bee3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '891005fa5ff74041b75747d16cc66798', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7047912c5acd4adeba5d87c3bf895e39', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80378) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:16:17.284817 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63ca0e33-8d41-4c27-aa09-85d12f42bee3 admin admin] REQUEST_METHOD: `GET` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:16:17.285083 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63ca0e33-8d41-4c27-aa09-85d12f42bee3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:16:17.285357 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63ca0e33-8d41-4c27-aa09-85d12f42bee3 admin admin] PATH_INFO: `/v3/roles` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:16:17.286450 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63ca0e33-8d41-4c27-aa09-85d12f42bee3 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:16:17.288406 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63ca0e33-8d41-4c27-aa09-85d12f42bee3 admin admin] RBAC: Authorization granted {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:16:17.334748 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG dbcounter [-] [80377] Writing DB stats keystone:SELECT=86,keystone:DELETE=13,keystone:INSERT=1 {{(pid=80377) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 10:16:17.342466 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.notifications [None req-7bea79cb-99c1-4aea-9392-e16a790112c4 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'b349cf4ab586491097572ca9556b268f', 'request_id': 'req-7bea79cb-99c1-4aea-9392-e16a790112c4'} {{(pid=80377) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 10:16:17.348148 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 858/1723] 10.222.0.82 () {58 vars in 1122 bytes} [Tue Aug 2 10:16:17 2022] GET /identity/v3/roles => generated 1420 bytes in 113 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 02 10:16:17.350364 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63ca0e33-8d41-4c27-aa09-85d12f42bee3 admin admin] Authenticating user token {{(pid=80378) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:16:17.358985 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.notifications [None req-7bea79cb-99c1-4aea-9392-e16a790112c4 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'b349cf4ab586491097572ca9556b268f', 'request_id': 'req-7bea79cb-99c1-4aea-9392-e16a790112c4'} {{(pid=80377) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 10:16:17.372009 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.notifications [None req-7bea79cb-99c1-4aea-9392-e16a790112c4 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'b349cf4ab586491097572ca9556b268f', 'request_id': 'req-7bea79cb-99c1-4aea-9392-e16a790112c4'} {{(pid=80377) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 10:16:17.374423 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63ca0e33-8d41-4c27-aa09-85d12f42bee3 admin admin] Validating token access rules against request {{(pid=80378) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:16:17.398999 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.notifications [None req-7bea79cb-99c1-4aea-9392-e16a790112c4 admin admin] The token cache is being invalidate because project b349cf4ab586491097572ca9556b268f was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80377) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 02 10:16:17.400280 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.notifications [None req-7bea79cb-99c1-4aea-9392-e16a790112c4 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80377) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 10:16:17.403666 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.notifications [None req-7bea79cb-99c1-4aea-9392-e16a790112c4 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80377) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 10:16:17.406488 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: [pid: 80377|app: 0|req: 866/1724] 10.222.0.82 () {58 vars in 1233 bytes} [Tue Aug 2 10:16:16 2022] DELETE /identity/v3/projects/b349cf4ab586491097572ca9556b268f => generated 0 bytes in 551 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 02 10:16:17.412896 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da3a6272-29cf-401a-9495-062949920f65 None None] REQUEST_METHOD: `POST` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:16:17.413337 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da3a6272-29cf-401a-9495-062949920f65 None None] SCRIPT_NAME: `/identity` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:16:17.414147 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da3a6272-29cf-401a-9495-062949920f65 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:16:17.468741 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.auth.core [None req-da3a6272-29cf-401a-9495-062949920f65 None None] MFA Rules not processed for user `891005fa5ff74041b75747d16cc66798`. Rule list: `[]` (Enabled: `True`). {{(pid=80377) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 02 10:16:17.471107 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.fernet_utils [None req-da3a6272-29cf-401a-9495-062949920f65 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80377) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 10:16:17.536865 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9c4f31c-a72b-486a-8c8b-23ccf8eaf99b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '891005fa5ff74041b75747d16cc66798', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7047912c5acd4adeba5d87c3bf895e39', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80378) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:16:17.539996 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9c4f31c-a72b-486a-8c8b-23ccf8eaf99b admin admin] REQUEST_METHOD: `PUT` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:16:17.540292 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9c4f31c-a72b-486a-8c8b-23ccf8eaf99b admin admin] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:16:17.540581 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9c4f31c-a72b-486a-8c8b-23ccf8eaf99b admin admin] PATH_INFO: `/v3/projects/cd52f29ebc21465197ac524eaf664d73/users/3808aca2179747538f6ee19cf9b1c817/roles/e1c04b19fc4543b8b54f58bbcffbc598` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:16:17.579840 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9c4f31c-a72b-486a-8c8b-23ccf8eaf99b admin admin] RBAC: Authorizing `identity:create_grant(project_id=cd52f29ebc21465197ac524eaf664d73, user_id=3808aca2179747538f6ee19cf9b1c817, role_id=e1c04b19fc4543b8b54f58bbcffbc598)` {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:16:17.582941 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9c4f31c-a72b-486a-8c8b-23ccf8eaf99b admin admin] RBAC: Authorization granted {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:16:17.602189 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: [pid: 80377|app: 0|req: 867/1725] 10.222.0.82 () {58 vars in 958 bytes} [Tue Aug 2 10:16:17 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 190 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 02 10:16:17.609237 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4b4b10a-5628-43c3-a0ec-66398f88bb77 None None] REQUEST_METHOD: `POST` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:16:17.609661 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4b4b10a-5628-43c3-a0ec-66398f88bb77 None None] SCRIPT_NAME: `/identity` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:16:17.610101 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4b4b10a-5628-43c3-a0ec-66398f88bb77 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:16:17.621018 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 859/1726] 10.222.0.82 () {60 vars in 1483 bytes} [Tue Aug 2 10:16:17 2022] PUT /identity/v3/projects/cd52f29ebc21465197ac524eaf664d73/users/3808aca2179747538f6ee19cf9b1c817/roles/e1c04b19fc4543b8b54f58bbcffbc598 => generated 0 bytes in 271 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 02 10:16:17.639532 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d95a6f11-102c-49ea-9cf1-23fe4c73c35b None None] REQUEST_METHOD: `POST` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:16:17.639842 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d95a6f11-102c-49ea-9cf1-23fe4c73c35b None None] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:16:17.640123 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d95a6f11-102c-49ea-9cf1-23fe4c73c35b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:16:17.658608 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.auth.core [None req-d4b4b10a-5628-43c3-a0ec-66398f88bb77 None None] MFA Rules not processed for user `891005fa5ff74041b75747d16cc66798`. Rule list: `[]` (Enabled: `True`). {{(pid=80377) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 02 10:16:17.661314 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.fernet_utils [None req-d4b4b10a-5628-43c3-a0ec-66398f88bb77 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80377) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 10:16:17.704348 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.auth.core [None req-d95a6f11-102c-49ea-9cf1-23fe4c73c35b None None] MFA Rules not processed for user `3808aca2179747538f6ee19cf9b1c817`. Rule list: `[]` (Enabled: `True`). {{(pid=80378) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 02 10:16:17.706151 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.fernet_utils [None req-d95a6f11-102c-49ea-9cf1-23fe4c73c35b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80378) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 10:16:17.803106 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 860/1727] 10.222.0.82 () {58 vars in 958 bytes} [Tue Aug 2 10:16:17 2022] POST /identity/v3/auth/tokens => generated 3234 bytes in 162 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 02 10:16:17.804815 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d95a6f11-102c-49ea-9cf1-23fe4c73c35b None None] Authenticating user token {{(pid=80378) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:16:17.809147 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.fernet_utils [None req-d95a6f11-102c-49ea-9cf1-23fe4c73c35b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80378) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 10:16:17.819625 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG dbcounter [-] [80378] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=80378) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 10:16:17.843503 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: [pid: 80377|app: 0|req: 868/1728] 10.222.0.82 () {58 vars in 958 bytes} [Tue Aug 2 10:16:17 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 236 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 02 10:16:17.847857 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4b4b10a-5628-43c3-a0ec-66398f88bb77 None None] Authenticating user token {{(pid=80377) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:16:17.873133 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4b4b10a-5628-43c3-a0ec-66398f88bb77 None None] Validating token access rules against request {{(pid=80377) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:16:17.890221 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24022955-daaa-4963-959b-6f1c67c9a658 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '891005fa5ff74041b75747d16cc66798', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7047912c5acd4adeba5d87c3bf895e39', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80377) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:16:17.891288 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24022955-daaa-4963-959b-6f1c67c9a658 admin admin] REQUEST_METHOD: `GET` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:16:17.891288 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24022955-daaa-4963-959b-6f1c67c9a658 admin admin] SCRIPT_NAME: `/identity` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:16:17.891510 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24022955-daaa-4963-959b-6f1c67c9a658 admin admin] PATH_INFO: `/v3/domains` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:16:17.892116 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24022955-daaa-4963-959b-6f1c67c9a658 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80377) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 02 10:16:17.892421 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24022955-daaa-4963-959b-6f1c67c9a658 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:16:17.894196 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24022955-daaa-4963-959b-6f1c67c9a658 admin admin] RBAC: Authorization granted {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:16:17.982628 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: [pid: 80377|app: 0|req: 869/1729] 10.222.0.82 () {58 vars in 1153 bytes} [Tue Aug 2 10:16:17 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 135 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 02 10:16:17.986790 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24022955-daaa-4963-959b-6f1c67c9a658 admin admin] Authenticating user token {{(pid=80377) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:16:18.012123 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24022955-daaa-4963-959b-6f1c67c9a658 admin admin] Validating token access rules against request {{(pid=80377) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:16:18.019655 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d95a6f11-102c-49ea-9cf1-23fe4c73c35b None None] Validating token access rules against request {{(pid=80378) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:16:18.028689 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b3c82a5-0d29-430a-9bd0-778541cfecb2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '891005fa5ff74041b75747d16cc66798', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7047912c5acd4adeba5d87c3bf895e39', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80377) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:16:18.030928 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b3c82a5-0d29-430a-9bd0-778541cfecb2 admin admin] REQUEST_METHOD: `POST` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:16:18.031050 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b3c82a5-0d29-430a-9bd0-778541cfecb2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:16:18.031246 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b3c82a5-0d29-430a-9bd0-778541cfecb2 admin admin] PATH_INFO: `/v3/projects` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:16:18.031940 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b3c82a5-0d29-430a-9bd0-778541cfecb2 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:16:18.034119 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bf7896d-f949-4215-9ea8-ce0c828e885b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '891005fa5ff74041b75747d16cc66798', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7047912c5acd4adeba5d87c3bf895e39', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80378) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:16:18.034756 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bf7896d-f949-4215-9ea8-ce0c828e885b admin admin] REQUEST_METHOD: `POST` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:16:18.035033 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bf7896d-f949-4215-9ea8-ce0c828e885b admin admin] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:16:18.035460 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bf7896d-f949-4215-9ea8-ce0c828e885b admin admin] PATH_INFO: `/v3/projects` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:16:18.036401 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1bf7896d-f949-4215-9ea8-ce0c828e885b admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:16:18.037613 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b3c82a5-0d29-430a-9bd0-778541cfecb2 admin admin] RBAC: Authorization granted {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:16:18.041462 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1bf7896d-f949-4215-9ea8-ce0c828e885b admin admin] RBAC: Authorization granted {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:16:18.123269 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: [pid: 80377|app: 0|req: 870/1730] 10.222.0.82 () {60 vars in 1153 bytes} [Tue Aug 2 10:16:17 2022] POST /identity/v3/projects => generated 375 bytes in 136 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 02 10:16:18.123501 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 861/1731] 10.222.0.82 () {60 vars in 1153 bytes} [Tue Aug 2 10:16:17 2022] POST /identity/v3/projects => generated 357 bytes in 319 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 02 10:16:18.125759 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bf7896d-f949-4215-9ea8-ce0c828e885b admin admin] Authenticating user token {{(pid=80378) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:16:18.126020 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b3c82a5-0d29-430a-9bd0-778541cfecb2 admin admin] Authenticating user token {{(pid=80377) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:16:18.257097 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG dbcounter [-] [80377] Writing DB stats keystone:SELECT=94,keystone:DELETE=4,keystone:INSERT=2 {{(pid=80377) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 10:16:18.261679 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bf7896d-f949-4215-9ea8-ce0c828e885b admin admin] Validating token access rules against request {{(pid=80378) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:16:18.272352 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea1b7595-94e5-416b-86ec-c3cb7a93d63b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '891005fa5ff74041b75747d16cc66798', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7047912c5acd4adeba5d87c3bf895e39', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80378) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:16:18.272988 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea1b7595-94e5-416b-86ec-c3cb7a93d63b admin admin] REQUEST_METHOD: `POST` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:16:18.273321 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea1b7595-94e5-416b-86ec-c3cb7a93d63b admin admin] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:16:18.273597 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea1b7595-94e5-416b-86ec-c3cb7a93d63b admin admin] PATH_INFO: `/v3/users` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:16:18.274405 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea1b7595-94e5-416b-86ec-c3cb7a93d63b admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:16:18.276629 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea1b7595-94e5-416b-86ec-c3cb7a93d63b admin admin] RBAC: Authorization granted {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:16:18.299654 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b3c82a5-0d29-430a-9bd0-778541cfecb2 admin admin] Validating token access rules against request {{(pid=80377) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:16:18.310000 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 862/1732] 10.222.0.82 () {60 vars in 1144 bytes} [Tue Aug 2 10:16:18 2022] POST /identity/v3/users => generated 288 bytes in 184 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 02 10:16:18.313765 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea1b7595-94e5-416b-86ec-c3cb7a93d63b admin admin] Authenticating user token {{(pid=80378) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:16:18.316232 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-661abb2f-eafa-4fb2-a4cd-463c5c0032ae admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '891005fa5ff74041b75747d16cc66798', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7047912c5acd4adeba5d87c3bf895e39', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80377) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:16:18.317120 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-661abb2f-eafa-4fb2-a4cd-463c5c0032ae admin admin] REQUEST_METHOD: `POST` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:16:18.318001 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-661abb2f-eafa-4fb2-a4cd-463c5c0032ae admin admin] SCRIPT_NAME: `/identity` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:16:18.320719 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-661abb2f-eafa-4fb2-a4cd-463c5c0032ae admin admin] PATH_INFO: `/v3/users` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:16:18.322058 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-661abb2f-eafa-4fb2-a4cd-463c5c0032ae admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:16:18.326958 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-661abb2f-eafa-4fb2-a4cd-463c5c0032ae admin admin] RBAC: Authorization granted {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:16:18.328777 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea1b7595-94e5-416b-86ec-c3cb7a93d63b admin admin] Validating token access rules against request {{(pid=80378) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:16:18.335122 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9d7a887-cae3-47c8-8637-94dcf3e64f57 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '891005fa5ff74041b75747d16cc66798', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7047912c5acd4adeba5d87c3bf895e39', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80378) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:16:18.335805 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9d7a887-cae3-47c8-8637-94dcf3e64f57 admin admin] REQUEST_METHOD: `GET` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:16:18.336085 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9d7a887-cae3-47c8-8637-94dcf3e64f57 admin admin] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:16:18.336357 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9d7a887-cae3-47c8-8637-94dcf3e64f57 admin admin] PATH_INFO: `/v3/roles` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:16:18.337142 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9d7a887-cae3-47c8-8637-94dcf3e64f57 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:16:18.339429 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9d7a887-cae3-47c8-8637-94dcf3e64f57 admin admin] RBAC: Authorization granted {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:16:18.359762 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 863/1733] 10.222.0.82 () {58 vars in 1122 bytes} [Tue Aug 2 10:16:18 2022] GET /identity/v3/roles => generated 1420 bytes in 46 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 02 10:16:18.366524 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9d7a887-cae3-47c8-8637-94dcf3e64f57 admin admin] Authenticating user token {{(pid=80378) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:16:18.370401 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: [pid: 80377|app: 0|req: 871/1734] 10.222.0.82 () {60 vars in 1144 bytes} [Tue Aug 2 10:16:18 2022] POST /identity/v3/users => generated 285 bytes in 246 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 02 10:16:18.376640 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-661abb2f-eafa-4fb2-a4cd-463c5c0032ae admin admin] Authenticating user token {{(pid=80377) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:16:18.380695 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9d7a887-cae3-47c8-8637-94dcf3e64f57 admin admin] Validating token access rules against request {{(pid=80378) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:16:18.386801 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66c8163b-8c18-4adb-96c9-ff89984e181c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '891005fa5ff74041b75747d16cc66798', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7047912c5acd4adeba5d87c3bf895e39', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80378) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:16:18.387192 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66c8163b-8c18-4adb-96c9-ff89984e181c admin admin] REQUEST_METHOD: `GET` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:16:18.388044 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66c8163b-8c18-4adb-96c9-ff89984e181c admin admin] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:16:18.388322 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66c8163b-8c18-4adb-96c9-ff89984e181c admin admin] PATH_INFO: `/v3/roles` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:16:18.389571 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66c8163b-8c18-4adb-96c9-ff89984e181c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:16:18.391757 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66c8163b-8c18-4adb-96c9-ff89984e181c admin admin] RBAC: Authorization granted {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:16:18.405053 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-661abb2f-eafa-4fb2-a4cd-463c5c0032ae admin admin] Validating token access rules against request {{(pid=80377) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:16:18.406075 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 864/1735] 10.222.0.82 () {58 vars in 1122 bytes} [Tue Aug 2 10:16:18 2022] GET /identity/v3/roles => generated 1420 bytes in 43 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 02 10:16:18.410544 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66c8163b-8c18-4adb-96c9-ff89984e181c admin admin] Authenticating user token {{(pid=80378) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:16:18.423764 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66c8163b-8c18-4adb-96c9-ff89984e181c admin admin] Validating token access rules against request {{(pid=80378) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:16:18.424789 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c259c6e-373e-46d8-ada6-993dea2b5a20 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '891005fa5ff74041b75747d16cc66798', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7047912c5acd4adeba5d87c3bf895e39', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80377) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:16:18.425681 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c259c6e-373e-46d8-ada6-993dea2b5a20 admin admin] REQUEST_METHOD: `GET` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:16:18.426443 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c259c6e-373e-46d8-ada6-993dea2b5a20 admin admin] SCRIPT_NAME: `/identity` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:16:18.427064 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c259c6e-373e-46d8-ada6-993dea2b5a20 admin admin] PATH_INFO: `/v3/roles` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:16:18.428362 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c259c6e-373e-46d8-ada6-993dea2b5a20 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:16:18.430101 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90c4b124-cde5-4619-9a27-95e0488f7ac1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '891005fa5ff74041b75747d16cc66798', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7047912c5acd4adeba5d87c3bf895e39', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80378) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:16:18.430978 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c259c6e-373e-46d8-ada6-993dea2b5a20 admin admin] RBAC: Authorization granted {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:16:18.431287 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90c4b124-cde5-4619-9a27-95e0488f7ac1 admin admin] REQUEST_METHOD: `PUT` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:16:18.431573 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90c4b124-cde5-4619-9a27-95e0488f7ac1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:16:18.432013 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90c4b124-cde5-4619-9a27-95e0488f7ac1 admin admin] PATH_INFO: `/v3/projects/321ebe0084074e3cab37facfacd53607/users/aabf3ada63b74bdb925096e197e6ecdd/roles/e1c04b19fc4543b8b54f58bbcffbc598` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:16:18.457708 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: [pid: 80377|app: 0|req: 872/1736] 10.222.0.82 () {58 vars in 1122 bytes} [Tue Aug 2 10:16:18 2022] GET /identity/v3/roles => generated 1420 bytes in 77 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 02 10:16:18.458016 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90c4b124-cde5-4619-9a27-95e0488f7ac1 admin admin] RBAC: Authorizing `identity:create_grant(project_id=321ebe0084074e3cab37facfacd53607, user_id=aabf3ada63b74bdb925096e197e6ecdd, role_id=e1c04b19fc4543b8b54f58bbcffbc598)` {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:16:18.458016 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90c4b124-cde5-4619-9a27-95e0488f7ac1 admin admin] RBAC: Authorization granted {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:16:18.460114 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c259c6e-373e-46d8-ada6-993dea2b5a20 admin admin] Authenticating user token {{(pid=80377) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:16:18.473070 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c259c6e-373e-46d8-ada6-993dea2b5a20 admin admin] Validating token access rules against request {{(pid=80377) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:16:18.484135 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 865/1737] 10.222.0.82 () {60 vars in 1483 bytes} [Tue Aug 2 10:16:18 2022] PUT /identity/v3/projects/321ebe0084074e3cab37facfacd53607/users/aabf3ada63b74bdb925096e197e6ecdd/roles/e1c04b19fc4543b8b54f58bbcffbc598 => generated 0 bytes in 65 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 02 10:16:18.488800 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4f39a21-a574-403b-b8c4-449af693847f None None] REQUEST_METHOD: `POST` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:16:18.489165 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4f39a21-a574-403b-b8c4-449af693847f None None] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:16:18.489348 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4f39a21-a574-403b-b8c4-449af693847f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:16:18.546707 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.auth.core [None req-f4f39a21-a574-403b-b8c4-449af693847f None None] MFA Rules not processed for user `aabf3ada63b74bdb925096e197e6ecdd`. Rule list: `[]` (Enabled: `True`). {{(pid=80378) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 02 10:16:18.549135 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.fernet_utils [None req-f4f39a21-a574-403b-b8c4-449af693847f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80378) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 10:16:18.613074 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b073b27-8182-443b-8508-87b4e7ed60bc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '891005fa5ff74041b75747d16cc66798', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7047912c5acd4adeba5d87c3bf895e39', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80377) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:16:18.614668 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b073b27-8182-443b-8508-87b4e7ed60bc admin admin] REQUEST_METHOD: `PUT` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:16:18.615054 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b073b27-8182-443b-8508-87b4e7ed60bc admin admin] SCRIPT_NAME: `/identity` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:16:18.615438 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b073b27-8182-443b-8508-87b4e7ed60bc admin admin] PATH_INFO: `/v3/projects/eecc188f6ae346b4aa105725a443d8c4/users/582a6da97d03445bad651118d5b215e2/roles/237966442f05411191105bf336bc14f3` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:16:18.627611 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG dbcounter [-] [80378] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80378) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 10:16:18.652277 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b073b27-8182-443b-8508-87b4e7ed60bc admin admin] RBAC: Authorizing `identity:create_grant(project_id=eecc188f6ae346b4aa105725a443d8c4, user_id=582a6da97d03445bad651118d5b215e2, role_id=237966442f05411191105bf336bc14f3)` {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:16:18.655754 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b073b27-8182-443b-8508-87b4e7ed60bc admin admin] RBAC: Authorization granted {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:16:18.676236 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: [pid: 80377|app: 0|req: 873/1738] 10.222.0.82 () {60 vars in 1483 bytes} [Tue Aug 2 10:16:18 2022] PUT /identity/v3/projects/eecc188f6ae346b4aa105725a443d8c4/users/582a6da97d03445bad651118d5b215e2/roles/237966442f05411191105bf336bc14f3 => generated 0 bytes in 217 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 02 10:16:18.678049 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b073b27-8182-443b-8508-87b4e7ed60bc admin admin] Authenticating user token {{(pid=80377) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:16:18.680528 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.fernet_utils [None req-1b073b27-8182-443b-8508-87b4e7ed60bc admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80377) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 10:16:18.783590 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b073b27-8182-443b-8508-87b4e7ed60bc admin admin] Validating token access rules against request {{(pid=80377) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:16:18.785969 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 866/1739] 10.222.0.82 () {58 vars in 958 bytes} [Tue Aug 2 10:16:18 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 298 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 02 10:16:18.788325 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4f39a21-a574-403b-b8c4-449af693847f None None] Authenticating user token {{(pid=80378) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:16:18.790233 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82a06a21-39d4-4af1-9839-e9a4c97cae9b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '891005fa5ff74041b75747d16cc66798', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7047912c5acd4adeba5d87c3bf895e39', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80377) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:16:18.791222 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82a06a21-39d4-4af1-9839-e9a4c97cae9b admin admin] REQUEST_METHOD: `DELETE` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:16:18.791718 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82a06a21-39d4-4af1-9839-e9a4c97cae9b admin admin] SCRIPT_NAME: `/identity` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:16:18.792218 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82a06a21-39d4-4af1-9839-e9a4c97cae9b admin admin] PATH_INFO: `/v3/projects/719a14aab1034164a033ddcb985695c1` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:16:18.794900 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82a06a21-39d4-4af1-9839-e9a4c97cae9b admin admin] RBAC: Authorizing `identity:delete_project(project_id=719a14aab1034164a033ddcb985695c1)` {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:16:18.798881 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82a06a21-39d4-4af1-9839-e9a4c97cae9b admin admin] RBAC: Authorization granted {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:16:18.818723 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4f39a21-a574-403b-b8c4-449af693847f None None] Validating token access rules against request {{(pid=80378) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:16:18.830135 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2385ac0-0736-4be7-be12-78d740b4c3be admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '891005fa5ff74041b75747d16cc66798', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7047912c5acd4adeba5d87c3bf895e39', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80378) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:16:18.830773 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2385ac0-0736-4be7-be12-78d740b4c3be admin admin] REQUEST_METHOD: `GET` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:16:18.831065 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2385ac0-0736-4be7-be12-78d740b4c3be admin admin] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:16:18.831342 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2385ac0-0736-4be7-be12-78d740b4c3be admin admin] PATH_INFO: `/v3/roles` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:16:18.832053 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2385ac0-0736-4be7-be12-78d740b4c3be admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:16:18.840756 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2385ac0-0736-4be7-be12-78d740b4c3be admin admin] RBAC: Authorization granted {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:16:18.865885 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG dbcounter [-] [80377] Writing DB stats keystone:SELECT=93,keystone:INSERT=4,keystone:DELETE=3 {{(pid=80377) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 10:16:18.877870 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 867/1740] 10.222.0.82 () {58 vars in 1122 bytes} [Tue Aug 2 10:16:18 2022] GET /identity/v3/roles => generated 1420 bytes in 90 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 02 10:16:18.878100 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.notifications [None req-82a06a21-39d4-4af1-9839-e9a4c97cae9b admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '719a14aab1034164a033ddcb985695c1', 'request_id': 'req-82a06a21-39d4-4af1-9839-e9a4c97cae9b'} {{(pid=80377) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 10:16:18.884456 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2385ac0-0736-4be7-be12-78d740b4c3be admin admin] Authenticating user token {{(pid=80378) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:16:18.899691 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.notifications [None req-82a06a21-39d4-4af1-9839-e9a4c97cae9b admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '719a14aab1034164a033ddcb985695c1', 'request_id': 'req-82a06a21-39d4-4af1-9839-e9a4c97cae9b'} {{(pid=80377) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 10:16:18.916474 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2385ac0-0736-4be7-be12-78d740b4c3be admin admin] Validating token access rules against request {{(pid=80378) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:16:18.921517 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.notifications [None req-82a06a21-39d4-4af1-9839-e9a4c97cae9b admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '719a14aab1034164a033ddcb985695c1', 'request_id': 'req-82a06a21-39d4-4af1-9839-e9a4c97cae9b'} {{(pid=80377) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 10:16:18.976683 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.notifications [None req-82a06a21-39d4-4af1-9839-e9a4c97cae9b admin admin] The token cache is being invalidate because project 719a14aab1034164a033ddcb985695c1 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80377) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 02 10:16:18.979458 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.notifications [None req-82a06a21-39d4-4af1-9839-e9a4c97cae9b admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80377) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 10:16:18.988436 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.notifications [None req-82a06a21-39d4-4af1-9839-e9a4c97cae9b admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80377) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 10:16:18.998210 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: [pid: 80377|app: 0|req: 874/1741] 10.222.0.82 () {58 vars in 1233 bytes} [Tue Aug 2 10:16:18 2022] DELETE /identity/v3/projects/719a14aab1034164a033ddcb985695c1 => generated 0 bytes in 321 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 02 10:16:19.007087 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82a06a21-39d4-4af1-9839-e9a4c97cae9b admin admin] Authenticating user token {{(pid=80377) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:16:19.012335 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.fernet_utils [None req-82a06a21-39d4-4af1-9839-e9a4c97cae9b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80377) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 10:16:19.131373 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a85c62f7-fef1-4b8d-82d7-467e9e3992a0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '891005fa5ff74041b75747d16cc66798', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7047912c5acd4adeba5d87c3bf895e39', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80378) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:16:19.132025 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a85c62f7-fef1-4b8d-82d7-467e9e3992a0 admin admin] REQUEST_METHOD: `POST` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:16:19.132314 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a85c62f7-fef1-4b8d-82d7-467e9e3992a0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:16:19.132594 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a85c62f7-fef1-4b8d-82d7-467e9e3992a0 admin admin] PATH_INFO: `/v3/projects` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:16:19.133295 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a85c62f7-fef1-4b8d-82d7-467e9e3992a0 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:16:19.138301 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a85c62f7-fef1-4b8d-82d7-467e9e3992a0 admin admin] RBAC: Authorization granted {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:16:19.178300 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82a06a21-39d4-4af1-9839-e9a4c97cae9b admin admin] Validating token access rules against request {{(pid=80377) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:16:19.196327 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-306b055c-9de1-4742-8e5b-8f8ac77edc21 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '891005fa5ff74041b75747d16cc66798', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7047912c5acd4adeba5d87c3bf895e39', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80377) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:16:19.197355 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-306b055c-9de1-4742-8e5b-8f8ac77edc21 admin admin] REQUEST_METHOD: `PUT` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:16:19.197857 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-306b055c-9de1-4742-8e5b-8f8ac77edc21 admin admin] SCRIPT_NAME: `/identity` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:16:19.201251 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-306b055c-9de1-4742-8e5b-8f8ac77edc21 admin admin] PATH_INFO: `/v3/projects/eecc188f6ae346b4aa105725a443d8c4/users/582a6da97d03445bad651118d5b215e2/roles/237966442f05411191105bf336bc14f3` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:16:19.206363 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-306b055c-9de1-4742-8e5b-8f8ac77edc21 admin admin] RBAC: Authorizing `identity:create_grant(project_id=eecc188f6ae346b4aa105725a443d8c4, user_id=582a6da97d03445bad651118d5b215e2, role_id=237966442f05411191105bf336bc14f3)` {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:16:19.209344 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-306b055c-9de1-4742-8e5b-8f8ac77edc21 admin admin] RBAC: Authorization granted {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:16:19.220387 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 868/1742] 10.222.0.82 () {60 vars in 1153 bytes} [Tue Aug 2 10:16:18 2022] POST /identity/v3/projects => generated 377 bytes in 340 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 02 10:16:19.224761 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a85c62f7-fef1-4b8d-82d7-467e9e3992a0 admin admin] Authenticating user token {{(pid=80378) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:16:19.230147 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.fernet_utils [None req-a85c62f7-fef1-4b8d-82d7-467e9e3992a0 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80378) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 10:16:19.237364 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: [pid: 80377|app: 0|req: 875/1743] 10.222.0.82 () {60 vars in 1483 bytes} [Tue Aug 2 10:16:18 2022] PUT /identity/v3/projects/eecc188f6ae346b4aa105725a443d8c4/users/582a6da97d03445bad651118d5b215e2/roles/237966442f05411191105bf336bc14f3 => generated 0 bytes in 238 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 02 10:16:19.238212 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-306b055c-9de1-4742-8e5b-8f8ac77edc21 admin admin] Authenticating user token {{(pid=80377) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:16:19.246245 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.fernet_utils [None req-306b055c-9de1-4742-8e5b-8f8ac77edc21 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80377) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 10:16:19.395552 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a85c62f7-fef1-4b8d-82d7-467e9e3992a0 admin admin] Validating token access rules against request {{(pid=80378) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:16:19.471034 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-306b055c-9de1-4742-8e5b-8f8ac77edc21 admin admin] Validating token access rules against request {{(pid=80377) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:16:19.485648 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e08d900-d5c3-412c-83f9-b6d5be633fa1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '891005fa5ff74041b75747d16cc66798', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7047912c5acd4adeba5d87c3bf895e39', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80377) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:16:19.487720 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e08d900-d5c3-412c-83f9-b6d5be633fa1 admin admin] REQUEST_METHOD: `POST` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:16:19.488003 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e08d900-d5c3-412c-83f9-b6d5be633fa1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:16:19.488293 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e08d900-d5c3-412c-83f9-b6d5be633fa1 admin admin] PATH_INFO: `/v3/users` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:16:19.489030 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e08d900-d5c3-412c-83f9-b6d5be633fa1 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:16:19.497475 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e08d900-d5c3-412c-83f9-b6d5be633fa1 admin admin] RBAC: Authorization granted {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:16:19.556016 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG dbcounter [-] [80378] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80378) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 10:16:19.587862 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: [pid: 80377|app: 0|req: 876/1744] 10.222.0.82 () {60 vars in 1144 bytes} [Tue Aug 2 10:16:19 2022] POST /identity/v3/users => generated 295 bytes in 350 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 02 10:16:19.590018 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6af7bc2-a42c-4979-a598-3815bf693d62 None None] REQUEST_METHOD: `POST` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:16:19.595288 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6af7bc2-a42c-4979-a598-3815bf693d62 None None] SCRIPT_NAME: `/identity` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:16:19.596364 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6af7bc2-a42c-4979-a598-3815bf693d62 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:16:19.609267 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10f30eff-e6db-491e-bd88-73be2c96e94d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '891005fa5ff74041b75747d16cc66798', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7047912c5acd4adeba5d87c3bf895e39', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80378) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:16:19.609987 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10f30eff-e6db-491e-bd88-73be2c96e94d admin admin] REQUEST_METHOD: `DELETE` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:16:19.611354 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10f30eff-e6db-491e-bd88-73be2c96e94d admin admin] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:16:19.611644 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10f30eff-e6db-491e-bd88-73be2c96e94d admin admin] PATH_INFO: `/v3/users/4d76a9dc026047038d14dadedb3df1fb` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:16:19.618095 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10f30eff-e6db-491e-bd88-73be2c96e94d admin admin] RBAC: Authorizing `identity:delete_user(user_id=4d76a9dc026047038d14dadedb3df1fb)` {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:16:19.624942 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10f30eff-e6db-491e-bd88-73be2c96e94d admin admin] RBAC: Authorization granted {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:16:19.678778 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.auth.core [None req-d6af7bc2-a42c-4979-a598-3815bf693d62 None None] MFA Rules not processed for user `582a6da97d03445bad651118d5b215e2`. Rule list: `[]` (Enabled: `True`). {{(pid=80377) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 02 10:16:19.682850 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.fernet_utils [None req-d6af7bc2-a42c-4979-a598-3815bf693d62 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80377) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 10:16:19.742258 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.notifications [None req-10f30eff-e6db-491e-bd88-73be2c96e94d admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '4d76a9dc026047038d14dadedb3df1fb'} {{(pid=80378) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 10:16:19.763803 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.notifications [None req-10f30eff-e6db-491e-bd88-73be2c96e94d admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '4d76a9dc026047038d14dadedb3df1fb'} {{(pid=80378) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 10:16:19.794714 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.notifications [None req-10f30eff-e6db-491e-bd88-73be2c96e94d admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '4d76a9dc026047038d14dadedb3df1fb'} {{(pid=80378) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 10:16:19.799700 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.notifications [None req-10f30eff-e6db-491e-bd88-73be2c96e94d admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '4d76a9dc026047038d14dadedb3df1fb'} {{(pid=80378) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 10:16:19.832209 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG dbcounter [-] [80377] Writing DB stats keystone:SELECT=91,keystone:DELETE=4,keystone:INSERT=5 {{(pid=80377) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 10:16:19.846660 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.notifications [None req-10f30eff-e6db-491e-bd88-73be2c96e94d admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '4d76a9dc026047038d14dadedb3df1fb'} {{(pid=80378) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 10:16:19.878287 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 869/1745] 10.222.0.82 () {58 vars in 1224 bytes} [Tue Aug 2 10:16:19 2022] DELETE /identity/v3/users/4d76a9dc026047038d14dadedb3df1fb => generated 0 bytes in 654 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 02 10:16:19.880263 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10f30eff-e6db-491e-bd88-73be2c96e94d admin admin] Authenticating user token {{(pid=80378) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:16:19.885049 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.fernet_utils [None req-10f30eff-e6db-491e-bd88-73be2c96e94d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80378) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 10:16:19.911507 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: [pid: 80377|app: 0|req: 877/1746] 10.222.0.82 () {58 vars in 958 bytes} [Tue Aug 2 10:16:19 2022] POST /identity/v3/auth/tokens => generated 3301 bytes in 322 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 02 10:16:19.924871 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6af7bc2-a42c-4979-a598-3815bf693d62 None None] Authenticating user token {{(pid=80377) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:16:19.928306 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.fernet_utils [None req-d6af7bc2-a42c-4979-a598-3815bf693d62 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80377) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 10:16:20.051911 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10f30eff-e6db-491e-bd88-73be2c96e94d admin admin] Validating token access rules against request {{(pid=80378) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:16:20.069945 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c037c0da-38dc-4cdf-a2e9-d424ecada543 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '891005fa5ff74041b75747d16cc66798', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7047912c5acd4adeba5d87c3bf895e39', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80378) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:16:20.076661 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c037c0da-38dc-4cdf-a2e9-d424ecada543 admin admin] REQUEST_METHOD: `GET` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:16:20.076952 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c037c0da-38dc-4cdf-a2e9-d424ecada543 admin admin] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:16:20.077232 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c037c0da-38dc-4cdf-a2e9-d424ecada543 admin admin] PATH_INFO: `/v3/roles` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:16:20.077985 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c037c0da-38dc-4cdf-a2e9-d424ecada543 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:16:20.079826 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c037c0da-38dc-4cdf-a2e9-d424ecada543 admin admin] RBAC: Authorization granted {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:16:20.109417 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6af7bc2-a42c-4979-a598-3815bf693d62 None None] Validating token access rules against request {{(pid=80377) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:16:20.124932 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 870/1747] 10.222.0.82 () {58 vars in 1122 bytes} [Tue Aug 2 10:16:19 2022] GET /identity/v3/roles => generated 1420 bytes in 246 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 02 10:16:20.125584 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c037c0da-38dc-4cdf-a2e9-d424ecada543 admin admin] Authenticating user token {{(pid=80378) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:16:20.168241 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c037c0da-38dc-4cdf-a2e9-d424ecada543 admin admin] Validating token access rules against request {{(pid=80378) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:16:20.180967 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-577fa254-e4af-4089-bfa0-f04969b97555 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '891005fa5ff74041b75747d16cc66798', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7047912c5acd4adeba5d87c3bf895e39', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80378) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:16:20.181822 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-577fa254-e4af-4089-bfa0-f04969b97555 admin admin] REQUEST_METHOD: `PUT` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:16:20.182085 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-577fa254-e4af-4089-bfa0-f04969b97555 admin admin] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:16:20.182367 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-577fa254-e4af-4089-bfa0-f04969b97555 admin admin] PATH_INFO: `/v3/projects/f46d6a6eea6c4f318667ca8f93f032e5/users/6edab13f3c094507ba5857e05b8f6467/roles/237966442f05411191105bf336bc14f3` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:16:20.256446 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-577fa254-e4af-4089-bfa0-f04969b97555 admin admin] RBAC: Authorizing `identity:create_grant(project_id=f46d6a6eea6c4f318667ca8f93f032e5, user_id=6edab13f3c094507ba5857e05b8f6467, role_id=237966442f05411191105bf336bc14f3)` {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:16:20.260071 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-577fa254-e4af-4089-bfa0-f04969b97555 admin admin] RBAC: Authorization granted {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:16:20.272208 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5291e99f-113b-4f4e-a2a8-2fe972450376 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1023de281ae74e9d8a6d6c312c1c97a7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f1b38ebb804c41c69772c998aae72c41', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80377) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:16:20.273120 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5291e99f-113b-4f4e-a2a8-2fe972450376 service cinder] REQUEST_METHOD: `GET` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:16:20.273521 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5291e99f-113b-4f4e-a2a8-2fe972450376 service cinder] SCRIPT_NAME: `/identity` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:16:20.273998 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5291e99f-113b-4f4e-a2a8-2fe972450376 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:16:20.294717 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 871/1748] 10.222.0.82 () {60 vars in 1483 bytes} [Tue Aug 2 10:16:20 2022] PUT /identity/v3/projects/f46d6a6eea6c4f318667ca8f93f032e5/users/6edab13f3c094507ba5857e05b8f6467/roles/237966442f05411191105bf336bc14f3 => generated 0 bytes in 170 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 02 10:16:20.298975 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-577fa254-e4af-4089-bfa0-f04969b97555 admin admin] Authenticating user token {{(pid=80378) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:16:20.412019 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG dbcounter [-] [80378] Writing DB stats keystone:SELECT=88,keystone:DELETE=10,keystone:INSERT=2 {{(pid=80378) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 10:16:20.494485 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-577fa254-e4af-4089-bfa0-f04969b97555 admin admin] Validating token access rules against request {{(pid=80378) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:16:20.498710 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5291e99f-113b-4f4e-a2a8-2fe972450376 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:16:20.510812 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5291e99f-113b-4f4e-a2a8-2fe972450376 service cinder] RBAC: Authorization granted {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:16:20.534902 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d7a34e8-3505-4f1a-8877-279c780de079 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '891005fa5ff74041b75747d16cc66798', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7047912c5acd4adeba5d87c3bf895e39', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80378) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:16:20.535560 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d7a34e8-3505-4f1a-8877-279c780de079 admin admin] REQUEST_METHOD: `GET` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:16:20.535844 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d7a34e8-3505-4f1a-8877-279c780de079 admin admin] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:16:20.536120 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d7a34e8-3505-4f1a-8877-279c780de079 admin admin] PATH_INFO: `/v3/roles` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:16:20.536827 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d7a34e8-3505-4f1a-8877-279c780de079 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:16:20.539010 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d7a34e8-3505-4f1a-8877-279c780de079 admin admin] RBAC: Authorization granted {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:16:20.560832 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: [pid: 80377|app: 0|req: 878/1749] 10.222.0.82 () {60 vars in 1365 bytes} [Tue Aug 2 10:16:19 2022] GET /identity/v3/auth/tokens => generated 3301 bytes in 636 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 02 10:16:20.567426 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6a89616-e86e-4783-95da-72a20007a586 None None] REQUEST_METHOD: `POST` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:16:20.571487 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6a89616-e86e-4783-95da-72a20007a586 None None] SCRIPT_NAME: `/identity` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:16:20.571487 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6a89616-e86e-4783-95da-72a20007a586 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:16:20.607799 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 872/1750] 10.222.0.82 () {58 vars in 1122 bytes} [Tue Aug 2 10:16:20 2022] GET /identity/v3/roles => generated 1420 bytes in 309 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 02 10:16:20.610536 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d7a34e8-3505-4f1a-8877-279c780de079 admin admin] Authenticating user token {{(pid=80378) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:16:20.647891 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d7a34e8-3505-4f1a-8877-279c780de079 admin admin] Validating token access rules against request {{(pid=80378) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:16:20.664082 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01cc8950-480f-4de5-9332-44325216479c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '891005fa5ff74041b75747d16cc66798', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7047912c5acd4adeba5d87c3bf895e39', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80378) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:16:20.665037 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01cc8950-480f-4de5-9332-44325216479c admin admin] REQUEST_METHOD: `PUT` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:16:20.665433 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01cc8950-480f-4de5-9332-44325216479c admin admin] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:16:20.665849 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01cc8950-480f-4de5-9332-44325216479c admin admin] PATH_INFO: `/v3/projects/f46d6a6eea6c4f318667ca8f93f032e5/users/6edab13f3c094507ba5857e05b8f6467/roles/237966442f05411191105bf336bc14f3` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:16:20.668219 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01cc8950-480f-4de5-9332-44325216479c admin admin] RBAC: Authorizing `identity:create_grant(project_id=f46d6a6eea6c4f318667ca8f93f032e5, user_id=6edab13f3c094507ba5857e05b8f6467, role_id=237966442f05411191105bf336bc14f3)` {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:16:20.682021 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.auth.core [None req-f6a89616-e86e-4783-95da-72a20007a586 None None] MFA Rules not processed for user `b6890a8127e04092920cb1d31ab56d91`. Rule list: `[]` (Enabled: `True`). {{(pid=80377) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 02 10:16:20.686834 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.fernet_utils [None req-f6a89616-e86e-4783-95da-72a20007a586 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80377) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 10:16:20.687157 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01cc8950-480f-4de5-9332-44325216479c admin admin] RBAC: Authorization granted {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:16:20.723757 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 873/1751] 10.222.0.82 () {60 vars in 1483 bytes} [Tue Aug 2 10:16:20 2022] PUT /identity/v3/projects/f46d6a6eea6c4f318667ca8f93f032e5/users/6edab13f3c094507ba5857e05b8f6467/roles/237966442f05411191105bf336bc14f3 => generated 0 bytes in 114 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 02 10:16:20.734866 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2cd5c87-a576-4f3f-8e68-0c8e4fe142e7 None None] REQUEST_METHOD: `POST` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:16:20.735417 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2cd5c87-a576-4f3f-8e68-0c8e4fe142e7 None None] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:16:20.735972 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2cd5c87-a576-4f3f-8e68-0c8e4fe142e7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:16:20.785851 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG dbcounter [-] [80377] Writing DB stats keystone:SELECT=100 {{(pid=80377) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 10:16:20.820637 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.auth.core [None req-c2cd5c87-a576-4f3f-8e68-0c8e4fe142e7 None None] MFA Rules not processed for user `6edab13f3c094507ba5857e05b8f6467`. Rule list: `[]` (Enabled: `True`). {{(pid=80378) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 02 10:16:20.821705 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.fernet_utils [None req-c2cd5c87-a576-4f3f-8e68-0c8e4fe142e7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80378) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 10:16:20.917988 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: [pid: 80377|app: 0|req: 879/1752] 10.222.0.82 () {58 vars in 958 bytes} [Tue Aug 2 10:16:20 2022] POST /identity/v3/auth/tokens => generated 3266 bytes in 357 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 02 10:16:20.919735 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6a89616-e86e-4783-95da-72a20007a586 None None] Authenticating user token {{(pid=80377) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:16:21.049868 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 874/1753] 10.222.0.82 () {58 vars in 958 bytes} [Tue Aug 2 10:16:20 2022] POST /identity/v3/auth/tokens => generated 3321 bytes in 316 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 02 10:16:21.051145 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2cd5c87-a576-4f3f-8e68-0c8e4fe142e7 None None] Authenticating user token {{(pid=80378) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:16:21.062349 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.fernet_utils [None req-c2cd5c87-a576-4f3f-8e68-0c8e4fe142e7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80378) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 10:16:21.097772 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6a89616-e86e-4783-95da-72a20007a586 None None] Validating token access rules against request {{(pid=80377) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:16:21.180848 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f784e4c1-523d-4d65-96df-c5bf54bb4529 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1023de281ae74e9d8a6d6c312c1c97a7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f1b38ebb804c41c69772c998aae72c41', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80377) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:16:21.181478 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f784e4c1-523d-4d65-96df-c5bf54bb4529 service cinder] REQUEST_METHOD: `GET` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:16:21.181763 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f784e4c1-523d-4d65-96df-c5bf54bb4529 service cinder] SCRIPT_NAME: `/identity` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:16:21.182076 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f784e4c1-523d-4d65-96df-c5bf54bb4529 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:16:21.184113 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.fernet_utils [None req-f784e4c1-523d-4d65-96df-c5bf54bb4529 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80377) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 10:16:21.230721 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2cd5c87-a576-4f3f-8e68-0c8e4fe142e7 None None] Validating token access rules against request {{(pid=80378) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:16:21.240472 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb3fd7e9-fd87-4b99-b073-1ba59c0b4500 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88af518196674fe593ad88d30111fb7d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f1b38ebb804c41c69772c998aae72c41', 'project_domain_id': 'default', 'roles': ['admin', 'ResellerAdmin', 'reader', 'service', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80378) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:16:21.243113 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb3fd7e9-fd87-4b99-b073-1ba59c0b4500 service nova] REQUEST_METHOD: `GET` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:16:21.243562 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb3fd7e9-fd87-4b99-b073-1ba59c0b4500 service nova] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:16:21.244259 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb3fd7e9-fd87-4b99-b073-1ba59c0b4500 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:16:21.261664 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f784e4c1-523d-4d65-96df-c5bf54bb4529 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:16:21.264312 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f784e4c1-523d-4d65-96df-c5bf54bb4529 service cinder] RBAC: Authorization granted {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:16:21.280214 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: [pid: 80377|app: 0|req: 880/1754] 10.222.0.82 () {60 vars in 1365 bytes} [Tue Aug 2 10:16:20 2022] GET /identity/v3/auth/tokens => generated 3234 bytes in 362 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 02 10:16:21.281351 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f784e4c1-523d-4d65-96df-c5bf54bb4529 service cinder] Authenticating user token {{(pid=80377) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:16:21.283184 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.fernet_utils [None req-f784e4c1-523d-4d65-96df-c5bf54bb4529 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80377) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 10:16:21.299382 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f784e4c1-523d-4d65-96df-c5bf54bb4529 service cinder] Validating token access rules against request {{(pid=80377) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:16:21.307648 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9687c70b-2cec-4647-b68e-ace7b7606462 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1023de281ae74e9d8a6d6c312c1c97a7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f1b38ebb804c41c69772c998aae72c41', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80377) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:16:21.308807 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9687c70b-2cec-4647-b68e-ace7b7606462 service cinder] REQUEST_METHOD: `GET` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:16:21.309930 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9687c70b-2cec-4647-b68e-ace7b7606462 service cinder] SCRIPT_NAME: `/identity` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:16:21.310640 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9687c70b-2cec-4647-b68e-ace7b7606462 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:16:21.318839 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG dbcounter [-] [80378] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80378) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 10:16:21.330673 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9687c70b-2cec-4647-b68e-ace7b7606462 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:16:21.332605 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9687c70b-2cec-4647-b68e-ace7b7606462 service cinder] RBAC: Authorization granted {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:16:21.341235 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb3fd7e9-fd87-4b99-b073-1ba59c0b4500 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:16:21.343094 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb3fd7e9-fd87-4b99-b073-1ba59c0b4500 service nova] RBAC: Authorization granted {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:16:21.354814 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 875/1755] 10.222.0.82 () {60 vars in 1365 bytes} [Tue Aug 2 10:16:21 2022] GET /identity/v3/auth/tokens => generated 3266 bytes in 304 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 02 10:16:21.355272 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb3fd7e9-fd87-4b99-b073-1ba59c0b4500 service nova] Authenticating user token {{(pid=80378) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:16:21.358210 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.fernet_utils [None req-cb3fd7e9-fd87-4b99-b073-1ba59c0b4500 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80378) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 10:16:21.364542 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: [pid: 80377|app: 0|req: 881/1756] 10.222.0.82 () {60 vars in 1365 bytes} [Tue Aug 2 10:16:21 2022] GET /identity/v3/auth/tokens => generated 3321 bytes in 83 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 02 10:16:21.488947 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9687c70b-2cec-4647-b68e-ace7b7606462 service cinder] Authenticating user token {{(pid=80377) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:16:21.508033 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb3fd7e9-fd87-4b99-b073-1ba59c0b4500 service nova] Validating token access rules against request {{(pid=80378) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:16:21.509430 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9687c70b-2cec-4647-b68e-ace7b7606462 service cinder] Validating token access rules against request {{(pid=80377) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:16:21.518583 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1dacbc6-9655-406f-a449-f156378d43b4 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1023de281ae74e9d8a6d6c312c1c97a7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f1b38ebb804c41c69772c998aae72c41', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80377) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:16:21.520942 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68f8375a-7083-48a8-99a4-8508605bfa59 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '891005fa5ff74041b75747d16cc66798', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7047912c5acd4adeba5d87c3bf895e39', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80378) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:16:21.521482 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68f8375a-7083-48a8-99a4-8508605bfa59 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:16:21.521586 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1dacbc6-9655-406f-a449-f156378d43b4 service cinder] REQUEST_METHOD: `GET` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:16:21.521842 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68f8375a-7083-48a8-99a4-8508605bfa59 admin admin] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:16:21.522158 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68f8375a-7083-48a8-99a4-8508605bfa59 admin admin] PATH_INFO: `/v3/projects/df2bc25ad42743939bd44e1f31049208` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:16:21.522295 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1dacbc6-9655-406f-a449-f156378d43b4 service cinder] SCRIPT_NAME: `/identity` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:16:21.523063 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1dacbc6-9655-406f-a449-f156378d43b4 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:16:21.523464 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68f8375a-7083-48a8-99a4-8508605bfa59 admin admin] RBAC: Authorizing `identity:delete_project(project_id=df2bc25ad42743939bd44e1f31049208)` {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:16:21.525444 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68f8375a-7083-48a8-99a4-8508605bfa59 admin admin] RBAC: Authorization granted {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:16:21.531111 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.fernet_utils [None req-d1dacbc6-9655-406f-a449-f156378d43b4 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80377) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 10:16:21.644993 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.notifications [None req-68f8375a-7083-48a8-99a4-8508605bfa59 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'df2bc25ad42743939bd44e1f31049208', 'request_id': 'req-68f8375a-7083-48a8-99a4-8508605bfa59'} {{(pid=80378) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 10:16:21.645962 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG dbcounter [-] [80377] Writing DB stats keystone:SELECT=100 {{(pid=80377) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 10:16:21.665981 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.notifications [None req-68f8375a-7083-48a8-99a4-8508605bfa59 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'df2bc25ad42743939bd44e1f31049208', 'request_id': 'req-68f8375a-7083-48a8-99a4-8508605bfa59'} {{(pid=80378) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 10:16:21.676202 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.notifications [None req-68f8375a-7083-48a8-99a4-8508605bfa59 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'df2bc25ad42743939bd44e1f31049208', 'request_id': 'req-68f8375a-7083-48a8-99a4-8508605bfa59'} {{(pid=80378) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 10:16:21.704302 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.notifications [None req-68f8375a-7083-48a8-99a4-8508605bfa59 admin admin] The token cache is being invalidate because project df2bc25ad42743939bd44e1f31049208 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80378) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 02 10:16:21.704800 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.notifications [None req-68f8375a-7083-48a8-99a4-8508605bfa59 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80378) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 10:16:21.706450 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.notifications [None req-68f8375a-7083-48a8-99a4-8508605bfa59 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80378) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 10:16:21.709602 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 876/1757] 10.222.0.82 () {58 vars in 1233 bytes} [Tue Aug 2 10:16:21 2022] DELETE /identity/v3/projects/df2bc25ad42743939bd44e1f31049208 => generated 0 bytes in 354 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 02 10:16:21.714440 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a15f952f-2518-4d7e-8eb8-c56ba3adc426 None None] REQUEST_METHOD: `POST` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:16:21.714798 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a15f952f-2518-4d7e-8eb8-c56ba3adc426 None None] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:16:21.715137 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a15f952f-2518-4d7e-8eb8-c56ba3adc426 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:16:21.719834 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1dacbc6-9655-406f-a449-f156378d43b4 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:16:21.723317 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1dacbc6-9655-406f-a449-f156378d43b4 service cinder] RBAC: Authorization granted {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:16:21.753490 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: [pid: 80377|app: 0|req: 882/1758] 10.222.0.82 () {60 vars in 1365 bytes} [Tue Aug 2 10:16:21 2022] GET /identity/v3/auth/tokens => generated 3252 bytes in 265 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 02 10:16:21.777575 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.auth.core [None req-a15f952f-2518-4d7e-8eb8-c56ba3adc426 None None] MFA Rules not processed for user `891005fa5ff74041b75747d16cc66798`. Rule list: `[]` (Enabled: `True`). {{(pid=80378) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 02 10:16:21.785903 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.fernet_utils [None req-a15f952f-2518-4d7e-8eb8-c56ba3adc426 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80378) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 10:16:21.869917 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2ddc1d8-fd05-4f0c-9cf4-685c6df231d5 None None] REQUEST_METHOD: `POST` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:16:21.870419 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2ddc1d8-fd05-4f0c-9cf4-685c6df231d5 None None] SCRIPT_NAME: `/identity` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:16:21.870919 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2ddc1d8-fd05-4f0c-9cf4-685c6df231d5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:16:21.944327 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.auth.core [None req-d2ddc1d8-fd05-4f0c-9cf4-685c6df231d5 None None] MFA Rules not processed for user `e881ae7c60254c12860fff0a40480641`. Rule list: `[]` (Enabled: `True`). {{(pid=80377) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 02 10:16:21.954884 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.fernet_utils [None req-d2ddc1d8-fd05-4f0c-9cf4-685c6df231d5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80377) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 10:16:21.997611 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 877/1759] 10.222.0.82 () {58 vars in 958 bytes} [Tue Aug 2 10:16:21 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 284 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 02 10:16:21.999753 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6937ca5f-2307-4d75-81cb-7fc68d927e1e None None] REQUEST_METHOD: `POST` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:16:22.000160 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6937ca5f-2307-4d75-81cb-7fc68d927e1e None None] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:16:22.000443 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6937ca5f-2307-4d75-81cb-7fc68d927e1e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:16:22.025118 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.auth.core [None req-6937ca5f-2307-4d75-81cb-7fc68d927e1e None None] MFA Rules not processed for user `891005fa5ff74041b75747d16cc66798`. Rule list: `[]` (Enabled: `True`). {{(pid=80378) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 02 10:16:22.027071 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.fernet_utils [None req-6937ca5f-2307-4d75-81cb-7fc68d927e1e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80378) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 10:16:22.040972 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 878/1760] 10.222.0.82 () {58 vars in 958 bytes} [Tue Aug 2 10:16:21 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 43 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 02 10:16:22.042036 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6937ca5f-2307-4d75-81cb-7fc68d927e1e None None] Authenticating user token {{(pid=80378) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:16:22.051702 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG dbcounter [-] [80378] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80378) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 10:16:22.059324 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6937ca5f-2307-4d75-81cb-7fc68d927e1e None None] Validating token access rules against request {{(pid=80378) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:16:22.071932 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e92e204-e241-4574-bc4e-485819d46443 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '891005fa5ff74041b75747d16cc66798', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7047912c5acd4adeba5d87c3bf895e39', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80378) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:16:22.072676 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e92e204-e241-4574-bc4e-485819d46443 admin admin] REQUEST_METHOD: `GET` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:16:22.073398 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e92e204-e241-4574-bc4e-485819d46443 admin admin] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:16:22.073690 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e92e204-e241-4574-bc4e-485819d46443 admin admin] PATH_INFO: `/v3/domains` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:16:22.076060 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e92e204-e241-4574-bc4e-485819d46443 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80378) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 02 10:16:22.076699 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e92e204-e241-4574-bc4e-485819d46443 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:16:22.079775 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e92e204-e241-4574-bc4e-485819d46443 admin admin] RBAC: Authorization granted {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:16:22.103755 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 879/1761] 10.222.0.82 () {58 vars in 1153 bytes} [Tue Aug 2 10:16:22 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 62 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 02 10:16:22.104696 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e92e204-e241-4574-bc4e-485819d46443 admin admin] Authenticating user token {{(pid=80378) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:16:22.121335 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e92e204-e241-4574-bc4e-485819d46443 admin admin] Validating token access rules against request {{(pid=80378) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:16:22.127750 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04e7db3c-74b5-4a52-a6f5-844565a4b839 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '891005fa5ff74041b75747d16cc66798', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7047912c5acd4adeba5d87c3bf895e39', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80378) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:16:22.128609 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04e7db3c-74b5-4a52-a6f5-844565a4b839 admin admin] REQUEST_METHOD: `POST` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:16:22.128886 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04e7db3c-74b5-4a52-a6f5-844565a4b839 admin admin] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:16:22.129220 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04e7db3c-74b5-4a52-a6f5-844565a4b839 admin admin] PATH_INFO: `/v3/projects` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:16:22.132038 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04e7db3c-74b5-4a52-a6f5-844565a4b839 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:16:22.135065 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04e7db3c-74b5-4a52-a6f5-844565a4b839 admin admin] RBAC: Authorization granted {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:16:22.191579 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 880/1762] 10.222.0.82 () {60 vars in 1153 bytes} [Tue Aug 2 10:16:22 2022] POST /identity/v3/projects => generated 375 bytes in 87 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 02 10:16:22.192866 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04e7db3c-74b5-4a52-a6f5-844565a4b839 admin admin] Authenticating user token {{(pid=80378) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:16:22.212094 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: [pid: 80377|app: 0|req: 883/1763] 10.222.0.82 () {58 vars in 958 bytes} [Tue Aug 2 10:16:21 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 344 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 02 10:16:22.328077 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04e7db3c-74b5-4a52-a6f5-844565a4b839 admin admin] Validating token access rules against request {{(pid=80378) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:16:22.340036 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7657375-02e8-4609-905e-72e31b2c47cd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '891005fa5ff74041b75747d16cc66798', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7047912c5acd4adeba5d87c3bf895e39', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80378) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:16:22.340731 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7657375-02e8-4609-905e-72e31b2c47cd admin admin] REQUEST_METHOD: `POST` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:16:22.341018 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7657375-02e8-4609-905e-72e31b2c47cd admin admin] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:16:22.342918 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7657375-02e8-4609-905e-72e31b2c47cd admin admin] PATH_INFO: `/v3/users` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:16:22.343645 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7657375-02e8-4609-905e-72e31b2c47cd admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:16:22.346689 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7657375-02e8-4609-905e-72e31b2c47cd admin admin] RBAC: Authorization granted {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:16:22.398779 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 881/1764] 10.222.0.82 () {60 vars in 1144 bytes} [Tue Aug 2 10:16:22 2022] POST /identity/v3/users => generated 288 bytes in 206 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 02 10:16:22.404248 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2ddc1d8-fd05-4f0c-9cf4-685c6df231d5 None None] Authenticating user token {{(pid=80377) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:16:22.417221 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2ddc1d8-fd05-4f0c-9cf4-685c6df231d5 None None] Validating token access rules against request {{(pid=80377) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:16:22.436171 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa1b469a-bca4-44c3-b380-ef028b24dc83 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '891005fa5ff74041b75747d16cc66798', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7047912c5acd4adeba5d87c3bf895e39', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80377) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:16:22.436826 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa1b469a-bca4-44c3-b380-ef028b24dc83 admin admin] REQUEST_METHOD: `GET` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:16:22.437113 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa1b469a-bca4-44c3-b380-ef028b24dc83 admin admin] SCRIPT_NAME: `/identity` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:16:22.437389 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa1b469a-bca4-44c3-b380-ef028b24dc83 admin admin] PATH_INFO: `/v3/roles` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:16:22.438161 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa1b469a-bca4-44c3-b380-ef028b24dc83 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:16:22.439785 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa1b469a-bca4-44c3-b380-ef028b24dc83 admin admin] RBAC: Authorization granted {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:16:22.455079 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: [pid: 80377|app: 0|req: 884/1765] 10.222.0.82 () {58 vars in 1122 bytes} [Tue Aug 2 10:16:22 2022] GET /identity/v3/roles => generated 1420 bytes in 51 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 02 10:16:22.459141 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7657375-02e8-4609-905e-72e31b2c47cd admin admin] Authenticating user token {{(pid=80378) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:16:22.489195 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7657375-02e8-4609-905e-72e31b2c47cd admin admin] Validating token access rules against request {{(pid=80378) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:16:22.502131 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b906fcc-8aaf-4f27-a87a-a2e82bf90420 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '891005fa5ff74041b75747d16cc66798', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7047912c5acd4adeba5d87c3bf895e39', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80378) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:16:22.502921 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b906fcc-8aaf-4f27-a87a-a2e82bf90420 admin admin] REQUEST_METHOD: `GET` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:16:22.507442 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b906fcc-8aaf-4f27-a87a-a2e82bf90420 admin admin] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:16:22.507923 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b906fcc-8aaf-4f27-a87a-a2e82bf90420 admin admin] PATH_INFO: `/v3/roles` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:16:22.509083 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b906fcc-8aaf-4f27-a87a-a2e82bf90420 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:16:22.511353 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b906fcc-8aaf-4f27-a87a-a2e82bf90420 admin admin] RBAC: Authorization granted {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:16:22.525635 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 882/1766] 10.222.0.82 () {58 vars in 1122 bytes} [Tue Aug 2 10:16:22 2022] GET /identity/v3/roles => generated 1420 bytes in 67 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 02 10:16:22.529731 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa1b469a-bca4-44c3-b380-ef028b24dc83 admin admin] Authenticating user token {{(pid=80377) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:16:22.545500 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa1b469a-bca4-44c3-b380-ef028b24dc83 admin admin] Validating token access rules against request {{(pid=80377) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:16:22.557998 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df33063d-a236-4e39-8067-a63c08b3b5d4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '891005fa5ff74041b75747d16cc66798', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7047912c5acd4adeba5d87c3bf895e39', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80377) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:16:22.558486 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df33063d-a236-4e39-8067-a63c08b3b5d4 admin admin] REQUEST_METHOD: `PUT` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:16:22.558789 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df33063d-a236-4e39-8067-a63c08b3b5d4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:16:22.559055 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df33063d-a236-4e39-8067-a63c08b3b5d4 admin admin] PATH_INFO: `/v3/projects/08cc037744b14429898b57ad3fe90d0b/users/5f848c9acf0a425c935bf996ae77df93/roles/e1c04b19fc4543b8b54f58bbcffbc598` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:16:22.610578 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df33063d-a236-4e39-8067-a63c08b3b5d4 admin admin] RBAC: Authorizing `identity:create_grant(project_id=08cc037744b14429898b57ad3fe90d0b, user_id=5f848c9acf0a425c935bf996ae77df93, role_id=e1c04b19fc4543b8b54f58bbcffbc598)` {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:16:22.613639 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df33063d-a236-4e39-8067-a63c08b3b5d4 admin admin] RBAC: Authorization granted {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:16:22.661270 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: [pid: 80377|app: 0|req: 885/1767] 10.222.0.82 () {60 vars in 1483 bytes} [Tue Aug 2 10:16:22 2022] PUT /identity/v3/projects/08cc037744b14429898b57ad3fe90d0b/users/5f848c9acf0a425c935bf996ae77df93/roles/e1c04b19fc4543b8b54f58bbcffbc598 => generated 0 bytes in 132 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 02 10:16:22.676457 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6788da67-1b59-4945-9945-15537fa70b65 None None] REQUEST_METHOD: `POST` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:16:22.676989 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6788da67-1b59-4945-9945-15537fa70b65 None None] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:16:22.677203 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6788da67-1b59-4945-9945-15537fa70b65 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:16:22.802029 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.auth.core [None req-6788da67-1b59-4945-9945-15537fa70b65 None None] MFA Rules not processed for user `5f848c9acf0a425c935bf996ae77df93`. Rule list: `[]` (Enabled: `True`). {{(pid=80378) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 02 10:16:22.808709 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.fernet_utils [None req-6788da67-1b59-4945-9945-15537fa70b65 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80378) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 10:16:23.043748 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 883/1768] 10.222.0.82 () {58 vars in 958 bytes} [Tue Aug 2 10:16:22 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 369 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 02 10:16:23.046738 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df33063d-a236-4e39-8067-a63c08b3b5d4 admin admin] Authenticating user token {{(pid=80377) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:16:23.273677 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df33063d-a236-4e39-8067-a63c08b3b5d4 admin admin] Validating token access rules against request {{(pid=80377) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:16:23.299742 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dce15007-322a-4d4d-8264-c6091119c3e9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '891005fa5ff74041b75747d16cc66798', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7047912c5acd4adeba5d87c3bf895e39', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80377) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:16:23.300486 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dce15007-322a-4d4d-8264-c6091119c3e9 admin admin] REQUEST_METHOD: `POST` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:16:23.300824 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dce15007-322a-4d4d-8264-c6091119c3e9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:16:23.301773 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dce15007-322a-4d4d-8264-c6091119c3e9 admin admin] PATH_INFO: `/v3/projects` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:16:23.309390 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dce15007-322a-4d4d-8264-c6091119c3e9 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:16:23.313365 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dce15007-322a-4d4d-8264-c6091119c3e9 admin admin] RBAC: Authorization granted {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:16:23.402222 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: [pid: 80377|app: 0|req: 886/1769] 10.222.0.82 () {60 vars in 1153 bytes} [Tue Aug 2 10:16:23 2022] POST /identity/v3/projects => generated 375 bytes in 356 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 02 10:16:23.407185 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6788da67-1b59-4945-9945-15537fa70b65 None None] Authenticating user token {{(pid=80378) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:16:23.423750 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6788da67-1b59-4945-9945-15537fa70b65 None None] Validating token access rules against request {{(pid=80378) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:16:23.473427 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f98007f8-6f4c-4204-b2c0-125e3b370211 None None] REQUEST_METHOD: `POST` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:16:23.473735 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f98007f8-6f4c-4204-b2c0-125e3b370211 None None] SCRIPT_NAME: `/identity` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:16:23.474057 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f98007f8-6f4c-4204-b2c0-125e3b370211 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:16:23.537678 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG dbcounter [-] [80378] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=80378) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 10:16:23.566291 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG dbcounter [-] [80377] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80377) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 10:16:23.601269 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.auth.core [None req-f98007f8-6f4c-4204-b2c0-125e3b370211 None None] MFA Rules not processed for user `bd8dd72086db48b8aaac2ed0058872df`. Rule list: `[]` (Enabled: `True`). {{(pid=80377) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 02 10:16:23.602654 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.fernet_utils [None req-f98007f8-6f4c-4204-b2c0-125e3b370211 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80377) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 10:16:23.603447 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8ead7cd-3c30-4241-a130-68ce4e84712d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '891005fa5ff74041b75747d16cc66798', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7047912c5acd4adeba5d87c3bf895e39', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80378) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:16:23.604106 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8ead7cd-3c30-4241-a130-68ce4e84712d admin admin] REQUEST_METHOD: `POST` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:16:23.604394 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8ead7cd-3c30-4241-a130-68ce4e84712d admin admin] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:16:23.606006 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8ead7cd-3c30-4241-a130-68ce4e84712d admin admin] PATH_INFO: `/v3/users` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:16:23.606770 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8ead7cd-3c30-4241-a130-68ce4e84712d admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:16:23.612212 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8ead7cd-3c30-4241-a130-68ce4e84712d admin admin] RBAC: Authorization granted {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:16:23.678427 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 884/1770] 10.222.0.82 () {60 vars in 1144 bytes} [Tue Aug 2 10:16:23 2022] POST /identity/v3/users => generated 294 bytes in 271 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 02 10:16:23.693399 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8ead7cd-3c30-4241-a130-68ce4e84712d admin admin] Authenticating user token {{(pid=80378) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:16:23.727127 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8ead7cd-3c30-4241-a130-68ce4e84712d admin admin] Validating token access rules against request {{(pid=80378) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:16:23.747837 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9347b734-fb43-4dbd-a260-7ebb20d177ef admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '891005fa5ff74041b75747d16cc66798', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7047912c5acd4adeba5d87c3bf895e39', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80378) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:16:23.751281 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9347b734-fb43-4dbd-a260-7ebb20d177ef admin admin] REQUEST_METHOD: `GET` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:16:23.752019 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9347b734-fb43-4dbd-a260-7ebb20d177ef admin admin] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:16:23.752558 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9347b734-fb43-4dbd-a260-7ebb20d177ef admin admin] PATH_INFO: `/v3/roles` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:16:23.754130 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9347b734-fb43-4dbd-a260-7ebb20d177ef admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:16:23.760787 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9347b734-fb43-4dbd-a260-7ebb20d177ef admin admin] RBAC: Authorization granted {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:16:23.789463 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 885/1771] 10.222.0.82 () {58 vars in 1122 bytes} [Tue Aug 2 10:16:23 2022] GET /identity/v3/roles => generated 1420 bytes in 110 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 02 10:16:23.791590 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9347b734-fb43-4dbd-a260-7ebb20d177ef admin admin] Authenticating user token {{(pid=80378) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:16:23.792748 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: [pid: 80377|app: 0|req: 887/1772] 10.222.0.82 () {58 vars in 1009 bytes} [Tue Aug 2 10:16:23 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 320 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 02 10:16:23.819512 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9347b734-fb43-4dbd-a260-7ebb20d177ef admin admin] Validating token access rules against request {{(pid=80378) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:16:23.831818 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ede0fd8-6505-47f4-a4ce-950a24745716 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '891005fa5ff74041b75747d16cc66798', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7047912c5acd4adeba5d87c3bf895e39', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80378) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:16:23.832517 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ede0fd8-6505-47f4-a4ce-950a24745716 admin admin] REQUEST_METHOD: `PUT` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:16:23.832806 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ede0fd8-6505-47f4-a4ce-950a24745716 admin admin] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:16:23.834287 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ede0fd8-6505-47f4-a4ce-950a24745716 admin admin] PATH_INFO: `/v3/projects/f7f58addc65446bdb71511d824001610/users/92b55078f99e4e34b00305540703fa85/roles/237966442f05411191105bf336bc14f3` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:16:23.883313 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ede0fd8-6505-47f4-a4ce-950a24745716 admin admin] RBAC: Authorizing `identity:create_grant(project_id=f7f58addc65446bdb71511d824001610, user_id=92b55078f99e4e34b00305540703fa85, role_id=237966442f05411191105bf336bc14f3)` {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:16:23.886115 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ede0fd8-6505-47f4-a4ce-950a24745716 admin admin] RBAC: Authorization granted {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:16:23.904086 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 886/1773] 10.222.0.82 () {60 vars in 1483 bytes} [Tue Aug 2 10:16:23 2022] PUT /identity/v3/projects/f7f58addc65446bdb71511d824001610/users/92b55078f99e4e34b00305540703fa85/roles/237966442f05411191105bf336bc14f3 => generated 0 bytes in 113 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 02 10:16:23.908468 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f98007f8-6f4c-4204-b2c0-125e3b370211 None None] Authenticating user token {{(pid=80377) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:16:23.932622 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f98007f8-6f4c-4204-b2c0-125e3b370211 None None] Validating token access rules against request {{(pid=80377) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:16:23.976749 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ede0fd8-6505-47f4-a4ce-950a24745716 admin admin] Authenticating user token {{(pid=80378) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:16:23.980384 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.fernet_utils [None req-3ede0fd8-6505-47f4-a4ce-950a24745716 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80378) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 10:16:24.172895 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e66713b0-64ce-4997-a26b-d55a78f7a514 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '891005fa5ff74041b75747d16cc66798', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7047912c5acd4adeba5d87c3bf895e39', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80377) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:16:24.184485 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e66713b0-64ce-4997-a26b-d55a78f7a514 admin admin] REQUEST_METHOD: `GET` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:16:24.187181 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e66713b0-64ce-4997-a26b-d55a78f7a514 admin admin] SCRIPT_NAME: `/identity` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:16:24.187181 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e66713b0-64ce-4997-a26b-d55a78f7a514 admin admin] PATH_INFO: `/v3/roles` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:16:24.191005 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e66713b0-64ce-4997-a26b-d55a78f7a514 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:16:24.197668 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e66713b0-64ce-4997-a26b-d55a78f7a514 admin admin] RBAC: Authorization granted {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:16:24.226182 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ede0fd8-6505-47f4-a4ce-950a24745716 admin admin] Validating token access rules against request {{(pid=80378) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:16:24.239597 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71b076a1-affb-4189-84ba-3287258c4eb3 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bef1b2e714cb45c4a51eb6d3e15aad6d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f1b38ebb804c41c69772c998aae72c41', 'project_domain_id': 'default', 'roles': ['admin', 'service', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80378) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:16:24.239895 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: [pid: 80377|app: 0|req: 888/1774] 10.222.0.82 () {58 vars in 1122 bytes} [Tue Aug 2 10:16:23 2022] GET /identity/v3/roles => generated 1420 bytes in 332 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 02 10:16:24.240313 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71b076a1-affb-4189-84ba-3287258c4eb3 service swift] REQUEST_METHOD: `GET` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:16:24.240604 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71b076a1-affb-4189-84ba-3287258c4eb3 service swift] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:16:24.240879 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71b076a1-affb-4189-84ba-3287258c4eb3 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:16:24.244731 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd8295c6-342c-4ea3-980f-54c14eed2da2 None None] REQUEST_METHOD: `POST` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:16:24.245075 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd8295c6-342c-4ea3-980f-54c14eed2da2 None None] SCRIPT_NAME: `/identity` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:16:24.245367 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd8295c6-342c-4ea3-980f-54c14eed2da2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:16:24.355739 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.auth.core [None req-cd8295c6-342c-4ea3-980f-54c14eed2da2 None None] MFA Rules not processed for user `99616b362eac4b7dba445dafd3ca1426`. Rule list: `[]` (Enabled: `True`). {{(pid=80377) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 02 10:16:24.379326 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.fernet_utils [None req-cd8295c6-342c-4ea3-980f-54c14eed2da2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80377) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 10:16:24.481536 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71b076a1-affb-4189-84ba-3287258c4eb3 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:16:24.486385 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71b076a1-affb-4189-84ba-3287258c4eb3 service swift] RBAC: Authorization granted {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:16:24.544248 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 887/1775] 10.222.0.82 () {60 vars in 1384 bytes} [Tue Aug 2 10:16:23 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 567 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Aug 02 10:16:24.545450 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71b076a1-affb-4189-84ba-3287258c4eb3 service swift] Authenticating user token {{(pid=80378) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:16:24.599899 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: [pid: 80377|app: 0|req: 889/1776] 10.222.0.82 () {58 vars in 958 bytes} [Tue Aug 2 10:16:24 2022] POST /identity/v3/auth/tokens => generated 3270 bytes in 359 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 02 10:16:24.610858 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-964479e6-a37c-42b7-a05e-ef977d79b814 None None] REQUEST_METHOD: `POST` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:16:24.611527 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-964479e6-a37c-42b7-a05e-ef977d79b814 None None] SCRIPT_NAME: `/identity` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:16:24.612031 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-964479e6-a37c-42b7-a05e-ef977d79b814 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:16:24.626379 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71b076a1-affb-4189-84ba-3287258c4eb3 service swift] Validating token access rules against request {{(pid=80378) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:16:24.649203 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ff6c4d8-3c58-4f6e-915b-c02747c9b05d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '891005fa5ff74041b75747d16cc66798', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7047912c5acd4adeba5d87c3bf895e39', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80378) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:16:24.663135 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ff6c4d8-3c58-4f6e-915b-c02747c9b05d admin admin] REQUEST_METHOD: `PUT` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:16:24.663874 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ff6c4d8-3c58-4f6e-915b-c02747c9b05d admin admin] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:16:24.666333 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ff6c4d8-3c58-4f6e-915b-c02747c9b05d admin admin] PATH_INFO: `/v3/projects/f7f58addc65446bdb71511d824001610/users/92b55078f99e4e34b00305540703fa85/roles/237966442f05411191105bf336bc14f3` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:16:24.681271 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ff6c4d8-3c58-4f6e-915b-c02747c9b05d admin admin] RBAC: Authorizing `identity:create_grant(project_id=f7f58addc65446bdb71511d824001610, user_id=92b55078f99e4e34b00305540703fa85, role_id=237966442f05411191105bf336bc14f3)` {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:16:24.700950 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.auth.core [None req-964479e6-a37c-42b7-a05e-ef977d79b814 None None] MFA Rules not processed for user `99616b362eac4b7dba445dafd3ca1426`. Rule list: `[]` (Enabled: `True`). {{(pid=80377) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 02 10:16:24.702938 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ff6c4d8-3c58-4f6e-915b-c02747c9b05d admin admin] RBAC: Authorization granted {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:16:24.705463 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.fernet_utils [None req-964479e6-a37c-42b7-a05e-ef977d79b814 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80377) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 10:16:24.741589 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: [pid: 80377|app: 0|req: 890/1777] 10.222.0.82 () {58 vars in 958 bytes} [Tue Aug 2 10:16:24 2022] POST /identity/v3/auth/tokens => generated 3270 bytes in 128 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 02 10:16:24.753321 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-964479e6-a37c-42b7-a05e-ef977d79b814 None None] Authenticating user token {{(pid=80377) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:16:24.755837 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.fernet_utils [None req-964479e6-a37c-42b7-a05e-ef977d79b814 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80377) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 10:16:24.768967 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG dbcounter [-] [80378] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80378) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 10:16:24.798886 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 888/1778] 10.222.0.82 () {60 vars in 1483 bytes} [Tue Aug 2 10:16:24 2022] PUT /identity/v3/projects/f7f58addc65446bdb71511d824001610/users/92b55078f99e4e34b00305540703fa85/roles/237966442f05411191105bf336bc14f3 => generated 0 bytes in 254 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 02 10:16:24.811707 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG dbcounter [-] [80377] Writing DB stats keystone:SELECT=100 {{(pid=80377) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 10:16:24.815852 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-993396d4-1d35-4ec3-b28a-905ab7889189 None None] REQUEST_METHOD: `POST` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:16:24.816152 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-993396d4-1d35-4ec3-b28a-905ab7889189 None None] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:16:24.820286 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-993396d4-1d35-4ec3-b28a-905ab7889189 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:16:24.969113 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.auth.core [None req-993396d4-1d35-4ec3-b28a-905ab7889189 None None] MFA Rules not processed for user `92b55078f99e4e34b00305540703fa85`. Rule list: `[]` (Enabled: `True`). {{(pid=80378) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 02 10:16:24.971910 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.fernet_utils [None req-993396d4-1d35-4ec3-b28a-905ab7889189 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80378) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 10:16:25.135627 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-964479e6-a37c-42b7-a05e-ef977d79b814 None None] Validating token access rules against request {{(pid=80377) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:16:25.272880 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 889/1779] 10.222.0.82 () {58 vars in 958 bytes} [Tue Aug 2 10:16:24 2022] POST /identity/v3/auth/tokens => generated 3319 bytes in 460 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 02 10:16:25.378706 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-993396d4-1d35-4ec3-b28a-905ab7889189 None None] Authenticating user token {{(pid=80378) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:16:25.381417 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.fernet_utils [None req-993396d4-1d35-4ec3-b28a-905ab7889189 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80378) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 10:16:25.408338 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a846583a-30f5-4948-bd54-687a9796f7fc service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88af518196674fe593ad88d30111fb7d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f1b38ebb804c41c69772c998aae72c41', 'project_domain_id': 'default', 'roles': ['admin', 'ResellerAdmin', 'reader', 'service', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80377) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:16:25.408919 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a846583a-30f5-4948-bd54-687a9796f7fc service nova] REQUEST_METHOD: `GET` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:16:25.409192 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a846583a-30f5-4948-bd54-687a9796f7fc service nova] SCRIPT_NAME: `/identity` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:16:25.409474 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a846583a-30f5-4948-bd54-687a9796f7fc service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:16:25.617662 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-993396d4-1d35-4ec3-b28a-905ab7889189 None None] Validating token access rules against request {{(pid=80378) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:16:25.638818 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-400fe3e0-2517-4012-9478-ef00385c1edc service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1023de281ae74e9d8a6d6c312c1c97a7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f1b38ebb804c41c69772c998aae72c41', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80378) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:16:25.640489 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-400fe3e0-2517-4012-9478-ef00385c1edc service cinder] REQUEST_METHOD: `GET` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:16:25.648450 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-400fe3e0-2517-4012-9478-ef00385c1edc service cinder] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:16:25.648746 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-400fe3e0-2517-4012-9478-ef00385c1edc service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:16:25.709083 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a846583a-30f5-4948-bd54-687a9796f7fc service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:16:25.716802 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a846583a-30f5-4948-bd54-687a9796f7fc service nova] RBAC: Authorization granted {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:16:25.764117 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: [pid: 80377|app: 0|req: 891/1780] 10.222.0.82 () {60 vars in 1365 bytes} [Tue Aug 2 10:16:24 2022] GET /identity/v3/auth/tokens => generated 3270 bytes in 1012 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 02 10:16:25.904955 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-400fe3e0-2517-4012-9478-ef00385c1edc service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:16:25.916642 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-400fe3e0-2517-4012-9478-ef00385c1edc service cinder] RBAC: Authorization granted {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:16:25.956077 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 890/1781] 10.222.0.82 () {60 vars in 1365 bytes} [Tue Aug 2 10:16:25 2022] GET /identity/v3/auth/tokens => generated 3252 bytes in 580 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 02 10:16:26.129045 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b8cce37-9390-4872-8299-d3382989c6d6 None None] REQUEST_METHOD: `POST` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:16:26.129334 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b8cce37-9390-4872-8299-d3382989c6d6 None None] SCRIPT_NAME: `/identity` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:16:26.129617 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b8cce37-9390-4872-8299-d3382989c6d6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:16:26.170027 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.auth.core [None req-6b8cce37-9390-4872-8299-d3382989c6d6 None None] MFA Rules not processed for user `b6890a8127e04092920cb1d31ab56d91`. Rule list: `[]` (Enabled: `True`). {{(pid=80377) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 02 10:16:26.171906 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.fernet_utils [None req-6b8cce37-9390-4872-8299-d3382989c6d6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80377) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 10:16:26.416438 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG dbcounter [-] [80377] Writing DB stats keystone:SELECT=100 {{(pid=80377) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 10:16:26.444286 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: [pid: 80377|app: 0|req: 892/1782] 10.222.0.82 () {58 vars in 958 bytes} [Tue Aug 2 10:16:26 2022] POST /identity/v3/auth/tokens => generated 3266 bytes in 316 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 02 10:16:27.299331 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-400fe3e0-2517-4012-9478-ef00385c1edc service cinder] Authenticating user token {{(pid=80378) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:16:27.301746 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.fernet_utils [None req-400fe3e0-2517-4012-9478-ef00385c1edc service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80378) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 10:16:27.468798 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG dbcounter [-] [80378] Writing DB stats keystone:SELECT=100 {{(pid=80378) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 10:16:27.536593 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-400fe3e0-2517-4012-9478-ef00385c1edc service cinder] Validating token access rules against request {{(pid=80378) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:16:27.556339 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7828b5a-477f-485d-9c4f-ebda69c281d4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '891005fa5ff74041b75747d16cc66798', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7047912c5acd4adeba5d87c3bf895e39', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80378) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:16:27.557221 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7828b5a-477f-485d-9c4f-ebda69c281d4 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:16:27.557689 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7828b5a-477f-485d-9c4f-ebda69c281d4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:16:27.558176 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7828b5a-477f-485d-9c4f-ebda69c281d4 admin admin] PATH_INFO: `/v3/users/cda272cef98e4ed6bcf00540dfdd2625` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:16:27.561467 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7828b5a-477f-485d-9c4f-ebda69c281d4 admin admin] RBAC: Authorizing `identity:delete_user(user_id=cda272cef98e4ed6bcf00540dfdd2625)` {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:16:27.567652 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7828b5a-477f-485d-9c4f-ebda69c281d4 admin admin] RBAC: Authorization granted {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:16:27.689291 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.notifications [None req-c7828b5a-477f-485d-9c4f-ebda69c281d4 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'cda272cef98e4ed6bcf00540dfdd2625'} {{(pid=80378) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 10:16:27.710055 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.notifications [None req-c7828b5a-477f-485d-9c4f-ebda69c281d4 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'cda272cef98e4ed6bcf00540dfdd2625'} {{(pid=80378) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 10:16:27.735630 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.notifications [None req-c7828b5a-477f-485d-9c4f-ebda69c281d4 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'cda272cef98e4ed6bcf00540dfdd2625'} {{(pid=80378) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 10:16:27.736434 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.notifications [None req-c7828b5a-477f-485d-9c4f-ebda69c281d4 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'cda272cef98e4ed6bcf00540dfdd2625'} {{(pid=80378) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 10:16:27.773054 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.notifications [None req-c7828b5a-477f-485d-9c4f-ebda69c281d4 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'cda272cef98e4ed6bcf00540dfdd2625'} {{(pid=80378) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 10:16:27.800272 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 891/1783] 10.222.0.82 () {58 vars in 1224 bytes} [Tue Aug 2 10:16:27 2022] DELETE /identity/v3/users/cda272cef98e4ed6bcf00540dfdd2625 => generated 0 bytes in 499 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 02 10:16:27.802557 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4efca14-2d50-4b77-9add-08f1f7d2e4a8 None None] REQUEST_METHOD: `POST` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:16:27.807202 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4efca14-2d50-4b77-9add-08f1f7d2e4a8 None None] SCRIPT_NAME: `/identity` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:16:27.807202 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4efca14-2d50-4b77-9add-08f1f7d2e4a8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:16:27.860993 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.auth.core [None req-a4efca14-2d50-4b77-9add-08f1f7d2e4a8 None None] MFA Rules not processed for user `891005fa5ff74041b75747d16cc66798`. Rule list: `[]` (Enabled: `True`). {{(pid=80377) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 02 10:16:27.862114 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.fernet_utils [None req-a4efca14-2d50-4b77-9add-08f1f7d2e4a8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80377) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 10:16:28.036230 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: [pid: 80377|app: 0|req: 893/1784] 10.222.0.82 () {58 vars in 958 bytes} [Tue Aug 2 10:16:27 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 235 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 02 10:16:28.046176 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7828b5a-477f-485d-9c4f-ebda69c281d4 admin admin] Authenticating user token {{(pid=80378) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:16:28.051991 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.fernet_utils [None req-c7828b5a-477f-485d-9c4f-ebda69c281d4 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80378) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 10:16:28.301940 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7828b5a-477f-485d-9c4f-ebda69c281d4 admin admin] Validating token access rules against request {{(pid=80378) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:16:28.318243 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4b8a579-c9b9-416b-8c66-3cd6447162ef service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '33b87170f0634b67bce60750847b5cbe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f1b38ebb804c41c69772c998aae72c41', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80378) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:16:28.319076 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4b8a579-c9b9-416b-8c66-3cd6447162ef service neutron] REQUEST_METHOD: `GET` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:16:28.319495 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4b8a579-c9b9-416b-8c66-3cd6447162ef service neutron] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:16:28.319887 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4b8a579-c9b9-416b-8c66-3cd6447162ef service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:16:28.346818 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4b8a579-c9b9-416b-8c66-3cd6447162ef service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:16:28.348780 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4b8a579-c9b9-416b-8c66-3cd6447162ef service neutron] RBAC: Authorization granted {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:16:28.372661 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-831fdf75-9cd7-4540-9975-9a4b59100eb4 None None] REQUEST_METHOD: `GET` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:16:28.372948 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-831fdf75-9cd7-4540-9975-9a4b59100eb4 None None] SCRIPT_NAME: `/identity` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:16:28.373228 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-831fdf75-9cd7-4540-9975-9a4b59100eb4 None None] PATH_INFO: `/` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:16:28.374942 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: [pid: 80377|app: 0|req: 894/1785] 10.222.0.82 () {54 vars in 912 bytes} [Tue Aug 2 10:16:28 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 02 10:16:28.403726 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d0d20b4-0fcc-4c37-8e98-5c733e44a067 None None] REQUEST_METHOD: `POST` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:16:28.403726 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d0d20b4-0fcc-4c37-8e98-5c733e44a067 None None] SCRIPT_NAME: `/identity` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:16:28.403726 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d0d20b4-0fcc-4c37-8e98-5c733e44a067 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:16:28.431107 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 892/1786] 10.222.0.82 () {60 vars in 1365 bytes} [Tue Aug 2 10:16:28 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 385 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 02 10:16:28.470796 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.auth.core [None req-7d0d20b4-0fcc-4c37-8e98-5c733e44a067 None None] MFA Rules not processed for user `33b87170f0634b67bce60750847b5cbe`. Rule list: `[]` (Enabled: `True`). {{(pid=80377) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 02 10:16:28.482797 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.fernet_utils [None req-7d0d20b4-0fcc-4c37-8e98-5c733e44a067 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80377) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 10:16:28.523250 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: [pid: 80377|app: 0|req: 895/1787] 10.222.0.82 () {58 vars in 1009 bytes} [Tue Aug 2 10:16:28 2022] POST /identity/v3/auth/tokens => generated 3121 bytes in 123 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 02 10:16:28.632104 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4b8a579-c9b9-416b-8c66-3cd6447162ef service neutron] Authenticating user token {{(pid=80378) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:16:28.633756 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.fernet_utils [None req-e4b8a579-c9b9-416b-8c66-3cd6447162ef service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80378) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 10:16:28.669991 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4b8a579-c9b9-416b-8c66-3cd6447162ef service neutron] Validating token access rules against request {{(pid=80378) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:16:28.688565 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00d01438-7641-45e4-b139-66b65778aa98 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '33b87170f0634b67bce60750847b5cbe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f1b38ebb804c41c69772c998aae72c41', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80378) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:16:28.689228 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00d01438-7641-45e4-b139-66b65778aa98 service neutron] REQUEST_METHOD: `GET` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:16:28.695126 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00d01438-7641-45e4-b139-66b65778aa98 service neutron] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:16:28.695126 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00d01438-7641-45e4-b139-66b65778aa98 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:16:28.751642 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00d01438-7641-45e4-b139-66b65778aa98 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:16:28.769886 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00d01438-7641-45e4-b139-66b65778aa98 service neutron] RBAC: Authorization granted {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:16:28.787598 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 893/1788] 10.222.0.82 () {60 vars in 1365 bytes} [Tue Aug 2 10:16:28 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 156 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 02 10:16:29.175272 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d0d20b4-0fcc-4c37-8e98-5c733e44a067 None None] Authenticating user token {{(pid=80377) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:16:29.177682 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.fernet_utils [None req-7d0d20b4-0fcc-4c37-8e98-5c733e44a067 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80377) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 10:16:29.207054 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d0d20b4-0fcc-4c37-8e98-5c733e44a067 None None] Validating token access rules against request {{(pid=80377) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:16:29.222249 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db34da58-918d-46a8-877d-9bc65a7ab590 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '891005fa5ff74041b75747d16cc66798', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7047912c5acd4adeba5d87c3bf895e39', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80377) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:16:29.222735 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db34da58-918d-46a8-877d-9bc65a7ab590 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:16:29.223018 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db34da58-918d-46a8-877d-9bc65a7ab590 admin admin] SCRIPT_NAME: `/identity` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:16:29.223292 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db34da58-918d-46a8-877d-9bc65a7ab590 admin admin] PATH_INFO: `/v3/projects/4b921cb3c8e941c58ac781f641e2830c` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:16:29.224549 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db34da58-918d-46a8-877d-9bc65a7ab590 admin admin] RBAC: Authorizing `identity:delete_project(project_id=4b921cb3c8e941c58ac781f641e2830c)` {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:16:29.226609 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db34da58-918d-46a8-877d-9bc65a7ab590 admin admin] RBAC: Authorization granted {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:16:29.324193 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.notifications [None req-db34da58-918d-46a8-877d-9bc65a7ab590 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '4b921cb3c8e941c58ac781f641e2830c', 'request_id': 'req-db34da58-918d-46a8-877d-9bc65a7ab590'} {{(pid=80377) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 10:16:29.342477 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.notifications [None req-db34da58-918d-46a8-877d-9bc65a7ab590 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '4b921cb3c8e941c58ac781f641e2830c', 'request_id': 'req-db34da58-918d-46a8-877d-9bc65a7ab590'} {{(pid=80377) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 10:16:29.356432 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.notifications [None req-db34da58-918d-46a8-877d-9bc65a7ab590 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '4b921cb3c8e941c58ac781f641e2830c', 'request_id': 'req-db34da58-918d-46a8-877d-9bc65a7ab590'} {{(pid=80377) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 10:16:29.389908 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.notifications [None req-db34da58-918d-46a8-877d-9bc65a7ab590 admin admin] The token cache is being invalidate because project 4b921cb3c8e941c58ac781f641e2830c was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80377) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 02 10:16:29.390152 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.notifications [None req-db34da58-918d-46a8-877d-9bc65a7ab590 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80377) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 10:16:29.392997 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.notifications [None req-db34da58-918d-46a8-877d-9bc65a7ab590 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80377) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 10:16:29.398969 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00d01438-7641-45e4-b139-66b65778aa98 service neutron] Authenticating user token {{(pid=80378) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:16:29.399760 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: [pid: 80377|app: 0|req: 896/1789] 10.222.0.82 () {58 vars in 1233 bytes} [Tue Aug 2 10:16:29 2022] DELETE /identity/v3/projects/4b921cb3c8e941c58ac781f641e2830c => generated 0 bytes in 220 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 02 10:16:29.415635 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00d01438-7641-45e4-b139-66b65778aa98 service neutron] Validating token access rules against request {{(pid=80378) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:16:29.469469 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG dbcounter [-] [80378] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80378) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 10:16:29.486761 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db39a17b-b0a7-445e-8822-0ea5c6f9ca19 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '891005fa5ff74041b75747d16cc66798', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7047912c5acd4adeba5d87c3bf895e39', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80378) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:16:29.487442 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db39a17b-b0a7-445e-8822-0ea5c6f9ca19 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:16:29.487726 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db39a17b-b0a7-445e-8822-0ea5c6f9ca19 admin admin] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:16:29.488187 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db39a17b-b0a7-445e-8822-0ea5c6f9ca19 admin admin] PATH_INFO: `/v3/users/2159c8b1594e482d9e1955b6491737b3` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:16:29.491245 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db39a17b-b0a7-445e-8822-0ea5c6f9ca19 admin admin] RBAC: Authorizing `identity:delete_user(user_id=2159c8b1594e482d9e1955b6491737b3)` {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:16:29.493755 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db39a17b-b0a7-445e-8822-0ea5c6f9ca19 admin admin] RBAC: Authorization granted {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:16:29.559078 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.notifications [None req-db39a17b-b0a7-445e-8822-0ea5c6f9ca19 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '2159c8b1594e482d9e1955b6491737b3'} {{(pid=80378) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 10:16:29.572057 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.notifications [None req-db39a17b-b0a7-445e-8822-0ea5c6f9ca19 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '2159c8b1594e482d9e1955b6491737b3'} {{(pid=80378) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 10:16:29.583020 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.notifications [None req-db39a17b-b0a7-445e-8822-0ea5c6f9ca19 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '2159c8b1594e482d9e1955b6491737b3'} {{(pid=80378) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 10:16:29.583774 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.notifications [None req-db39a17b-b0a7-445e-8822-0ea5c6f9ca19 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '2159c8b1594e482d9e1955b6491737b3'} {{(pid=80378) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 10:16:29.599652 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.notifications [None req-db39a17b-b0a7-445e-8822-0ea5c6f9ca19 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '2159c8b1594e482d9e1955b6491737b3'} {{(pid=80378) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 10:16:29.631697 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 894/1790] 10.222.0.82 () {58 vars in 1224 bytes} [Tue Aug 2 10:16:29 2022] DELETE /identity/v3/users/2159c8b1594e482d9e1955b6491737b3 => generated 0 bytes in 230 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 02 10:16:30.401651 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-279fa280-da99-4386-b9f0-d2a2e83c5910 None None] REQUEST_METHOD: `POST` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:16:30.402143 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-279fa280-da99-4386-b9f0-d2a2e83c5910 None None] SCRIPT_NAME: `/identity` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:16:30.410104 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-279fa280-da99-4386-b9f0-d2a2e83c5910 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:16:30.424971 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db39a17b-b0a7-445e-8822-0ea5c6f9ca19 admin admin] Authenticating user token {{(pid=80378) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:16:30.426778 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.fernet_utils [None req-db39a17b-b0a7-445e-8822-0ea5c6f9ca19 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80378) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 10:16:30.505563 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.auth.core [None req-279fa280-da99-4386-b9f0-d2a2e83c5910 None None] MFA Rules not processed for user `99616b362eac4b7dba445dafd3ca1426`. Rule list: `[]` (Enabled: `True`). {{(pid=80377) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 02 10:16:30.523887 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.fernet_utils [None req-279fa280-da99-4386-b9f0-d2a2e83c5910 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80377) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 10:16:30.618508 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db39a17b-b0a7-445e-8822-0ea5c6f9ca19 admin admin] Validating token access rules against request {{(pid=80378) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:16:30.624531 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1cf7fc1-757f-4de1-9a3f-11772afe512e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '891005fa5ff74041b75747d16cc66798', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7047912c5acd4adeba5d87c3bf895e39', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80378) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:16:30.625223 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1cf7fc1-757f-4de1-9a3f-11772afe512e admin admin] REQUEST_METHOD: `DELETE` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:16:30.625531 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1cf7fc1-757f-4de1-9a3f-11772afe512e admin admin] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:16:30.625865 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1cf7fc1-757f-4de1-9a3f-11772afe512e admin admin] PATH_INFO: `/v3/projects/01874b4ed3844636ad2cef0be4876db7` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:16:30.626793 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: [pid: 80377|app: 0|req: 897/1791] 10.222.0.82 () {58 vars in 958 bytes} [Tue Aug 2 10:16:30 2022] POST /identity/v3/auth/tokens => generated 3270 bytes in 229 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 02 10:16:30.627219 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1cf7fc1-757f-4de1-9a3f-11772afe512e admin admin] RBAC: Authorizing `identity:delete_project(project_id=01874b4ed3844636ad2cef0be4876db7)` {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:16:30.629460 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1cf7fc1-757f-4de1-9a3f-11772afe512e admin admin] RBAC: Authorization granted {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:16:30.725421 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.notifications [None req-e1cf7fc1-757f-4de1-9a3f-11772afe512e admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '01874b4ed3844636ad2cef0be4876db7', 'request_id': 'req-e1cf7fc1-757f-4de1-9a3f-11772afe512e'} {{(pid=80378) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 10:16:30.742887 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.notifications [None req-e1cf7fc1-757f-4de1-9a3f-11772afe512e admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '01874b4ed3844636ad2cef0be4876db7', 'request_id': 'req-e1cf7fc1-757f-4de1-9a3f-11772afe512e'} {{(pid=80378) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 10:16:30.754809 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.notifications [None req-e1cf7fc1-757f-4de1-9a3f-11772afe512e admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '01874b4ed3844636ad2cef0be4876db7', 'request_id': 'req-e1cf7fc1-757f-4de1-9a3f-11772afe512e'} {{(pid=80378) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 10:16:30.764222 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG dbcounter [-] [80378] Writing DB stats keystone:SELECT=81,keystone:DELETE=17,keystone:INSERT=2 {{(pid=80378) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 10:16:30.772254 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.notifications [None req-e1cf7fc1-757f-4de1-9a3f-11772afe512e admin admin] The token cache is being invalidate because project 01874b4ed3844636ad2cef0be4876db7 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80378) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 02 10:16:30.772861 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.notifications [None req-e1cf7fc1-757f-4de1-9a3f-11772afe512e admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80378) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 10:16:30.773945 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.notifications [None req-e1cf7fc1-757f-4de1-9a3f-11772afe512e admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80378) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 10:16:30.776240 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 895/1792] 10.222.0.82 () {58 vars in 1233 bytes} [Tue Aug 2 10:16:30 2022] DELETE /identity/v3/projects/01874b4ed3844636ad2cef0be4876db7 => generated 0 bytes in 352 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 02 10:16:30.783666 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73b57915-849c-48cb-be50-97c3a1125e0d None None] REQUEST_METHOD: `POST` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:16:30.784193 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73b57915-849c-48cb-be50-97c3a1125e0d None None] SCRIPT_NAME: `/identity` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:16:30.784469 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73b57915-849c-48cb-be50-97c3a1125e0d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:16:30.806553 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.auth.core [None req-73b57915-849c-48cb-be50-97c3a1125e0d None None] MFA Rules not processed for user `891005fa5ff74041b75747d16cc66798`. Rule list: `[]` (Enabled: `True`). {{(pid=80377) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 02 10:16:30.807773 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.fernet_utils [None req-73b57915-849c-48cb-be50-97c3a1125e0d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80377) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 10:16:30.821059 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG dbcounter [-] [80377] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80377) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 10:16:30.917688 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: [pid: 80377|app: 0|req: 898/1793] 10.222.0.82 () {58 vars in 958 bytes} [Tue Aug 2 10:16:30 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 135 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 02 10:16:30.926861 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01bbfc3e-1db3-4c39-917e-383996cebe53 None None] REQUEST_METHOD: `POST` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:16:30.927171 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01bbfc3e-1db3-4c39-917e-383996cebe53 None None] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:16:30.927437 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01bbfc3e-1db3-4c39-917e-383996cebe53 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:16:30.947551 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.auth.core [None req-01bbfc3e-1db3-4c39-917e-383996cebe53 None None] MFA Rules not processed for user `891005fa5ff74041b75747d16cc66798`. Rule list: `[]` (Enabled: `True`). {{(pid=80378) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 02 10:16:30.948772 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.fernet_utils [None req-01bbfc3e-1db3-4c39-917e-383996cebe53 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80378) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 10:16:30.958243 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 896/1794] 10.222.0.82 () {58 vars in 958 bytes} [Tue Aug 2 10:16:30 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 02 10:16:30.962638 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73b57915-849c-48cb-be50-97c3a1125e0d None None] Authenticating user token {{(pid=80377) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:16:30.974668 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73b57915-849c-48cb-be50-97c3a1125e0d None None] Validating token access rules against request {{(pid=80377) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:16:30.980118 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfced8f5-93d1-4648-ab37-a7df554e57e6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '891005fa5ff74041b75747d16cc66798', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7047912c5acd4adeba5d87c3bf895e39', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80377) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:16:30.980747 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfced8f5-93d1-4648-ab37-a7df554e57e6 admin admin] REQUEST_METHOD: `GET` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:16:30.981057 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfced8f5-93d1-4648-ab37-a7df554e57e6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:16:30.981366 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfced8f5-93d1-4648-ab37-a7df554e57e6 admin admin] PATH_INFO: `/v3/domains` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:16:30.982106 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cfced8f5-93d1-4648-ab37-a7df554e57e6 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80377) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 02 10:16:30.982415 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cfced8f5-93d1-4648-ab37-a7df554e57e6 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:16:30.984017 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cfced8f5-93d1-4648-ab37-a7df554e57e6 admin admin] RBAC: Authorization granted {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:16:30.995976 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: [pid: 80377|app: 0|req: 899/1795] 10.222.0.82 () {58 vars in 1153 bytes} [Tue Aug 2 10:16:30 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 02 10:16:31.000630 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01bbfc3e-1db3-4c39-917e-383996cebe53 None None] Authenticating user token {{(pid=80378) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:16:31.010289 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01bbfc3e-1db3-4c39-917e-383996cebe53 None None] Validating token access rules against request {{(pid=80378) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:16:31.015534 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a07a324-df9d-4722-bd70-f57bc2f766a0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '891005fa5ff74041b75747d16cc66798', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7047912c5acd4adeba5d87c3bf895e39', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80378) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:16:31.016392 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a07a324-df9d-4722-bd70-f57bc2f766a0 admin admin] REQUEST_METHOD: `POST` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:16:31.016673 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a07a324-df9d-4722-bd70-f57bc2f766a0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:16:31.016950 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a07a324-df9d-4722-bd70-f57bc2f766a0 admin admin] PATH_INFO: `/v3/projects` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:16:31.017670 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a07a324-df9d-4722-bd70-f57bc2f766a0 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:16:31.019581 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a07a324-df9d-4722-bd70-f57bc2f766a0 admin admin] RBAC: Authorization granted {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:16:31.064157 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 897/1796] 10.222.0.82 () {60 vars in 1153 bytes} [Tue Aug 2 10:16:30 2022] POST /identity/v3/projects => generated 385 bytes in 61 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 02 10:16:31.065238 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfced8f5-93d1-4648-ab37-a7df554e57e6 admin admin] Authenticating user token {{(pid=80377) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:16:31.078658 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfced8f5-93d1-4648-ab37-a7df554e57e6 admin admin] Validating token access rules against request {{(pid=80377) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:16:31.174485 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6988b6b3-fb44-44a1-a7c0-8e9d25f03d01 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '891005fa5ff74041b75747d16cc66798', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7047912c5acd4adeba5d87c3bf895e39', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80377) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:16:31.175181 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6988b6b3-fb44-44a1-a7c0-8e9d25f03d01 admin admin] REQUEST_METHOD: `POST` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:16:31.175465 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6988b6b3-fb44-44a1-a7c0-8e9d25f03d01 admin admin] SCRIPT_NAME: `/identity` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:16:31.175741 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6988b6b3-fb44-44a1-a7c0-8e9d25f03d01 admin admin] PATH_INFO: `/v3/users` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:16:31.176841 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6988b6b3-fb44-44a1-a7c0-8e9d25f03d01 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:16:31.178942 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6988b6b3-fb44-44a1-a7c0-8e9d25f03d01 admin admin] RBAC: Authorization granted {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:16:31.208415 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: [pid: 80377|app: 0|req: 900/1797] 10.222.0.82 () {60 vars in 1144 bytes} [Tue Aug 2 10:16:31 2022] POST /identity/v3/users => generated 293 bytes in 144 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 02 10:16:31.212719 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a07a324-df9d-4722-bd70-f57bc2f766a0 admin admin] Authenticating user token {{(pid=80378) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:16:31.224769 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a07a324-df9d-4722-bd70-f57bc2f766a0 admin admin] Validating token access rules against request {{(pid=80378) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:16:31.231244 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-deab1878-dff1-4b0c-b112-52ff16d5e13c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '891005fa5ff74041b75747d16cc66798', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7047912c5acd4adeba5d87c3bf895e39', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80378) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:16:31.232213 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-deab1878-dff1-4b0c-b112-52ff16d5e13c admin admin] REQUEST_METHOD: `GET` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:16:31.232597 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-deab1878-dff1-4b0c-b112-52ff16d5e13c admin admin] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:16:31.232971 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-deab1878-dff1-4b0c-b112-52ff16d5e13c admin admin] PATH_INFO: `/v3/roles` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:16:31.233813 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-deab1878-dff1-4b0c-b112-52ff16d5e13c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:16:31.236116 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-deab1878-dff1-4b0c-b112-52ff16d5e13c admin admin] RBAC: Authorization granted {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:16:31.251899 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 898/1798] 10.222.0.82 () {58 vars in 1122 bytes} [Tue Aug 2 10:16:31 2022] GET /identity/v3/roles => generated 1420 bytes in 40 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 02 10:16:31.253324 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6988b6b3-fb44-44a1-a7c0-8e9d25f03d01 admin admin] Authenticating user token {{(pid=80377) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:16:31.266440 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6988b6b3-fb44-44a1-a7c0-8e9d25f03d01 admin admin] Validating token access rules against request {{(pid=80377) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:16:31.275494 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-864844bf-0a4f-4f63-ae6c-cc20a485a2bb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '891005fa5ff74041b75747d16cc66798', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7047912c5acd4adeba5d87c3bf895e39', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80377) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:16:31.276146 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-864844bf-0a4f-4f63-ae6c-cc20a485a2bb admin admin] REQUEST_METHOD: `GET` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:16:31.276433 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-864844bf-0a4f-4f63-ae6c-cc20a485a2bb admin admin] SCRIPT_NAME: `/identity` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:16:31.276712 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-864844bf-0a4f-4f63-ae6c-cc20a485a2bb admin admin] PATH_INFO: `/v3/roles` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:16:31.279255 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-864844bf-0a4f-4f63-ae6c-cc20a485a2bb admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:16:31.281715 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-864844bf-0a4f-4f63-ae6c-cc20a485a2bb admin admin] RBAC: Authorization granted {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:16:31.301152 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: [pid: 80377|app: 0|req: 901/1799] 10.222.0.82 () {58 vars in 1122 bytes} [Tue Aug 2 10:16:31 2022] GET /identity/v3/roles => generated 1420 bytes in 45 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 02 10:16:31.302727 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-deab1878-dff1-4b0c-b112-52ff16d5e13c admin admin] Authenticating user token {{(pid=80378) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:16:31.314799 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-deab1878-dff1-4b0c-b112-52ff16d5e13c admin admin] Validating token access rules against request {{(pid=80378) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:16:31.319611 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c681d3b7-e18b-407f-951f-fa49d93a5002 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '891005fa5ff74041b75747d16cc66798', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7047912c5acd4adeba5d87c3bf895e39', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80378) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:16:31.320561 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c681d3b7-e18b-407f-951f-fa49d93a5002 admin admin] REQUEST_METHOD: `PUT` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:16:31.320943 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c681d3b7-e18b-407f-951f-fa49d93a5002 admin admin] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:16:31.321390 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c681d3b7-e18b-407f-951f-fa49d93a5002 admin admin] PATH_INFO: `/v3/projects/c84b85209a1e44c98f302799f7b491a1/users/5d1d53eb5bda4de0b93e33fb3cc9d1e6/roles/e1c04b19fc4543b8b54f58bbcffbc598` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:16:31.344411 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c681d3b7-e18b-407f-951f-fa49d93a5002 admin admin] RBAC: Authorizing `identity:create_grant(project_id=c84b85209a1e44c98f302799f7b491a1, user_id=5d1d53eb5bda4de0b93e33fb3cc9d1e6, role_id=e1c04b19fc4543b8b54f58bbcffbc598)` {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:16:31.347264 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c681d3b7-e18b-407f-951f-fa49d93a5002 admin admin] RBAC: Authorization granted {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:16:31.366759 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 899/1800] 10.222.0.82 () {60 vars in 1483 bytes} [Tue Aug 2 10:16:31 2022] PUT /identity/v3/projects/c84b85209a1e44c98f302799f7b491a1/users/5d1d53eb5bda4de0b93e33fb3cc9d1e6/roles/e1c04b19fc4543b8b54f58bbcffbc598 => generated 0 bytes in 64 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 02 10:16:31.377589 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54cb5a64-cc7c-4371-bbe8-9b5546460373 None None] REQUEST_METHOD: `POST` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:16:31.378119 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54cb5a64-cc7c-4371-bbe8-9b5546460373 None None] SCRIPT_NAME: `/identity` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:16:31.378513 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54cb5a64-cc7c-4371-bbe8-9b5546460373 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:16:31.409917 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.auth.core [None req-54cb5a64-cc7c-4371-bbe8-9b5546460373 None None] MFA Rules not processed for user `5d1d53eb5bda4de0b93e33fb3cc9d1e6`. Rule list: `[]` (Enabled: `True`). {{(pid=80377) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 02 10:16:31.411234 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.fernet_utils [None req-54cb5a64-cc7c-4371-bbe8-9b5546460373 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80377) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 10:16:31.500510 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c681d3b7-e18b-407f-951f-fa49d93a5002 admin admin] Authenticating user token {{(pid=80378) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:16:31.511096 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: [pid: 80377|app: 0|req: 902/1801] 10.222.0.82 () {58 vars in 958 bytes} [Tue Aug 2 10:16:31 2022] POST /identity/v3/auth/tokens => generated 3262 bytes in 135 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 02 10:16:31.601968 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c681d3b7-e18b-407f-951f-fa49d93a5002 admin admin] Validating token access rules against request {{(pid=80378) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:16:31.619764 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77d704d6-f180-4b2f-aec5-9b95ea94ef50 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '891005fa5ff74041b75747d16cc66798', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7047912c5acd4adeba5d87c3bf895e39', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80378) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:16:31.620607 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77d704d6-f180-4b2f-aec5-9b95ea94ef50 admin admin] REQUEST_METHOD: `POST` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:16:31.620898 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77d704d6-f180-4b2f-aec5-9b95ea94ef50 admin admin] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:16:31.621199 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77d704d6-f180-4b2f-aec5-9b95ea94ef50 admin admin] PATH_INFO: `/v3/projects` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:16:31.622651 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77d704d6-f180-4b2f-aec5-9b95ea94ef50 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:16:31.628947 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77d704d6-f180-4b2f-aec5-9b95ea94ef50 admin admin] RBAC: Authorization granted {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:16:31.676826 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 900/1802] 10.222.0.82 () {60 vars in 1153 bytes} [Tue Aug 2 10:16:31 2022] POST /identity/v3/projects => generated 387 bytes in 176 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 02 10:16:31.681098 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54cb5a64-cc7c-4371-bbe8-9b5546460373 None None] Authenticating user token {{(pid=80377) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:16:31.697455 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54cb5a64-cc7c-4371-bbe8-9b5546460373 None None] Validating token access rules against request {{(pid=80377) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:16:31.714011 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG dbcounter [-] [80377] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80377) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 10:16:31.789648 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a8e772f-8c27-44b1-8e4a-840f4365eef1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '891005fa5ff74041b75747d16cc66798', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7047912c5acd4adeba5d87c3bf895e39', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80377) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:16:31.790401 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a8e772f-8c27-44b1-8e4a-840f4365eef1 admin admin] REQUEST_METHOD: `POST` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:16:31.790761 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a8e772f-8c27-44b1-8e4a-840f4365eef1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:16:31.791106 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a8e772f-8c27-44b1-8e4a-840f4365eef1 admin admin] PATH_INFO: `/v3/users` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:16:31.791858 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a8e772f-8c27-44b1-8e4a-840f4365eef1 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:16:31.794414 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a8e772f-8c27-44b1-8e4a-840f4365eef1 admin admin] RBAC: Authorization granted {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:16:31.818602 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: [pid: 80377|app: 0|req: 903/1803] 10.222.0.82 () {60 vars in 1144 bytes} [Tue Aug 2 10:16:31 2022] POST /identity/v3/users => generated 300 bytes in 138 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 02 10:16:31.822452 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77d704d6-f180-4b2f-aec5-9b95ea94ef50 admin admin] Authenticating user token {{(pid=80378) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:16:31.834172 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77d704d6-f180-4b2f-aec5-9b95ea94ef50 admin admin] Validating token access rules against request {{(pid=80378) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:16:31.839768 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c4604e0-98b8-4b48-aebe-461032968580 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '891005fa5ff74041b75747d16cc66798', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7047912c5acd4adeba5d87c3bf895e39', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80378) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:16:31.840414 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c4604e0-98b8-4b48-aebe-461032968580 admin admin] REQUEST_METHOD: `GET` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:16:31.840756 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c4604e0-98b8-4b48-aebe-461032968580 admin admin] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:16:31.841561 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c4604e0-98b8-4b48-aebe-461032968580 admin admin] PATH_INFO: `/v3/roles` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:16:31.842300 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c4604e0-98b8-4b48-aebe-461032968580 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:16:31.844039 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c4604e0-98b8-4b48-aebe-461032968580 admin admin] RBAC: Authorization granted {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:16:31.857023 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 901/1804] 10.222.0.82 () {58 vars in 1122 bytes} [Tue Aug 2 10:16:31 2022] GET /identity/v3/roles => generated 1420 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 02 10:16:31.861750 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a8e772f-8c27-44b1-8e4a-840f4365eef1 admin admin] Authenticating user token {{(pid=80377) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:16:31.875991 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a8e772f-8c27-44b1-8e4a-840f4365eef1 admin admin] Validating token access rules against request {{(pid=80377) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:16:31.881945 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8070ec49-5455-4e97-b7ad-dedcfb69b1a4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '891005fa5ff74041b75747d16cc66798', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7047912c5acd4adeba5d87c3bf895e39', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80377) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:16:31.882964 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8070ec49-5455-4e97-b7ad-dedcfb69b1a4 admin admin] REQUEST_METHOD: `PUT` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:16:31.883432 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8070ec49-5455-4e97-b7ad-dedcfb69b1a4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:16:31.883965 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8070ec49-5455-4e97-b7ad-dedcfb69b1a4 admin admin] PATH_INFO: `/v3/projects/0fc50cd7647f43a7a33287a0b6bcf2da/users/aa9f81a3a6364fe3a7c7bdf693b37661/roles/237966442f05411191105bf336bc14f3` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:16:31.913448 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8070ec49-5455-4e97-b7ad-dedcfb69b1a4 admin admin] RBAC: Authorizing `identity:create_grant(project_id=0fc50cd7647f43a7a33287a0b6bcf2da, user_id=aa9f81a3a6364fe3a7c7bdf693b37661, role_id=237966442f05411191105bf336bc14f3)` {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:16:31.919854 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8070ec49-5455-4e97-b7ad-dedcfb69b1a4 admin admin] RBAC: Authorization granted {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:16:32.005902 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c4604e0-98b8-4b48-aebe-461032968580 admin admin] Authenticating user token {{(pid=80378) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:16:32.018482 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: [pid: 80377|app: 0|req: 904/1805] 10.222.0.82 () {60 vars in 1483 bytes} [Tue Aug 2 10:16:31 2022] PUT /identity/v3/projects/0fc50cd7647f43a7a33287a0b6bcf2da/users/aa9f81a3a6364fe3a7c7bdf693b37661/roles/237966442f05411191105bf336bc14f3 => generated 0 bytes in 140 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 02 10:16:32.022319 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c4604e0-98b8-4b48-aebe-461032968580 admin admin] Validating token access rules against request {{(pid=80378) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:16:32.101512 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75fa0509-cb5a-49bf-a985-b0bc2e3db85a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '891005fa5ff74041b75747d16cc66798', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7047912c5acd4adeba5d87c3bf895e39', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80378) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:16:32.102385 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75fa0509-cb5a-49bf-a985-b0bc2e3db85a admin admin] REQUEST_METHOD: `GET` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:16:32.102703 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75fa0509-cb5a-49bf-a985-b0bc2e3db85a admin admin] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:16:32.102985 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75fa0509-cb5a-49bf-a985-b0bc2e3db85a admin admin] PATH_INFO: `/v3/roles` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:16:32.103684 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75fa0509-cb5a-49bf-a985-b0bc2e3db85a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:16:32.105335 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75fa0509-cb5a-49bf-a985-b0bc2e3db85a admin admin] RBAC: Authorization granted {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:16:32.131296 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 902/1806] 10.222.0.82 () {58 vars in 1122 bytes} [Tue Aug 2 10:16:32 2022] GET /identity/v3/roles => generated 1420 bytes in 126 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 02 10:16:32.132458 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8070ec49-5455-4e97-b7ad-dedcfb69b1a4 admin admin] Authenticating user token {{(pid=80377) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:16:32.163476 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8070ec49-5455-4e97-b7ad-dedcfb69b1a4 admin admin] Validating token access rules against request {{(pid=80377) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:16:32.185858 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a45f83be-7d96-4dd1-9c54-7853060e12c3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '891005fa5ff74041b75747d16cc66798', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7047912c5acd4adeba5d87c3bf895e39', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80377) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:16:32.186421 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a45f83be-7d96-4dd1-9c54-7853060e12c3 admin admin] REQUEST_METHOD: `PUT` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:16:32.186668 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a45f83be-7d96-4dd1-9c54-7853060e12c3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:16:32.186944 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a45f83be-7d96-4dd1-9c54-7853060e12c3 admin admin] PATH_INFO: `/v3/projects/0fc50cd7647f43a7a33287a0b6bcf2da/users/aa9f81a3a6364fe3a7c7bdf693b37661/roles/237966442f05411191105bf336bc14f3` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:16:32.189004 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a45f83be-7d96-4dd1-9c54-7853060e12c3 admin admin] RBAC: Authorizing `identity:create_grant(project_id=0fc50cd7647f43a7a33287a0b6bcf2da, user_id=aa9f81a3a6364fe3a7c7bdf693b37661, role_id=237966442f05411191105bf336bc14f3)` {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:16:32.192020 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a45f83be-7d96-4dd1-9c54-7853060e12c3 admin admin] RBAC: Authorization granted {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:16:32.218602 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: [pid: 80377|app: 0|req: 905/1807] 10.222.0.82 () {60 vars in 1483 bytes} [Tue Aug 2 10:16:32 2022] PUT /identity/v3/projects/0fc50cd7647f43a7a33287a0b6bcf2da/users/aa9f81a3a6364fe3a7c7bdf693b37661/roles/237966442f05411191105bf336bc14f3 => generated 0 bytes in 86 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 02 10:16:32.232106 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d82aee7-3afd-4e69-8366-ff159f415340 None None] REQUEST_METHOD: `POST` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:16:32.233422 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d82aee7-3afd-4e69-8366-ff159f415340 None None] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:16:32.233821 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d82aee7-3afd-4e69-8366-ff159f415340 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:16:32.251918 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG dbcounter [-] [80378] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80378) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 10:16:32.331294 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.auth.core [None req-1d82aee7-3afd-4e69-8366-ff159f415340 None None] MFA Rules not processed for user `aa9f81a3a6364fe3a7c7bdf693b37661`. Rule list: `[]` (Enabled: `True`). {{(pid=80378) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 02 10:16:32.332569 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.fernet_utils [None req-1d82aee7-3afd-4e69-8366-ff159f415340 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80378) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 10:16:32.495549 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 903/1808] 10.222.0.82 () {58 vars in 958 bytes} [Tue Aug 2 10:16:32 2022] POST /identity/v3/auth/tokens => generated 3331 bytes in 266 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 02 10:16:32.506911 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a45f83be-7d96-4dd1-9c54-7853060e12c3 admin admin] Authenticating user token {{(pid=80377) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:16:32.512316 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.fernet_utils [None req-a45f83be-7d96-4dd1-9c54-7853060e12c3 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80377) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 10:16:32.611107 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a45f83be-7d96-4dd1-9c54-7853060e12c3 admin admin] Validating token access rules against request {{(pid=80377) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:16:32.615265 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07223c1e-ac90-4450-a3fe-4ccc6e1b1140 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1023de281ae74e9d8a6d6c312c1c97a7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f1b38ebb804c41c69772c998aae72c41', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80377) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:16:32.615904 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07223c1e-ac90-4450-a3fe-4ccc6e1b1140 service cinder] REQUEST_METHOD: `GET` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:16:32.616189 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07223c1e-ac90-4450-a3fe-4ccc6e1b1140 service cinder] SCRIPT_NAME: `/identity` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:16:32.616744 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07223c1e-ac90-4450-a3fe-4ccc6e1b1140 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:16:32.640550 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07223c1e-ac90-4450-a3fe-4ccc6e1b1140 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:16:32.642300 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07223c1e-ac90-4450-a3fe-4ccc6e1b1140 service cinder] RBAC: Authorization granted {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:16:32.663052 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: [pid: 80377|app: 0|req: 906/1809] 10.222.0.82 () {60 vars in 1365 bytes} [Tue Aug 2 10:16:32 2022] GET /identity/v3/auth/tokens => generated 3331 bytes in 156 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 02 10:16:33.769611 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d82aee7-3afd-4e69-8366-ff159f415340 None None] Authenticating user token {{(pid=80378) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:16:33.836052 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d82aee7-3afd-4e69-8366-ff159f415340 None None] Validating token access rules against request {{(pid=80378) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:16:33.841415 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4aec2a6-6933-4eb7-a122-6c6dd1350b30 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '891005fa5ff74041b75747d16cc66798', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7047912c5acd4adeba5d87c3bf895e39', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80378) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:16:33.842121 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4aec2a6-6933-4eb7-a122-6c6dd1350b30 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:16:33.842433 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4aec2a6-6933-4eb7-a122-6c6dd1350b30 admin admin] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:16:33.842709 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4aec2a6-6933-4eb7-a122-6c6dd1350b30 admin admin] PATH_INFO: `/v3/users/5d1d53eb5bda4de0b93e33fb3cc9d1e6` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:16:33.843901 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4aec2a6-6933-4eb7-a122-6c6dd1350b30 admin admin] RBAC: Authorizing `identity:delete_user(user_id=5d1d53eb5bda4de0b93e33fb3cc9d1e6)` {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:16:33.845973 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4aec2a6-6933-4eb7-a122-6c6dd1350b30 admin admin] RBAC: Authorization granted {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:16:33.904158 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.notifications [None req-e4aec2a6-6933-4eb7-a122-6c6dd1350b30 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '5d1d53eb5bda4de0b93e33fb3cc9d1e6'} {{(pid=80378) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 10:16:33.911168 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.notifications [None req-e4aec2a6-6933-4eb7-a122-6c6dd1350b30 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '5d1d53eb5bda4de0b93e33fb3cc9d1e6'} {{(pid=80378) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 10:16:33.919113 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.notifications [None req-e4aec2a6-6933-4eb7-a122-6c6dd1350b30 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '5d1d53eb5bda4de0b93e33fb3cc9d1e6'} {{(pid=80378) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 10:16:33.919892 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.notifications [None req-e4aec2a6-6933-4eb7-a122-6c6dd1350b30 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '5d1d53eb5bda4de0b93e33fb3cc9d1e6'} {{(pid=80378) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 10:16:33.933355 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.notifications [None req-e4aec2a6-6933-4eb7-a122-6c6dd1350b30 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '5d1d53eb5bda4de0b93e33fb3cc9d1e6'} {{(pid=80378) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 10:16:33.939528 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 904/1810] 10.222.0.82 () {58 vars in 1224 bytes} [Tue Aug 2 10:16:33 2022] DELETE /identity/v3/users/5d1d53eb5bda4de0b93e33fb3cc9d1e6 => generated 0 bytes in 170 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 02 10:16:33.944136 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe4256d0-6358-4861-a445-7d197daf22cd None None] REQUEST_METHOD: `POST` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:16:33.944633 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe4256d0-6358-4861-a445-7d197daf22cd None None] SCRIPT_NAME: `/identity` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:16:33.944737 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe4256d0-6358-4861-a445-7d197daf22cd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:16:33.966397 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.auth.core [None req-fe4256d0-6358-4861-a445-7d197daf22cd None None] MFA Rules not processed for user `891005fa5ff74041b75747d16cc66798`. Rule list: `[]` (Enabled: `True`). {{(pid=80377) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 02 10:16:33.967661 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.fernet_utils [None req-fe4256d0-6358-4861-a445-7d197daf22cd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80377) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 10:16:34.034562 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: [pid: 80377|app: 0|req: 907/1811] 10.222.0.82 () {58 vars in 958 bytes} [Tue Aug 2 10:16:33 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 91 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 02 10:16:34.046869 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4aec2a6-6933-4eb7-a122-6c6dd1350b30 admin admin] Authenticating user token {{(pid=80378) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:16:34.048440 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.fernet_utils [None req-e4aec2a6-6933-4eb7-a122-6c6dd1350b30 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80378) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 10:16:34.063122 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG dbcounter [-] [80378] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80378) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 10:16:34.125564 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4aec2a6-6933-4eb7-a122-6c6dd1350b30 admin admin] Validating token access rules against request {{(pid=80378) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:16:34.129717 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef140704-32dd-4204-b1cc-ebe4c610b941 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '33b87170f0634b67bce60750847b5cbe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f1b38ebb804c41c69772c998aae72c41', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80378) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:16:34.130505 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef140704-32dd-4204-b1cc-ebe4c610b941 service neutron] REQUEST_METHOD: `GET` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:16:34.130896 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef140704-32dd-4204-b1cc-ebe4c610b941 service neutron] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:16:34.131240 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef140704-32dd-4204-b1cc-ebe4c610b941 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:16:34.142156 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef140704-32dd-4204-b1cc-ebe4c610b941 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:16:34.143686 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef140704-32dd-4204-b1cc-ebe4c610b941 service neutron] RBAC: Authorization granted {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:16:34.157111 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 905/1812] 10.222.0.82 () {60 vars in 1365 bytes} [Tue Aug 2 10:16:34 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 111 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 02 10:16:34.575672 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe4256d0-6358-4861-a445-7d197daf22cd None None] Authenticating user token {{(pid=80377) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:16:34.577854 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.fernet_utils [None req-fe4256d0-6358-4861-a445-7d197daf22cd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80377) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 10:16:34.594170 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe4256d0-6358-4861-a445-7d197daf22cd None None] Validating token access rules against request {{(pid=80377) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:16:34.600836 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bbbfebd-02ab-47f4-abab-b5773f037e00 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '891005fa5ff74041b75747d16cc66798', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7047912c5acd4adeba5d87c3bf895e39', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80377) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:16:34.601692 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bbbfebd-02ab-47f4-abab-b5773f037e00 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:16:34.602127 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bbbfebd-02ab-47f4-abab-b5773f037e00 admin admin] SCRIPT_NAME: `/identity` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:16:34.602529 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bbbfebd-02ab-47f4-abab-b5773f037e00 admin admin] PATH_INFO: `/v3/projects/c84b85209a1e44c98f302799f7b491a1` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:16:34.603869 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1bbbfebd-02ab-47f4-abab-b5773f037e00 admin admin] RBAC: Authorizing `identity:delete_project(project_id=c84b85209a1e44c98f302799f7b491a1)` {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:16:34.606334 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1bbbfebd-02ab-47f4-abab-b5773f037e00 admin admin] RBAC: Authorization granted {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:16:34.610559 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG dbcounter [-] [80377] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80377) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 10:16:34.688024 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.notifications [None req-1bbbfebd-02ab-47f4-abab-b5773f037e00 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'c84b85209a1e44c98f302799f7b491a1', 'request_id': 'req-1bbbfebd-02ab-47f4-abab-b5773f037e00'} {{(pid=80377) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 10:16:34.694998 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.notifications [None req-1bbbfebd-02ab-47f4-abab-b5773f037e00 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'c84b85209a1e44c98f302799f7b491a1', 'request_id': 'req-1bbbfebd-02ab-47f4-abab-b5773f037e00'} {{(pid=80377) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 10:16:34.701636 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.notifications [None req-1bbbfebd-02ab-47f4-abab-b5773f037e00 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'c84b85209a1e44c98f302799f7b491a1', 'request_id': 'req-1bbbfebd-02ab-47f4-abab-b5773f037e00'} {{(pid=80377) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 10:16:34.715291 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.notifications [None req-1bbbfebd-02ab-47f4-abab-b5773f037e00 admin admin] The token cache is being invalidate because project c84b85209a1e44c98f302799f7b491a1 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80377) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 02 10:16:34.715691 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.notifications [None req-1bbbfebd-02ab-47f4-abab-b5773f037e00 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80377) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 10:16:34.716422 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.notifications [None req-1bbbfebd-02ab-47f4-abab-b5773f037e00 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80377) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 10:16:34.718144 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: [pid: 80377|app: 0|req: 908/1813] 10.222.0.82 () {58 vars in 1233 bytes} [Tue Aug 2 10:16:34 2022] DELETE /identity/v3/projects/c84b85209a1e44c98f302799f7b491a1 => generated 0 bytes in 144 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 02 10:16:34.722195 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef140704-32dd-4204-b1cc-ebe4c610b941 service neutron] Authenticating user token {{(pid=80378) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:16:34.739161 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef140704-32dd-4204-b1cc-ebe4c610b941 service neutron] Validating token access rules against request {{(pid=80378) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:16:34.835283 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b521306-32d0-4d74-9922-26893439bb55 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '891005fa5ff74041b75747d16cc66798', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7047912c5acd4adeba5d87c3bf895e39', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80378) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:16:34.835991 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b521306-32d0-4d74-9922-26893439bb55 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:16:34.836150 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b521306-32d0-4d74-9922-26893439bb55 admin admin] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:16:34.836425 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b521306-32d0-4d74-9922-26893439bb55 admin admin] PATH_INFO: `/v3/users/aa9f81a3a6364fe3a7c7bdf693b37661` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:16:34.837634 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b521306-32d0-4d74-9922-26893439bb55 admin admin] RBAC: Authorizing `identity:delete_user(user_id=aa9f81a3a6364fe3a7c7bdf693b37661)` {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:16:34.839649 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b521306-32d0-4d74-9922-26893439bb55 admin admin] RBAC: Authorization granted {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:16:34.900498 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.notifications [None req-2b521306-32d0-4d74-9922-26893439bb55 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'aa9f81a3a6364fe3a7c7bdf693b37661'} {{(pid=80378) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 10:16:34.908682 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.notifications [None req-2b521306-32d0-4d74-9922-26893439bb55 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'aa9f81a3a6364fe3a7c7bdf693b37661'} {{(pid=80378) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 10:16:34.915925 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.notifications [None req-2b521306-32d0-4d74-9922-26893439bb55 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'aa9f81a3a6364fe3a7c7bdf693b37661'} {{(pid=80378) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 10:16:34.916566 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.notifications [None req-2b521306-32d0-4d74-9922-26893439bb55 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'aa9f81a3a6364fe3a7c7bdf693b37661'} {{(pid=80378) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 10:16:34.929671 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.notifications [None req-2b521306-32d0-4d74-9922-26893439bb55 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'aa9f81a3a6364fe3a7c7bdf693b37661'} {{(pid=80378) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 10:16:34.943763 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 906/1814] 10.222.0.82 () {58 vars in 1224 bytes} [Tue Aug 2 10:16:34 2022] DELETE /identity/v3/users/aa9f81a3a6364fe3a7c7bdf693b37661 => generated 0 bytes in 222 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 02 10:16:35.491693 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bbbfebd-02ab-47f4-abab-b5773f037e00 admin admin] Authenticating user token {{(pid=80377) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:16:35.493433 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.fernet_utils [None req-1bbbfebd-02ab-47f4-abab-b5773f037e00 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80377) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 10:16:35.511926 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bbbfebd-02ab-47f4-abab-b5773f037e00 admin admin] Validating token access rules against request {{(pid=80377) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:16:35.615149 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d103041-e1a1-472b-85be-9cbe283590fd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '891005fa5ff74041b75747d16cc66798', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7047912c5acd4adeba5d87c3bf895e39', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80377) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:16:35.615918 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d103041-e1a1-472b-85be-9cbe283590fd admin admin] REQUEST_METHOD: `DELETE` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:16:35.616275 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d103041-e1a1-472b-85be-9cbe283590fd admin admin] SCRIPT_NAME: `/identity` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:16:35.616622 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d103041-e1a1-472b-85be-9cbe283590fd admin admin] PATH_INFO: `/v3/projects/0fc50cd7647f43a7a33287a0b6bcf2da` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:16:35.617932 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d103041-e1a1-472b-85be-9cbe283590fd admin admin] RBAC: Authorizing `identity:delete_project(project_id=0fc50cd7647f43a7a33287a0b6bcf2da)` {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:16:35.619947 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d103041-e1a1-472b-85be-9cbe283590fd admin admin] RBAC: Authorization granted {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:16:35.703555 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.notifications [None req-7d103041-e1a1-472b-85be-9cbe283590fd admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '0fc50cd7647f43a7a33287a0b6bcf2da', 'request_id': 'req-7d103041-e1a1-472b-85be-9cbe283590fd'} {{(pid=80377) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 10:16:35.703555 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.notifications [None req-7d103041-e1a1-472b-85be-9cbe283590fd admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '0fc50cd7647f43a7a33287a0b6bcf2da', 'request_id': 'req-7d103041-e1a1-472b-85be-9cbe283590fd'} {{(pid=80377) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 10:16:35.720736 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.notifications [None req-7d103041-e1a1-472b-85be-9cbe283590fd admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '0fc50cd7647f43a7a33287a0b6bcf2da', 'request_id': 'req-7d103041-e1a1-472b-85be-9cbe283590fd'} {{(pid=80377) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 10:16:35.739132 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.notifications [None req-7d103041-e1a1-472b-85be-9cbe283590fd admin admin] The token cache is being invalidate because project 0fc50cd7647f43a7a33287a0b6bcf2da was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80377) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 02 10:16:35.739546 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.notifications [None req-7d103041-e1a1-472b-85be-9cbe283590fd admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80377) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 10:16:35.740591 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.notifications [None req-7d103041-e1a1-472b-85be-9cbe283590fd admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80377) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 10:16:35.747861 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c207825-2bef-48b1-969c-7903b00a0b3e None None] REQUEST_METHOD: `POST` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:16:35.748311 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c207825-2bef-48b1-969c-7903b00a0b3e None None] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:16:35.748655 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c207825-2bef-48b1-969c-7903b00a0b3e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:16:35.752745 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: [pid: 80377|app: 0|req: 909/1815] 10.222.0.82 () {58 vars in 1233 bytes} [Tue Aug 2 10:16:35 2022] DELETE /identity/v3/projects/0fc50cd7647f43a7a33287a0b6bcf2da => generated 0 bytes in 252 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 02 10:16:35.772374 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.auth.core [None req-3c207825-2bef-48b1-969c-7903b00a0b3e None None] MFA Rules not processed for user `891005fa5ff74041b75747d16cc66798`. Rule list: `[]` (Enabled: `True`). {{(pid=80378) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 02 10:16:35.773691 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.fernet_utils [None req-3c207825-2bef-48b1-969c-7903b00a0b3e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80378) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 10:16:35.808548 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG dbcounter [-] [80378] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80378) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 10:16:35.867340 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 907/1816] 10.222.0.82 () {58 vars in 958 bytes} [Tue Aug 2 10:16:35 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 121 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 02 10:16:35.875303 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79fa0ac1-14f5-4cc8-a888-a50a98943f45 None None] REQUEST_METHOD: `POST` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:16:35.875598 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79fa0ac1-14f5-4cc8-a888-a50a98943f45 None None] SCRIPT_NAME: `/identity` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:16:35.875877 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79fa0ac1-14f5-4cc8-a888-a50a98943f45 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:16:35.896664 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.auth.core [None req-79fa0ac1-14f5-4cc8-a888-a50a98943f45 None None] MFA Rules not processed for user `891005fa5ff74041b75747d16cc66798`. Rule list: `[]` (Enabled: `True`). {{(pid=80377) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 02 10:16:35.897993 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.fernet_utils [None req-79fa0ac1-14f5-4cc8-a888-a50a98943f45 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80377) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 10:16:35.909484 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: [pid: 80377|app: 0|req: 910/1817] 10.222.0.82 () {58 vars in 958 bytes} [Tue Aug 2 10:16:35 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 02 10:16:35.910987 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c207825-2bef-48b1-969c-7903b00a0b3e None None] Authenticating user token {{(pid=80378) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:16:35.921630 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c207825-2bef-48b1-969c-7903b00a0b3e None None] Validating token access rules against request {{(pid=80378) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:16:35.926146 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff5a5439-b868-4982-8a01-8a990303a337 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '891005fa5ff74041b75747d16cc66798', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7047912c5acd4adeba5d87c3bf895e39', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80378) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:16:35.927013 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff5a5439-b868-4982-8a01-8a990303a337 admin admin] REQUEST_METHOD: `GET` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:16:35.927290 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff5a5439-b868-4982-8a01-8a990303a337 admin admin] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:16:35.927561 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff5a5439-b868-4982-8a01-8a990303a337 admin admin] PATH_INFO: `/v3/domains` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:16:35.928249 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff5a5439-b868-4982-8a01-8a990303a337 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80378) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 02 10:16:35.928551 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff5a5439-b868-4982-8a01-8a990303a337 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:16:35.930166 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff5a5439-b868-4982-8a01-8a990303a337 admin admin] RBAC: Authorization granted {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:16:35.941682 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 908/1818] 10.222.0.82 () {58 vars in 1153 bytes} [Tue Aug 2 10:16:35 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 02 10:16:35.946146 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79fa0ac1-14f5-4cc8-a888-a50a98943f45 None None] Authenticating user token {{(pid=80377) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:16:35.955788 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79fa0ac1-14f5-4cc8-a888-a50a98943f45 None None] Validating token access rules against request {{(pid=80377) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:16:35.960447 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ac68875-268a-44b9-8724-8f3f6f20b68c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '891005fa5ff74041b75747d16cc66798', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7047912c5acd4adeba5d87c3bf895e39', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80377) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:16:35.961078 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ac68875-268a-44b9-8724-8f3f6f20b68c admin admin] REQUEST_METHOD: `POST` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:16:35.961837 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ac68875-268a-44b9-8724-8f3f6f20b68c admin admin] SCRIPT_NAME: `/identity` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:16:35.961837 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ac68875-268a-44b9-8724-8f3f6f20b68c admin admin] PATH_INFO: `/v3/projects` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:16:35.962321 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ac68875-268a-44b9-8724-8f3f6f20b68c admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:16:35.964224 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ac68875-268a-44b9-8724-8f3f6f20b68c admin admin] RBAC: Authorization granted {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:16:35.980095 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG dbcounter [-] [80377] Writing DB stats keystone:SELECT=84,keystone:DELETE=14,keystone:INSERT=2 {{(pid=80377) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 10:16:35.994387 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: [pid: 80377|app: 0|req: 911/1819] 10.222.0.82 () {60 vars in 1153 bytes} [Tue Aug 2 10:16:35 2022] POST /identity/v3/projects => generated 385 bytes in 49 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 02 10:16:35.998601 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff5a5439-b868-4982-8a01-8a990303a337 admin admin] Authenticating user token {{(pid=80378) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:16:36.008534 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff5a5439-b868-4982-8a01-8a990303a337 admin admin] Validating token access rules against request {{(pid=80378) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:16:36.065913 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86900b81-2cc4-400d-8245-33697a142f67 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '891005fa5ff74041b75747d16cc66798', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7047912c5acd4adeba5d87c3bf895e39', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80378) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:16:36.066543 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86900b81-2cc4-400d-8245-33697a142f67 admin admin] REQUEST_METHOD: `POST` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:16:36.066832 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86900b81-2cc4-400d-8245-33697a142f67 admin admin] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:16:36.067096 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86900b81-2cc4-400d-8245-33697a142f67 admin admin] PATH_INFO: `/v3/users` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:16:36.067764 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86900b81-2cc4-400d-8245-33697a142f67 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:16:36.069694 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86900b81-2cc4-400d-8245-33697a142f67 admin admin] RBAC: Authorization granted {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:16:36.092397 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 909/1820] 10.222.0.82 () {60 vars in 1144 bytes} [Tue Aug 2 10:16:35 2022] POST /identity/v3/users => generated 293 bytes in 94 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 02 10:16:36.096696 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ac68875-268a-44b9-8724-8f3f6f20b68c admin admin] Authenticating user token {{(pid=80377) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:16:36.107006 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ac68875-268a-44b9-8724-8f3f6f20b68c admin admin] Validating token access rules against request {{(pid=80377) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:16:36.111593 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d57b3fc-34b2-44c3-b0e6-19887da6b8ba admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '891005fa5ff74041b75747d16cc66798', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7047912c5acd4adeba5d87c3bf895e39', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80377) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:16:36.112213 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d57b3fc-34b2-44c3-b0e6-19887da6b8ba admin admin] REQUEST_METHOD: `GET` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:16:36.112500 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d57b3fc-34b2-44c3-b0e6-19887da6b8ba admin admin] SCRIPT_NAME: `/identity` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:16:36.112804 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d57b3fc-34b2-44c3-b0e6-19887da6b8ba admin admin] PATH_INFO: `/v3/roles` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:16:36.113501 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d57b3fc-34b2-44c3-b0e6-19887da6b8ba admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:16:36.115134 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d57b3fc-34b2-44c3-b0e6-19887da6b8ba admin admin] RBAC: Authorization granted {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:16:36.125834 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: [pid: 80377|app: 0|req: 912/1821] 10.222.0.82 () {58 vars in 1122 bytes} [Tue Aug 2 10:16:36 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 02 10:16:36.129286 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86900b81-2cc4-400d-8245-33697a142f67 admin admin] Authenticating user token {{(pid=80378) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:16:36.143405 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86900b81-2cc4-400d-8245-33697a142f67 admin admin] Validating token access rules against request {{(pid=80378) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:16:36.154236 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dad8107f-7940-4e0b-97c1-82818c0909d6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '891005fa5ff74041b75747d16cc66798', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7047912c5acd4adeba5d87c3bf895e39', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80378) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:16:36.154612 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dad8107f-7940-4e0b-97c1-82818c0909d6 admin admin] REQUEST_METHOD: `GET` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:16:36.154895 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dad8107f-7940-4e0b-97c1-82818c0909d6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:16:36.155168 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dad8107f-7940-4e0b-97c1-82818c0909d6 admin admin] PATH_INFO: `/v3/roles` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:16:36.155864 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dad8107f-7940-4e0b-97c1-82818c0909d6 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:16:36.157451 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dad8107f-7940-4e0b-97c1-82818c0909d6 admin admin] RBAC: Authorization granted {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:16:36.168646 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 910/1822] 10.222.0.82 () {58 vars in 1122 bytes} [Tue Aug 2 10:16:36 2022] GET /identity/v3/roles => generated 1420 bytes in 40 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 02 10:16:36.172984 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d57b3fc-34b2-44c3-b0e6-19887da6b8ba admin admin] Authenticating user token {{(pid=80377) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:16:36.183280 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d57b3fc-34b2-44c3-b0e6-19887da6b8ba admin admin] Validating token access rules against request {{(pid=80377) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:16:36.188129 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03740635-6815-4e67-8a91-f28d7ffb0487 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '891005fa5ff74041b75747d16cc66798', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7047912c5acd4adeba5d87c3bf895e39', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80377) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:16:36.188793 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03740635-6815-4e67-8a91-f28d7ffb0487 admin admin] REQUEST_METHOD: `PUT` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:16:36.189076 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03740635-6815-4e67-8a91-f28d7ffb0487 admin admin] SCRIPT_NAME: `/identity` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:16:36.189352 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03740635-6815-4e67-8a91-f28d7ffb0487 admin admin] PATH_INFO: `/v3/projects/b88e65e0895748b89c4156f4d00b4461/users/7e8d3d65b49440fab43627ca92c509ca/roles/e1c04b19fc4543b8b54f58bbcffbc598` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:16:36.204440 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03740635-6815-4e67-8a91-f28d7ffb0487 admin admin] RBAC: Authorizing `identity:create_grant(project_id=b88e65e0895748b89c4156f4d00b4461, user_id=7e8d3d65b49440fab43627ca92c509ca, role_id=e1c04b19fc4543b8b54f58bbcffbc598)` {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:16:36.207211 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03740635-6815-4e67-8a91-f28d7ffb0487 admin admin] RBAC: Authorization granted {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:16:36.224338 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: [pid: 80377|app: 0|req: 913/1823] 10.222.0.82 () {60 vars in 1483 bytes} [Tue Aug 2 10:16:36 2022] PUT /identity/v3/projects/b88e65e0895748b89c4156f4d00b4461/users/7e8d3d65b49440fab43627ca92c509ca/roles/e1c04b19fc4543b8b54f58bbcffbc598 => generated 0 bytes in 52 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 02 10:16:36.235026 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ee3c5f6-d3f3-42a0-b8ee-b9347bb2edec None None] REQUEST_METHOD: `POST` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:16:36.235320 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ee3c5f6-d3f3-42a0-b8ee-b9347bb2edec None None] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:16:36.235601 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ee3c5f6-d3f3-42a0-b8ee-b9347bb2edec None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:16:36.272649 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.auth.core [None req-5ee3c5f6-d3f3-42a0-b8ee-b9347bb2edec None None] MFA Rules not processed for user `7e8d3d65b49440fab43627ca92c509ca`. Rule list: `[]` (Enabled: `True`). {{(pid=80378) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 02 10:16:36.274221 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.fernet_utils [None req-5ee3c5f6-d3f3-42a0-b8ee-b9347bb2edec None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80378) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 10:16:36.362666 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 911/1824] 10.222.0.82 () {58 vars in 958 bytes} [Tue Aug 2 10:16:36 2022] POST /identity/v3/auth/tokens => generated 3262 bytes in 129 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 02 10:16:36.367212 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03740635-6815-4e67-8a91-f28d7ffb0487 admin admin] Authenticating user token {{(pid=80377) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:16:36.429615 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03740635-6815-4e67-8a91-f28d7ffb0487 admin admin] Validating token access rules against request {{(pid=80377) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:16:36.434235 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2597aa12-b808-4fd6-b857-90a6ae77769f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '891005fa5ff74041b75747d16cc66798', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7047912c5acd4adeba5d87c3bf895e39', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80377) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:16:36.434852 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2597aa12-b808-4fd6-b857-90a6ae77769f admin admin] REQUEST_METHOD: `POST` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:16:36.435129 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2597aa12-b808-4fd6-b857-90a6ae77769f admin admin] SCRIPT_NAME: `/identity` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:16:36.435402 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2597aa12-b808-4fd6-b857-90a6ae77769f admin admin] PATH_INFO: `/v3/projects` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:16:36.436072 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2597aa12-b808-4fd6-b857-90a6ae77769f admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:16:36.437997 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2597aa12-b808-4fd6-b857-90a6ae77769f admin admin] RBAC: Authorization granted {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:16:36.468890 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: [pid: 80377|app: 0|req: 914/1825] 10.222.0.82 () {60 vars in 1153 bytes} [Tue Aug 2 10:16:36 2022] POST /identity/v3/projects => generated 383 bytes in 98 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 02 10:16:36.470134 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ee3c5f6-d3f3-42a0-b8ee-b9347bb2edec None None] Authenticating user token {{(pid=80378) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:16:36.483821 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ee3c5f6-d3f3-42a0-b8ee-b9347bb2edec None None] Validating token access rules against request {{(pid=80378) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:16:36.518301 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG dbcounter [-] [80378] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80378) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 10:16:36.557824 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6ccd19d-7608-4e70-8966-6f74dafc8334 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '891005fa5ff74041b75747d16cc66798', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7047912c5acd4adeba5d87c3bf895e39', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80378) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:16:36.558528 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6ccd19d-7608-4e70-8966-6f74dafc8334 admin admin] REQUEST_METHOD: `POST` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:16:36.558869 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6ccd19d-7608-4e70-8966-6f74dafc8334 admin admin] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:16:36.559409 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6ccd19d-7608-4e70-8966-6f74dafc8334 admin admin] PATH_INFO: `/v3/users` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:16:36.560141 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6ccd19d-7608-4e70-8966-6f74dafc8334 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:16:36.562394 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6ccd19d-7608-4e70-8966-6f74dafc8334 admin admin] RBAC: Authorization granted {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:16:36.587860 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 912/1826] 10.222.0.82 () {60 vars in 1144 bytes} [Tue Aug 2 10:16:36 2022] POST /identity/v3/users => generated 298 bytes in 118 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 02 10:16:36.592382 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2597aa12-b808-4fd6-b857-90a6ae77769f admin admin] Authenticating user token {{(pid=80377) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:16:36.615498 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2597aa12-b808-4fd6-b857-90a6ae77769f admin admin] Validating token access rules against request {{(pid=80377) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:16:36.622629 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98eec74a-dd9a-45c7-8645-e9da4fd91e09 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '891005fa5ff74041b75747d16cc66798', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7047912c5acd4adeba5d87c3bf895e39', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80377) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:16:36.623272 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98eec74a-dd9a-45c7-8645-e9da4fd91e09 admin admin] REQUEST_METHOD: `GET` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:16:36.623559 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98eec74a-dd9a-45c7-8645-e9da4fd91e09 admin admin] SCRIPT_NAME: `/identity` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:16:36.623829 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98eec74a-dd9a-45c7-8645-e9da4fd91e09 admin admin] PATH_INFO: `/v3/roles` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:16:36.624544 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98eec74a-dd9a-45c7-8645-e9da4fd91e09 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:16:36.626358 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98eec74a-dd9a-45c7-8645-e9da4fd91e09 admin admin] RBAC: Authorization granted {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:16:36.644811 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: [pid: 80377|app: 0|req: 915/1827] 10.222.0.82 () {58 vars in 1122 bytes} [Tue Aug 2 10:16:36 2022] GET /identity/v3/roles => generated 1420 bytes in 53 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 02 10:16:36.651124 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6ccd19d-7608-4e70-8966-6f74dafc8334 admin admin] Authenticating user token {{(pid=80378) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:16:36.667913 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6ccd19d-7608-4e70-8966-6f74dafc8334 admin admin] Validating token access rules against request {{(pid=80378) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:16:36.673636 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d05cd6a9-10fa-444b-9cc9-dfd984e90837 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '891005fa5ff74041b75747d16cc66798', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7047912c5acd4adeba5d87c3bf895e39', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80378) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:16:36.674445 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d05cd6a9-10fa-444b-9cc9-dfd984e90837 admin admin] REQUEST_METHOD: `PUT` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:16:36.674808 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d05cd6a9-10fa-444b-9cc9-dfd984e90837 admin admin] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:16:36.675177 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d05cd6a9-10fa-444b-9cc9-dfd984e90837 admin admin] PATH_INFO: `/v3/projects/1aa245953e8e4a25943016810bb93904/users/edc86e1645d742d48a50e74e9060cd6e/roles/237966442f05411191105bf336bc14f3` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:16:36.694301 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d05cd6a9-10fa-444b-9cc9-dfd984e90837 admin admin] RBAC: Authorizing `identity:create_grant(project_id=1aa245953e8e4a25943016810bb93904, user_id=edc86e1645d742d48a50e74e9060cd6e, role_id=237966442f05411191105bf336bc14f3)` {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:16:36.697095 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d05cd6a9-10fa-444b-9cc9-dfd984e90837 admin admin] RBAC: Authorization granted {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:16:36.723634 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 913/1828] 10.222.0.82 () {60 vars in 1483 bytes} [Tue Aug 2 10:16:36 2022] PUT /identity/v3/projects/1aa245953e8e4a25943016810bb93904/users/edc86e1645d742d48a50e74e9060cd6e/roles/237966442f05411191105bf336bc14f3 => generated 0 bytes in 73 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 02 10:16:36.727811 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98eec74a-dd9a-45c7-8645-e9da4fd91e09 admin admin] Authenticating user token {{(pid=80377) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:16:36.738539 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98eec74a-dd9a-45c7-8645-e9da4fd91e09 admin admin] Validating token access rules against request {{(pid=80377) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:16:36.823613 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbcee521-7a59-4c58-9905-1edfbd8006aa admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '891005fa5ff74041b75747d16cc66798', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7047912c5acd4adeba5d87c3bf895e39', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80377) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:16:36.824894 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbcee521-7a59-4c58-9905-1edfbd8006aa admin admin] REQUEST_METHOD: `GET` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:16:36.825981 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbcee521-7a59-4c58-9905-1edfbd8006aa admin admin] SCRIPT_NAME: `/identity` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:16:36.826548 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbcee521-7a59-4c58-9905-1edfbd8006aa admin admin] PATH_INFO: `/v3/roles` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:16:36.827659 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cbcee521-7a59-4c58-9905-1edfbd8006aa admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:16:36.829461 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cbcee521-7a59-4c58-9905-1edfbd8006aa admin admin] RBAC: Authorization granted {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:16:36.840527 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: [pid: 80377|app: 0|req: 916/1829] 10.222.0.82 () {58 vars in 1122 bytes} [Tue Aug 2 10:16:36 2022] GET /identity/v3/roles => generated 1420 bytes in 113 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 02 10:16:36.845330 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d05cd6a9-10fa-444b-9cc9-dfd984e90837 admin admin] Authenticating user token {{(pid=80378) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:16:36.857474 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d05cd6a9-10fa-444b-9cc9-dfd984e90837 admin admin] Validating token access rules against request {{(pid=80378) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:16:36.863004 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fc75144-64cd-4f85-a0e9-b015302e22d9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '891005fa5ff74041b75747d16cc66798', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7047912c5acd4adeba5d87c3bf895e39', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80378) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:16:36.863700 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fc75144-64cd-4f85-a0e9-b015302e22d9 admin admin] REQUEST_METHOD: `PUT` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:16:36.863986 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fc75144-64cd-4f85-a0e9-b015302e22d9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:16:36.864264 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fc75144-64cd-4f85-a0e9-b015302e22d9 admin admin] PATH_INFO: `/v3/projects/1aa245953e8e4a25943016810bb93904/users/edc86e1645d742d48a50e74e9060cd6e/roles/237966442f05411191105bf336bc14f3` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:16:36.866462 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6fc75144-64cd-4f85-a0e9-b015302e22d9 admin admin] RBAC: Authorizing `identity:create_grant(project_id=1aa245953e8e4a25943016810bb93904, user_id=edc86e1645d742d48a50e74e9060cd6e, role_id=237966442f05411191105bf336bc14f3)` {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:16:36.869581 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6fc75144-64cd-4f85-a0e9-b015302e22d9 admin admin] RBAC: Authorization granted {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:16:36.889444 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 914/1830] 10.222.0.82 () {60 vars in 1483 bytes} [Tue Aug 2 10:16:36 2022] PUT /identity/v3/projects/1aa245953e8e4a25943016810bb93904/users/edc86e1645d742d48a50e74e9060cd6e/roles/237966442f05411191105bf336bc14f3 => generated 0 bytes in 45 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 02 10:16:36.901550 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0621ba90-60fd-47c4-897d-a26e63578a6a None None] REQUEST_METHOD: `POST` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:16:36.901898 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0621ba90-60fd-47c4-897d-a26e63578a6a None None] SCRIPT_NAME: `/identity` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:16:36.902181 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0621ba90-60fd-47c4-897d-a26e63578a6a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:16:36.937241 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.auth.core [None req-0621ba90-60fd-47c4-897d-a26e63578a6a None None] MFA Rules not processed for user `edc86e1645d742d48a50e74e9060cd6e`. Rule list: `[]` (Enabled: `True`). {{(pid=80377) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 02 10:16:36.938799 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.fernet_utils [None req-0621ba90-60fd-47c4-897d-a26e63578a6a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80377) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 10:16:36.976437 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG dbcounter [-] [80377] Writing DB stats keystone:INSERT=3,keystone:SELECT=97 {{(pid=80377) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 10:16:37.092308 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: [pid: 80377|app: 0|req: 917/1831] 10.222.0.82 () {58 vars in 958 bytes} [Tue Aug 2 10:16:36 2022] POST /identity/v3/auth/tokens => generated 3327 bytes in 192 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 02 10:16:37.096863 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fc75144-64cd-4f85-a0e9-b015302e22d9 admin admin] Authenticating user token {{(pid=80378) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:16:37.098596 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.fernet_utils [None req-6fc75144-64cd-4f85-a0e9-b015302e22d9 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80378) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 10:16:37.197354 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fc75144-64cd-4f85-a0e9-b015302e22d9 admin admin] Validating token access rules against request {{(pid=80378) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:16:37.202143 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58838795-6609-49ba-835d-2ea1352994cb service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1023de281ae74e9d8a6d6c312c1c97a7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f1b38ebb804c41c69772c998aae72c41', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80378) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:16:37.202786 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58838795-6609-49ba-835d-2ea1352994cb service cinder] REQUEST_METHOD: `GET` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:16:37.203068 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58838795-6609-49ba-835d-2ea1352994cb service cinder] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:16:37.203342 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58838795-6609-49ba-835d-2ea1352994cb service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:16:37.263456 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58838795-6609-49ba-835d-2ea1352994cb service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:16:37.264947 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58838795-6609-49ba-835d-2ea1352994cb service cinder] RBAC: Authorization granted {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:16:37.274861 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 915/1832] 10.222.0.82 () {60 vars in 1365 bytes} [Tue Aug 2 10:16:37 2022] GET /identity/v3/auth/tokens => generated 3262 bytes in 178 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 02 10:16:39.265694 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0621ba90-60fd-47c4-897d-a26e63578a6a None None] Authenticating user token {{(pid=80377) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:16:39.277909 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0621ba90-60fd-47c4-897d-a26e63578a6a None None] Validating token access rules against request {{(pid=80377) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:16:39.281459 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74a1d64d-6214-49f6-a0e6-b2ab58325584 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1023de281ae74e9d8a6d6c312c1c97a7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f1b38ebb804c41c69772c998aae72c41', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80377) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:16:39.282124 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74a1d64d-6214-49f6-a0e6-b2ab58325584 service cinder] REQUEST_METHOD: `GET` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:16:39.282412 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74a1d64d-6214-49f6-a0e6-b2ab58325584 service cinder] SCRIPT_NAME: `/identity` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:16:39.282690 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74a1d64d-6214-49f6-a0e6-b2ab58325584 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:16:39.284478 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.fernet_utils [None req-74a1d64d-6214-49f6-a0e6-b2ab58325584 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80377) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 10:16:39.361458 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74a1d64d-6214-49f6-a0e6-b2ab58325584 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:16:39.363162 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74a1d64d-6214-49f6-a0e6-b2ab58325584 service cinder] RBAC: Authorization granted {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:16:39.375655 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: [pid: 80377|app: 0|req: 918/1833] 10.222.0.82 () {60 vars in 1365 bytes} [Tue Aug 2 10:16:39 2022] GET /identity/v3/auth/tokens => generated 3319 bytes in 110 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 02 10:16:44.760407 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cc5c45f-622f-4824-90d9-1e78416b764d None None] REQUEST_METHOD: `POST` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:16:44.761595 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cc5c45f-622f-4824-90d9-1e78416b764d None None] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:16:44.762190 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cc5c45f-622f-4824-90d9-1e78416b764d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:16:44.783432 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.auth.core [None req-8cc5c45f-622f-4824-90d9-1e78416b764d None None] MFA Rules not processed for user `33b87170f0634b67bce60750847b5cbe`. Rule list: `[]` (Enabled: `True`). {{(pid=80378) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 02 10:16:44.784614 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.fernet_utils [None req-8cc5c45f-622f-4824-90d9-1e78416b764d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80378) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 10:16:44.817964 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG dbcounter [-] [80378] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80378) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 10:16:44.843143 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 916/1834] 10.222.0.79 () {58 vars in 1015 bytes} [Tue Aug 2 10:16:44 2022] POST /identity/v3/auth/tokens => generated 3121 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 02 10:16:44.871550 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74a1d64d-6214-49f6-a0e6-b2ab58325584 service cinder] Authenticating user token {{(pid=80377) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:16:44.873461 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.fernet_utils [None req-74a1d64d-6214-49f6-a0e6-b2ab58325584 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80377) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 10:16:44.892705 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74a1d64d-6214-49f6-a0e6-b2ab58325584 service cinder] Validating token access rules against request {{(pid=80377) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:16:44.896535 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7501e405-6423-433b-98c3-2e1ba6116f07 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '33b87170f0634b67bce60750847b5cbe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f1b38ebb804c41c69772c998aae72c41', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80377) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:16:44.897167 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7501e405-6423-433b-98c3-2e1ba6116f07 service neutron] REQUEST_METHOD: `GET` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:16:44.897450 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7501e405-6423-433b-98c3-2e1ba6116f07 service neutron] SCRIPT_NAME: `/identity` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:16:44.897728 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7501e405-6423-433b-98c3-2e1ba6116f07 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:16:44.905771 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7501e405-6423-433b-98c3-2e1ba6116f07 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:16:44.907258 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7501e405-6423-433b-98c3-2e1ba6116f07 service neutron] RBAC: Authorization granted {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:16:44.915834 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: [pid: 80377|app: 0|req: 919/1835] 10.222.0.82 () {60 vars in 1365 bytes} [Tue Aug 2 10:16:44 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 45 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 02 10:16:46.168726 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-353b04ff-42e3-4017-b11b-1dd53840ff2a None None] REQUEST_METHOD: `GET` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:16:46.169423 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-353b04ff-42e3-4017-b11b-1dd53840ff2a None None] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:16:46.169571 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-353b04ff-42e3-4017-b11b-1dd53840ff2a None None] PATH_INFO: `/` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:16:46.171328 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 917/1836] 10.222.0.82 () {54 vars in 921 bytes} [Tue Aug 2 10:16:46 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 02 10:16:46.186471 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28bcb011-516a-4128-a587-d01ef730944a None None] REQUEST_METHOD: `POST` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:16:46.187158 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28bcb011-516a-4128-a587-d01ef730944a None None] SCRIPT_NAME: `/identity` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:16:46.187158 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28bcb011-516a-4128-a587-d01ef730944a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:16:46.224286 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.auth.core [None req-28bcb011-516a-4128-a587-d01ef730944a None None] MFA Rules not processed for user `88af518196674fe593ad88d30111fb7d`. Rule list: `[]` (Enabled: `True`). {{(pid=80377) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 02 10:16:46.230872 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.fernet_utils [None req-28bcb011-516a-4128-a587-d01ef730944a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80377) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 10:16:46.350021 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: [pid: 80377|app: 0|req: 920/1837] 10.222.0.82 () {58 vars in 1018 bytes} [Tue Aug 2 10:16:46 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 165 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 02 10:16:46.365983 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-353b04ff-42e3-4017-b11b-1dd53840ff2a None None] Authenticating user token {{(pid=80378) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:16:46.367905 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.fernet_utils [None req-353b04ff-42e3-4017-b11b-1dd53840ff2a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80378) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 10:16:46.391852 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-353b04ff-42e3-4017-b11b-1dd53840ff2a None None] Validating token access rules against request {{(pid=80378) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:16:46.399209 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c4852af-0070-434f-a882-1d4815931179 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88af518196674fe593ad88d30111fb7d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f1b38ebb804c41c69772c998aae72c41', 'project_domain_id': 'default', 'roles': ['admin', 'ResellerAdmin', 'reader', 'service', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80378) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:16:46.400081 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c4852af-0070-434f-a882-1d4815931179 service nova] REQUEST_METHOD: `GET` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:16:46.400429 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c4852af-0070-434f-a882-1d4815931179 service nova] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:16:46.400775 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c4852af-0070-434f-a882-1d4815931179 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:16:46.411247 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c4852af-0070-434f-a882-1d4815931179 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:16:46.413615 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c4852af-0070-434f-a882-1d4815931179 service nova] RBAC: Authorization granted {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:16:46.426716 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 918/1838] 10.222.0.82 () {60 vars in 1365 bytes} [Tue Aug 2 10:16:46 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 61 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 02 10:16:47.242488 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28bcb011-516a-4128-a587-d01ef730944a None None] Authenticating user token {{(pid=80377) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:16:47.245342 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.fernet_utils [None req-28bcb011-516a-4128-a587-d01ef730944a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80377) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 10:16:47.376126 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG dbcounter [-] [80377] Writing DB stats keystone:SELECT=100 {{(pid=80377) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 10:16:47.545556 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28bcb011-516a-4128-a587-d01ef730944a None None] Validating token access rules against request {{(pid=80377) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:16:47.558965 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5cb97b7c-6646-42e3-bca0-7965c6b6b991 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '891005fa5ff74041b75747d16cc66798', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7047912c5acd4adeba5d87c3bf895e39', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80377) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:16:47.560406 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cb97b7c-6646-42e3-bca0-7965c6b6b991 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:16:47.560748 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cb97b7c-6646-42e3-bca0-7965c6b6b991 admin admin] SCRIPT_NAME: `/identity` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:16:47.561069 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cb97b7c-6646-42e3-bca0-7965c6b6b991 admin admin] PATH_INFO: `/v3/users/aabf3ada63b74bdb925096e197e6ecdd` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:16:47.565059 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5cb97b7c-6646-42e3-bca0-7965c6b6b991 admin admin] RBAC: Authorizing `identity:delete_user(user_id=aabf3ada63b74bdb925096e197e6ecdd)` {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:16:47.573969 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5cb97b7c-6646-42e3-bca0-7965c6b6b991 admin admin] RBAC: Authorization granted {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:16:47.743141 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.notifications [None req-5cb97b7c-6646-42e3-bca0-7965c6b6b991 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'aabf3ada63b74bdb925096e197e6ecdd'} {{(pid=80377) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 10:16:47.762000 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.notifications [None req-5cb97b7c-6646-42e3-bca0-7965c6b6b991 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'aabf3ada63b74bdb925096e197e6ecdd'} {{(pid=80377) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 10:16:47.787722 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.notifications [None req-5cb97b7c-6646-42e3-bca0-7965c6b6b991 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'aabf3ada63b74bdb925096e197e6ecdd'} {{(pid=80377) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 10:16:47.789579 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.notifications [None req-5cb97b7c-6646-42e3-bca0-7965c6b6b991 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'aabf3ada63b74bdb925096e197e6ecdd'} {{(pid=80377) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 10:16:47.832557 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.notifications [None req-5cb97b7c-6646-42e3-bca0-7965c6b6b991 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'aabf3ada63b74bdb925096e197e6ecdd'} {{(pid=80377) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 10:16:47.843729 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: [pid: 80377|app: 0|req: 921/1839] 10.222.0.82 () {58 vars in 1224 bytes} [Tue Aug 2 10:16:47 2022] DELETE /identity/v3/users/aabf3ada63b74bdb925096e197e6ecdd => generated 0 bytes in 602 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 02 10:16:47.858971 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8a44fd2-67db-42d2-ab66-edc031fb1331 None None] REQUEST_METHOD: `POST` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:16:47.859387 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8a44fd2-67db-42d2-ab66-edc031fb1331 None None] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:16:47.859545 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8a44fd2-67db-42d2-ab66-edc031fb1331 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:16:47.895328 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.auth.core [None req-b8a44fd2-67db-42d2-ab66-edc031fb1331 None None] MFA Rules not processed for user `891005fa5ff74041b75747d16cc66798`. Rule list: `[]` (Enabled: `True`). {{(pid=80378) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 02 10:16:47.896852 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.fernet_utils [None req-b8a44fd2-67db-42d2-ab66-edc031fb1331 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80378) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 10:16:48.049076 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 919/1840] 10.222.0.82 () {58 vars in 958 bytes} [Tue Aug 2 10:16:47 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 201 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 02 10:16:48.084571 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5cb97b7c-6646-42e3-bca0-7965c6b6b991 admin admin] Authenticating user token {{(pid=80377) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:16:48.086084 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.fernet_utils [None req-5cb97b7c-6646-42e3-bca0-7965c6b6b991 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80377) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 10:16:48.208745 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5cb97b7c-6646-42e3-bca0-7965c6b6b991 admin admin] Validating token access rules against request {{(pid=80377) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:16:48.215344 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88965f23-5b09-463b-805c-f9a5adf1c351 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '33b87170f0634b67bce60750847b5cbe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f1b38ebb804c41c69772c998aae72c41', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80377) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:16:48.217736 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88965f23-5b09-463b-805c-f9a5adf1c351 service neutron] REQUEST_METHOD: `GET` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:16:48.218529 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88965f23-5b09-463b-805c-f9a5adf1c351 service neutron] SCRIPT_NAME: `/identity` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:16:48.219132 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88965f23-5b09-463b-805c-f9a5adf1c351 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:16:48.234837 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88965f23-5b09-463b-805c-f9a5adf1c351 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:16:48.246756 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88965f23-5b09-463b-805c-f9a5adf1c351 service neutron] RBAC: Authorization granted {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:16:48.252024 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: [pid: 80377|app: 0|req: 922/1841] 10.222.0.82 () {60 vars in 1365 bytes} [Tue Aug 2 10:16:48 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 168 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 02 10:16:48.437453 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8a44fd2-67db-42d2-ab66-edc031fb1331 None None] Authenticating user token {{(pid=80378) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:16:48.438927 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.fernet_utils [None req-b8a44fd2-67db-42d2-ab66-edc031fb1331 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80378) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 10:16:48.508139 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8a44fd2-67db-42d2-ab66-edc031fb1331 None None] Validating token access rules against request {{(pid=80378) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:16:48.512709 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1dcaeec6-ce71-447b-9b1f-b5056facfdd5 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1023de281ae74e9d8a6d6c312c1c97a7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f1b38ebb804c41c69772c998aae72c41', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80378) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:16:48.514079 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1dcaeec6-ce71-447b-9b1f-b5056facfdd5 service cinder] REQUEST_METHOD: `GET` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:16:48.514602 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1dcaeec6-ce71-447b-9b1f-b5056facfdd5 service cinder] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:16:48.515094 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1dcaeec6-ce71-447b-9b1f-b5056facfdd5 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:16:48.517947 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.fernet_utils [None req-1dcaeec6-ce71-447b-9b1f-b5056facfdd5 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80378) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 10:16:48.611364 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1dcaeec6-ce71-447b-9b1f-b5056facfdd5 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:16:48.613843 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1dcaeec6-ce71-447b-9b1f-b5056facfdd5 service cinder] RBAC: Authorization granted {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:16:48.627477 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 920/1842] 10.222.0.82 () {60 vars in 1365 bytes} [Tue Aug 2 10:16:48 2022] GET /identity/v3/auth/tokens => generated 3327 bytes in 191 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 02 10:16:48.764535 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14db436c-9a5c-4dc3-b4fe-0d3abf0701cd None None] REQUEST_METHOD: `GET` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:16:48.764818 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14db436c-9a5c-4dc3-b4fe-0d3abf0701cd None None] SCRIPT_NAME: `/identity` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:16:48.765102 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14db436c-9a5c-4dc3-b4fe-0d3abf0701cd None None] PATH_INFO: `/` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:16:48.766642 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: [pid: 80377|app: 0|req: 923/1843] 10.222.0.82 () {54 vars in 921 bytes} [Tue Aug 2 10:16:48 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 02 10:16:48.775322 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55f44096-fff9-4d1e-b292-7c0fa58ebcfc None None] REQUEST_METHOD: `POST` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:16:48.777054 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55f44096-fff9-4d1e-b292-7c0fa58ebcfc None None] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:16:48.777625 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55f44096-fff9-4d1e-b292-7c0fa58ebcfc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:16:48.814531 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.auth.core [None req-55f44096-fff9-4d1e-b292-7c0fa58ebcfc None None] MFA Rules not processed for user `88af518196674fe593ad88d30111fb7d`. Rule list: `[]` (Enabled: `True`). {{(pid=80378) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 02 10:16:48.816648 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.fernet_utils [None req-55f44096-fff9-4d1e-b292-7c0fa58ebcfc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80378) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 10:16:48.841822 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG dbcounter [-] [80378] Writing DB stats keystone:SELECT=100 {{(pid=80378) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 10:16:48.920996 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 921/1844] 10.222.0.82 () {58 vars in 1018 bytes} [Tue Aug 2 10:16:48 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 149 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 02 10:16:48.933191 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14db436c-9a5c-4dc3-b4fe-0d3abf0701cd None None] Authenticating user token {{(pid=80377) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:16:48.936313 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.fernet_utils [None req-14db436c-9a5c-4dc3-b4fe-0d3abf0701cd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80377) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 10:16:48.957934 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14db436c-9a5c-4dc3-b4fe-0d3abf0701cd None None] Validating token access rules against request {{(pid=80377) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:16:48.964033 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-612e62d9-5234-4f04-a2ae-d6046d86c6da service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88af518196674fe593ad88d30111fb7d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f1b38ebb804c41c69772c998aae72c41', 'project_domain_id': 'default', 'roles': ['admin', 'ResellerAdmin', 'reader', 'service', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80377) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:16:48.965049 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-612e62d9-5234-4f04-a2ae-d6046d86c6da service nova] REQUEST_METHOD: `GET` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:16:48.965533 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-612e62d9-5234-4f04-a2ae-d6046d86c6da service nova] SCRIPT_NAME: `/identity` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:16:48.966082 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-612e62d9-5234-4f04-a2ae-d6046d86c6da service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:16:48.977842 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-612e62d9-5234-4f04-a2ae-d6046d86c6da service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:16:48.979226 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-612e62d9-5234-4f04-a2ae-d6046d86c6da service nova] RBAC: Authorization granted {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:16:48.992916 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: [pid: 80377|app: 0|req: 924/1845] 10.222.0.82 () {60 vars in 1365 bytes} [Tue Aug 2 10:16:48 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 60 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 02 10:16:49.609572 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55f44096-fff9-4d1e-b292-7c0fa58ebcfc None None] Authenticating user token {{(pid=80378) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:16:49.611818 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.fernet_utils [None req-55f44096-fff9-4d1e-b292-7c0fa58ebcfc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80378) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 10:16:49.634148 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55f44096-fff9-4d1e-b292-7c0fa58ebcfc None None] Validating token access rules against request {{(pid=80378) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:16:49.639846 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83dcd326-097a-4cdd-a42b-06b7bf900233 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '891005fa5ff74041b75747d16cc66798', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7047912c5acd4adeba5d87c3bf895e39', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80378) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:16:49.640719 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83dcd326-097a-4cdd-a42b-06b7bf900233 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:16:49.641484 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83dcd326-097a-4cdd-a42b-06b7bf900233 admin admin] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:16:49.641778 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83dcd326-097a-4cdd-a42b-06b7bf900233 admin admin] PATH_INFO: `/v3/projects/321ebe0084074e3cab37facfacd53607` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:16:49.643008 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83dcd326-097a-4cdd-a42b-06b7bf900233 admin admin] RBAC: Authorizing `identity:delete_project(project_id=321ebe0084074e3cab37facfacd53607)` {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:16:49.644930 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83dcd326-097a-4cdd-a42b-06b7bf900233 admin admin] RBAC: Authorization granted {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:16:49.703262 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.notifications [None req-83dcd326-097a-4cdd-a42b-06b7bf900233 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '321ebe0084074e3cab37facfacd53607', 'request_id': 'req-83dcd326-097a-4cdd-a42b-06b7bf900233'} {{(pid=80378) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 10:16:49.713244 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.notifications [None req-83dcd326-097a-4cdd-a42b-06b7bf900233 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '321ebe0084074e3cab37facfacd53607', 'request_id': 'req-83dcd326-097a-4cdd-a42b-06b7bf900233'} {{(pid=80378) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 10:16:49.722045 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.notifications [None req-83dcd326-097a-4cdd-a42b-06b7bf900233 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '321ebe0084074e3cab37facfacd53607', 'request_id': 'req-83dcd326-097a-4cdd-a42b-06b7bf900233'} {{(pid=80378) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 10:16:49.738875 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.notifications [None req-83dcd326-097a-4cdd-a42b-06b7bf900233 admin admin] The token cache is being invalidate because project 321ebe0084074e3cab37facfacd53607 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80378) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 02 10:16:49.739175 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.notifications [None req-83dcd326-097a-4cdd-a42b-06b7bf900233 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80378) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 10:16:49.739839 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.notifications [None req-83dcd326-097a-4cdd-a42b-06b7bf900233 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80378) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 10:16:49.741497 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 922/1846] 10.222.0.82 () {58 vars in 1233 bytes} [Tue Aug 2 10:16:49 2022] DELETE /identity/v3/projects/321ebe0084074e3cab37facfacd53607 => generated 0 bytes in 133 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 02 10:16:49.745564 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-612e62d9-5234-4f04-a2ae-d6046d86c6da service nova] Authenticating user token {{(pid=80377) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:16:49.757061 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-612e62d9-5234-4f04-a2ae-d6046d86c6da service nova] Validating token access rules against request {{(pid=80377) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:16:49.798081 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG dbcounter [-] [80377] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80377) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 10:16:49.826305 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e81caac-ce07-4f4d-a135-edf0583a2f9d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '891005fa5ff74041b75747d16cc66798', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7047912c5acd4adeba5d87c3bf895e39', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80377) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:16:49.827012 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e81caac-ce07-4f4d-a135-edf0583a2f9d admin admin] REQUEST_METHOD: `DELETE` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:16:49.827548 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e81caac-ce07-4f4d-a135-edf0583a2f9d admin admin] SCRIPT_NAME: `/identity` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:16:49.827836 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e81caac-ce07-4f4d-a135-edf0583a2f9d admin admin] PATH_INFO: `/v3/users/6edab13f3c094507ba5857e05b8f6467` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:16:49.829091 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e81caac-ce07-4f4d-a135-edf0583a2f9d admin admin] RBAC: Authorizing `identity:delete_user(user_id=6edab13f3c094507ba5857e05b8f6467)` {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:16:49.831423 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e81caac-ce07-4f4d-a135-edf0583a2f9d admin admin] RBAC: Authorization granted {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:16:49.904484 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.notifications [None req-9e81caac-ce07-4f4d-a135-edf0583a2f9d admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '6edab13f3c094507ba5857e05b8f6467'} {{(pid=80377) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 10:16:49.914974 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.notifications [None req-9e81caac-ce07-4f4d-a135-edf0583a2f9d admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '6edab13f3c094507ba5857e05b8f6467'} {{(pid=80377) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 10:16:49.926096 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.notifications [None req-9e81caac-ce07-4f4d-a135-edf0583a2f9d admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '6edab13f3c094507ba5857e05b8f6467'} {{(pid=80377) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 10:16:49.927333 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.notifications [None req-9e81caac-ce07-4f4d-a135-edf0583a2f9d admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '6edab13f3c094507ba5857e05b8f6467'} {{(pid=80377) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 10:16:49.956206 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.notifications [None req-9e81caac-ce07-4f4d-a135-edf0583a2f9d admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '6edab13f3c094507ba5857e05b8f6467'} {{(pid=80377) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 10:16:49.964919 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: [pid: 80377|app: 0|req: 925/1847] 10.222.0.82 () {58 vars in 1224 bytes} [Tue Aug 2 10:16:49 2022] DELETE /identity/v3/users/6edab13f3c094507ba5857e05b8f6467 => generated 0 bytes in 220 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 02 10:16:50.366129 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83dcd326-097a-4cdd-a42b-06b7bf900233 admin admin] Authenticating user token {{(pid=80378) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:16:50.367600 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.fernet_utils [None req-83dcd326-097a-4cdd-a42b-06b7bf900233 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80378) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 10:16:50.384879 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83dcd326-097a-4cdd-a42b-06b7bf900233 admin admin] Validating token access rules against request {{(pid=80378) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:16:50.487379 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11a40fa3-97f5-4681-a84e-adeea19d2171 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '891005fa5ff74041b75747d16cc66798', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7047912c5acd4adeba5d87c3bf895e39', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80378) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:16:50.488042 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11a40fa3-97f5-4681-a84e-adeea19d2171 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:16:50.488321 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11a40fa3-97f5-4681-a84e-adeea19d2171 admin admin] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:16:50.488709 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11a40fa3-97f5-4681-a84e-adeea19d2171 admin admin] PATH_INFO: `/v3/projects/f46d6a6eea6c4f318667ca8f93f032e5` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:16:50.489943 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11a40fa3-97f5-4681-a84e-adeea19d2171 admin admin] RBAC: Authorizing `identity:delete_project(project_id=f46d6a6eea6c4f318667ca8f93f032e5)` {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:16:50.492743 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11a40fa3-97f5-4681-a84e-adeea19d2171 admin admin] RBAC: Authorization granted {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:16:50.547400 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG dbcounter [-] [80378] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80378) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 10:16:50.560124 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.notifications [None req-11a40fa3-97f5-4681-a84e-adeea19d2171 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'f46d6a6eea6c4f318667ca8f93f032e5', 'request_id': 'req-11a40fa3-97f5-4681-a84e-adeea19d2171'} {{(pid=80378) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 10:16:50.568270 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.notifications [None req-11a40fa3-97f5-4681-a84e-adeea19d2171 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'f46d6a6eea6c4f318667ca8f93f032e5', 'request_id': 'req-11a40fa3-97f5-4681-a84e-adeea19d2171'} {{(pid=80378) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 10:16:50.576415 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.notifications [None req-11a40fa3-97f5-4681-a84e-adeea19d2171 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'f46d6a6eea6c4f318667ca8f93f032e5', 'request_id': 'req-11a40fa3-97f5-4681-a84e-adeea19d2171'} {{(pid=80378) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 10:16:50.596015 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.notifications [None req-11a40fa3-97f5-4681-a84e-adeea19d2171 admin admin] The token cache is being invalidate because project f46d6a6eea6c4f318667ca8f93f032e5 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80378) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 02 10:16:50.597018 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.notifications [None req-11a40fa3-97f5-4681-a84e-adeea19d2171 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80378) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 10:16:50.598128 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.notifications [None req-11a40fa3-97f5-4681-a84e-adeea19d2171 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80378) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 10:16:50.601580 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 923/1848] 10.222.0.82 () {58 vars in 1233 bytes} [Tue Aug 2 10:16:50 2022] DELETE /identity/v3/projects/f46d6a6eea6c4f318667ca8f93f032e5 => generated 0 bytes in 236 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 02 10:16:50.606270 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09cb1849-1ae2-45ba-bc87-8bf098861b90 None None] REQUEST_METHOD: `POST` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:16:50.607823 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09cb1849-1ae2-45ba-bc87-8bf098861b90 None None] SCRIPT_NAME: `/identity` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:16:50.608164 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09cb1849-1ae2-45ba-bc87-8bf098861b90 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:16:50.634096 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.auth.core [None req-09cb1849-1ae2-45ba-bc87-8bf098861b90 None None] MFA Rules not processed for user `891005fa5ff74041b75747d16cc66798`. Rule list: `[]` (Enabled: `True`). {{(pid=80377) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 02 10:16:50.635568 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.fernet_utils [None req-09cb1849-1ae2-45ba-bc87-8bf098861b90 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80377) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 10:16:50.706508 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: [pid: 80377|app: 0|req: 926/1849] 10.222.0.82 () {58 vars in 958 bytes} [Tue Aug 2 10:16:50 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 02 10:16:50.716658 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23aa0403-a13a-432e-ae8d-f50b56549153 None None] REQUEST_METHOD: `POST` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:16:50.717824 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23aa0403-a13a-432e-ae8d-f50b56549153 None None] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:16:50.717824 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23aa0403-a13a-432e-ae8d-f50b56549153 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:16:50.739731 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.auth.core [None req-23aa0403-a13a-432e-ae8d-f50b56549153 None None] MFA Rules not processed for user `891005fa5ff74041b75747d16cc66798`. Rule list: `[]` (Enabled: `True`). {{(pid=80378) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 02 10:16:50.740930 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.fernet_utils [None req-23aa0403-a13a-432e-ae8d-f50b56549153 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80378) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 10:16:50.749336 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 924/1850] 10.222.0.82 () {58 vars in 958 bytes} [Tue Aug 2 10:16:50 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 35 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 02 10:16:50.753659 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09cb1849-1ae2-45ba-bc87-8bf098861b90 None None] Authenticating user token {{(pid=80377) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:16:50.764108 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09cb1849-1ae2-45ba-bc87-8bf098861b90 None None] Validating token access rules against request {{(pid=80377) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:16:50.769267 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b89ef5d-f4cd-4b17-a642-acfdd34b46a0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '891005fa5ff74041b75747d16cc66798', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7047912c5acd4adeba5d87c3bf895e39', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80377) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:16:50.769940 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b89ef5d-f4cd-4b17-a642-acfdd34b46a0 admin admin] REQUEST_METHOD: `GET` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:16:50.770227 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b89ef5d-f4cd-4b17-a642-acfdd34b46a0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:16:50.770502 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b89ef5d-f4cd-4b17-a642-acfdd34b46a0 admin admin] PATH_INFO: `/v3/domains` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:16:50.771246 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b89ef5d-f4cd-4b17-a642-acfdd34b46a0 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80377) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 02 10:16:50.771599 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b89ef5d-f4cd-4b17-a642-acfdd34b46a0 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:16:50.773345 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b89ef5d-f4cd-4b17-a642-acfdd34b46a0 admin admin] RBAC: Authorization granted {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:16:50.786264 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: [pid: 80377|app: 0|req: 927/1851] 10.222.0.82 () {58 vars in 1153 bytes} [Tue Aug 2 10:16:50 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 02 10:16:50.790763 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23aa0403-a13a-432e-ae8d-f50b56549153 None None] Authenticating user token {{(pid=80378) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:16:50.801138 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23aa0403-a13a-432e-ae8d-f50b56549153 None None] Validating token access rules against request {{(pid=80378) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:16:50.806971 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e571939-fa4c-40ac-bd2b-81392dde086e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '891005fa5ff74041b75747d16cc66798', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7047912c5acd4adeba5d87c3bf895e39', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80378) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:16:50.807718 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e571939-fa4c-40ac-bd2b-81392dde086e admin admin] REQUEST_METHOD: `POST` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:16:50.808066 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e571939-fa4c-40ac-bd2b-81392dde086e admin admin] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:16:50.808409 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e571939-fa4c-40ac-bd2b-81392dde086e admin admin] PATH_INFO: `/v3/projects` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:16:50.809156 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e571939-fa4c-40ac-bd2b-81392dde086e admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:16:50.811210 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e571939-fa4c-40ac-bd2b-81392dde086e admin admin] RBAC: Authorization granted {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:16:50.839225 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 925/1852] 10.222.0.82 () {60 vars in 1153 bytes} [Tue Aug 2 10:16:50 2022] POST /identity/v3/projects => generated 369 bytes in 49 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 02 10:16:50.843989 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b89ef5d-f4cd-4b17-a642-acfdd34b46a0 admin admin] Authenticating user token {{(pid=80377) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:16:50.854644 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b89ef5d-f4cd-4b17-a642-acfdd34b46a0 admin admin] Validating token access rules against request {{(pid=80377) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:16:50.896538 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG dbcounter [-] [80377] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80377) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 10:16:50.914735 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bddb3a5f-5ebe-4208-9fe6-5532b35c3678 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '891005fa5ff74041b75747d16cc66798', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7047912c5acd4adeba5d87c3bf895e39', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80377) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:16:50.915467 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bddb3a5f-5ebe-4208-9fe6-5532b35c3678 admin admin] REQUEST_METHOD: `POST` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:16:50.915659 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bddb3a5f-5ebe-4208-9fe6-5532b35c3678 admin admin] SCRIPT_NAME: `/identity` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:16:50.915937 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bddb3a5f-5ebe-4208-9fe6-5532b35c3678 admin admin] PATH_INFO: `/v3/users` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:16:50.916609 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bddb3a5f-5ebe-4208-9fe6-5532b35c3678 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:16:50.918600 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bddb3a5f-5ebe-4208-9fe6-5532b35c3678 admin admin] RBAC: Authorization granted {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:16:50.940454 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: [pid: 80377|app: 0|req: 928/1853] 10.222.0.82 () {60 vars in 1144 bytes} [Tue Aug 2 10:16:50 2022] POST /identity/v3/users => generated 285 bytes in 97 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 02 10:16:50.944702 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e571939-fa4c-40ac-bd2b-81392dde086e admin admin] Authenticating user token {{(pid=80378) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:16:50.955224 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e571939-fa4c-40ac-bd2b-81392dde086e admin admin] Validating token access rules against request {{(pid=80378) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:16:50.960117 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd2074d9-7462-45f6-baed-25417fc356b4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '891005fa5ff74041b75747d16cc66798', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7047912c5acd4adeba5d87c3bf895e39', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80378) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:16:50.960737 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd2074d9-7462-45f6-baed-25417fc356b4 admin admin] REQUEST_METHOD: `GET` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:16:50.961022 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd2074d9-7462-45f6-baed-25417fc356b4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:16:50.961294 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd2074d9-7462-45f6-baed-25417fc356b4 admin admin] PATH_INFO: `/v3/roles` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:16:50.962250 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd2074d9-7462-45f6-baed-25417fc356b4 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:16:50.963869 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd2074d9-7462-45f6-baed-25417fc356b4 admin admin] RBAC: Authorization granted {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:16:50.975086 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 926/1854] 10.222.0.82 () {58 vars in 1122 bytes} [Tue Aug 2 10:16:50 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 02 10:16:50.979126 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bddb3a5f-5ebe-4208-9fe6-5532b35c3678 admin admin] Authenticating user token {{(pid=80377) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:16:50.989710 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bddb3a5f-5ebe-4208-9fe6-5532b35c3678 admin admin] Validating token access rules against request {{(pid=80377) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:16:50.994335 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01ebc5c5-0a9a-476e-95a2-fea4c4563eba admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '891005fa5ff74041b75747d16cc66798', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7047912c5acd4adeba5d87c3bf895e39', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80377) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:16:50.994951 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01ebc5c5-0a9a-476e-95a2-fea4c4563eba admin admin] REQUEST_METHOD: `GET` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:16:50.995238 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01ebc5c5-0a9a-476e-95a2-fea4c4563eba admin admin] SCRIPT_NAME: `/identity` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:16:50.995515 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01ebc5c5-0a9a-476e-95a2-fea4c4563eba admin admin] PATH_INFO: `/v3/roles` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:16:50.996215 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01ebc5c5-0a9a-476e-95a2-fea4c4563eba admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:16:51.001424 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01ebc5c5-0a9a-476e-95a2-fea4c4563eba admin admin] RBAC: Authorization granted {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:16:51.019921 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: [pid: 80377|app: 0|req: 929/1855] 10.222.0.82 () {58 vars in 1122 bytes} [Tue Aug 2 10:16:50 2022] GET /identity/v3/roles => generated 1420 bytes in 41 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 02 10:16:51.024200 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd2074d9-7462-45f6-baed-25417fc356b4 admin admin] Authenticating user token {{(pid=80378) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:16:51.035839 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd2074d9-7462-45f6-baed-25417fc356b4 admin admin] Validating token access rules against request {{(pid=80378) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:16:51.042051 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b8b8661-30c3-4f2e-adae-45c7e6e52136 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '891005fa5ff74041b75747d16cc66798', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7047912c5acd4adeba5d87c3bf895e39', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80378) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:16:51.042723 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b8b8661-30c3-4f2e-adae-45c7e6e52136 admin admin] REQUEST_METHOD: `PUT` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:16:51.043277 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b8b8661-30c3-4f2e-adae-45c7e6e52136 admin admin] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:16:51.043277 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b8b8661-30c3-4f2e-adae-45c7e6e52136 admin admin] PATH_INFO: `/v3/projects/1ca1b599936540508bd567f10db6581c/users/56e4e985a7d24ab5888e0726265735dd/roles/e1c04b19fc4543b8b54f58bbcffbc598` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:16:51.062429 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b8b8661-30c3-4f2e-adae-45c7e6e52136 admin admin] RBAC: Authorizing `identity:create_grant(project_id=1ca1b599936540508bd567f10db6581c, user_id=56e4e985a7d24ab5888e0726265735dd, role_id=e1c04b19fc4543b8b54f58bbcffbc598)` {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:16:51.065006 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b8b8661-30c3-4f2e-adae-45c7e6e52136 admin admin] RBAC: Authorization granted {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:16:51.078786 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 927/1856] 10.222.0.82 () {60 vars in 1483 bytes} [Tue Aug 2 10:16:51 2022] PUT /identity/v3/projects/1ca1b599936540508bd567f10db6581c/users/56e4e985a7d24ab5888e0726265735dd/roles/e1c04b19fc4543b8b54f58bbcffbc598 => generated 0 bytes in 55 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 02 10:16:51.089603 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c98f7c6a-cdb1-44e2-9ea4-7c14a1e7742f None None] REQUEST_METHOD: `POST` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:16:51.089931 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c98f7c6a-cdb1-44e2-9ea4-7c14a1e7742f None None] SCRIPT_NAME: `/identity` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:16:51.090208 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c98f7c6a-cdb1-44e2-9ea4-7c14a1e7742f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:16:51.121544 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.auth.core [None req-c98f7c6a-cdb1-44e2-9ea4-7c14a1e7742f None None] MFA Rules not processed for user `56e4e985a7d24ab5888e0726265735dd`. Rule list: `[]` (Enabled: `True`). {{(pid=80377) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 02 10:16:51.122988 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.fernet_utils [None req-c98f7c6a-cdb1-44e2-9ea4-7c14a1e7742f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80377) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 10:16:51.192006 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: [pid: 80377|app: 0|req: 930/1857] 10.222.0.82 () {58 vars in 958 bytes} [Tue Aug 2 10:16:51 2022] POST /identity/v3/auth/tokens => generated 3246 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 02 10:16:51.196269 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b8b8661-30c3-4f2e-adae-45c7e6e52136 admin admin] Authenticating user token {{(pid=80378) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:16:51.269303 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b8b8661-30c3-4f2e-adae-45c7e6e52136 admin admin] Validating token access rules against request {{(pid=80378) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:16:51.274004 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79a2a464-c9db-4e25-a4fc-a87b932c7401 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '891005fa5ff74041b75747d16cc66798', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7047912c5acd4adeba5d87c3bf895e39', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80378) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:16:51.274627 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79a2a464-c9db-4e25-a4fc-a87b932c7401 admin admin] REQUEST_METHOD: `POST` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:16:51.274912 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79a2a464-c9db-4e25-a4fc-a87b932c7401 admin admin] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:16:51.275184 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79a2a464-c9db-4e25-a4fc-a87b932c7401 admin admin] PATH_INFO: `/v3/projects` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:16:51.276151 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79a2a464-c9db-4e25-a4fc-a87b932c7401 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:16:51.278071 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79a2a464-c9db-4e25-a4fc-a87b932c7401 admin admin] RBAC: Authorization granted {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:16:51.305945 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 928/1858] 10.222.0.82 () {60 vars in 1153 bytes} [Tue Aug 2 10:16:51 2022] POST /identity/v3/projects => generated 369 bytes in 110 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 02 10:16:51.310131 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c98f7c6a-cdb1-44e2-9ea4-7c14a1e7742f None None] Authenticating user token {{(pid=80377) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:16:51.320827 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c98f7c6a-cdb1-44e2-9ea4-7c14a1e7742f None None] Validating token access rules against request {{(pid=80377) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:16:51.380583 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8ae798a-573c-47c5-b083-2aad90022b1f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '891005fa5ff74041b75747d16cc66798', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7047912c5acd4adeba5d87c3bf895e39', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80377) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:16:51.381251 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8ae798a-573c-47c5-b083-2aad90022b1f admin admin] REQUEST_METHOD: `POST` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:16:51.381559 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8ae798a-573c-47c5-b083-2aad90022b1f admin admin] SCRIPT_NAME: `/identity` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:16:51.381893 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8ae798a-573c-47c5-b083-2aad90022b1f admin admin] PATH_INFO: `/v3/users` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:16:51.382715 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8ae798a-573c-47c5-b083-2aad90022b1f admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:16:51.384652 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8ae798a-573c-47c5-b083-2aad90022b1f admin admin] RBAC: Authorization granted {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:16:51.409815 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: [pid: 80377|app: 0|req: 931/1859] 10.222.0.82 () {60 vars in 1144 bytes} [Tue Aug 2 10:16:51 2022] POST /identity/v3/users => generated 291 bytes in 97 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 02 10:16:51.411047 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79a2a464-c9db-4e25-a4fc-a87b932c7401 admin admin] Authenticating user token {{(pid=80378) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:16:51.422023 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79a2a464-c9db-4e25-a4fc-a87b932c7401 admin admin] Validating token access rules against request {{(pid=80378) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:16:51.426738 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-403b9fd9-2063-4cb2-a33a-928475f3ff35 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '891005fa5ff74041b75747d16cc66798', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7047912c5acd4adeba5d87c3bf895e39', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80378) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:16:51.427656 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-403b9fd9-2063-4cb2-a33a-928475f3ff35 admin admin] REQUEST_METHOD: `GET` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:16:51.428117 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-403b9fd9-2063-4cb2-a33a-928475f3ff35 admin admin] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:16:51.428392 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-403b9fd9-2063-4cb2-a33a-928475f3ff35 admin admin] PATH_INFO: `/v3/roles` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:16:51.429124 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-403b9fd9-2063-4cb2-a33a-928475f3ff35 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:16:51.431077 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-403b9fd9-2063-4cb2-a33a-928475f3ff35 admin admin] RBAC: Authorization granted {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:16:51.446214 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 929/1860] 10.222.0.82 () {58 vars in 1122 bytes} [Tue Aug 2 10:16:51 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 02 10:16:51.447483 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8ae798a-573c-47c5-b083-2aad90022b1f admin admin] Authenticating user token {{(pid=80377) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:16:51.458402 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8ae798a-573c-47c5-b083-2aad90022b1f admin admin] Validating token access rules against request {{(pid=80377) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:16:51.463372 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e382673c-ce08-4b63-b68a-e99b6830835f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '891005fa5ff74041b75747d16cc66798', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7047912c5acd4adeba5d87c3bf895e39', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80377) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:16:51.465502 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e382673c-ce08-4b63-b68a-e99b6830835f admin admin] REQUEST_METHOD: `PUT` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:16:51.465502 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e382673c-ce08-4b63-b68a-e99b6830835f admin admin] SCRIPT_NAME: `/identity` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:16:51.465502 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e382673c-ce08-4b63-b68a-e99b6830835f admin admin] PATH_INFO: `/v3/projects/bc024990583a4b63aa458eb2d75d595e/users/3731a3a452204dba9437be7303bc6617/roles/237966442f05411191105bf336bc14f3` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:16:51.482211 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e382673c-ce08-4b63-b68a-e99b6830835f admin admin] RBAC: Authorizing `identity:create_grant(project_id=bc024990583a4b63aa458eb2d75d595e, user_id=3731a3a452204dba9437be7303bc6617, role_id=237966442f05411191105bf336bc14f3)` {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:16:51.484875 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e382673c-ce08-4b63-b68a-e99b6830835f admin admin] RBAC: Authorization granted {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:16:51.504207 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: [pid: 80377|app: 0|req: 932/1861] 10.222.0.82 () {60 vars in 1483 bytes} [Tue Aug 2 10:16:51 2022] PUT /identity/v3/projects/bc024990583a4b63aa458eb2d75d595e/users/3731a3a452204dba9437be7303bc6617/roles/237966442f05411191105bf336bc14f3 => generated 0 bytes in 54 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 02 10:16:51.505119 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-403b9fd9-2063-4cb2-a33a-928475f3ff35 admin admin] Authenticating user token {{(pid=80378) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:16:51.519307 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-403b9fd9-2063-4cb2-a33a-928475f3ff35 admin admin] Validating token access rules against request {{(pid=80378) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:16:51.560222 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG dbcounter [-] [80378] Writing DB stats keystone:SELECT=92,keystone:DELETE=4,keystone:INSERT=4 {{(pid=80378) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 10:16:51.586757 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef57ffdc-00e8-480a-b526-1fceefb6527c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '891005fa5ff74041b75747d16cc66798', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7047912c5acd4adeba5d87c3bf895e39', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80378) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:16:51.587596 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef57ffdc-00e8-480a-b526-1fceefb6527c admin admin] REQUEST_METHOD: `GET` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:16:51.588045 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef57ffdc-00e8-480a-b526-1fceefb6527c admin admin] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:16:51.588355 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef57ffdc-00e8-480a-b526-1fceefb6527c admin admin] PATH_INFO: `/v3/roles` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:16:51.589229 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef57ffdc-00e8-480a-b526-1fceefb6527c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:16:51.590850 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef57ffdc-00e8-480a-b526-1fceefb6527c admin admin] RBAC: Authorization granted {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:16:51.601094 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 930/1862] 10.222.0.82 () {58 vars in 1122 bytes} [Tue Aug 2 10:16:51 2022] GET /identity/v3/roles => generated 1420 bytes in 96 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 02 10:16:51.605462 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e382673c-ce08-4b63-b68a-e99b6830835f admin admin] Authenticating user token {{(pid=80377) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:16:51.616513 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e382673c-ce08-4b63-b68a-e99b6830835f admin admin] Validating token access rules against request {{(pid=80377) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:16:51.621387 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5224a8bd-05e2-497e-ab43-6bad4df90300 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '891005fa5ff74041b75747d16cc66798', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7047912c5acd4adeba5d87c3bf895e39', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80377) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:16:51.622641 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5224a8bd-05e2-497e-ab43-6bad4df90300 admin admin] REQUEST_METHOD: `PUT` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:16:51.622927 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5224a8bd-05e2-497e-ab43-6bad4df90300 admin admin] SCRIPT_NAME: `/identity` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:16:51.623207 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5224a8bd-05e2-497e-ab43-6bad4df90300 admin admin] PATH_INFO: `/v3/projects/bc024990583a4b63aa458eb2d75d595e/users/3731a3a452204dba9437be7303bc6617/roles/237966442f05411191105bf336bc14f3` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:16:51.625253 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5224a8bd-05e2-497e-ab43-6bad4df90300 admin admin] RBAC: Authorizing `identity:create_grant(project_id=bc024990583a4b63aa458eb2d75d595e, user_id=3731a3a452204dba9437be7303bc6617, role_id=237966442f05411191105bf336bc14f3)` {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:16:51.627957 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5224a8bd-05e2-497e-ab43-6bad4df90300 admin admin] RBAC: Authorization granted {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:16:51.639585 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: [pid: 80377|app: 0|req: 933/1863] 10.222.0.82 () {60 vars in 1483 bytes} [Tue Aug 2 10:16:51 2022] PUT /identity/v3/projects/bc024990583a4b63aa458eb2d75d595e/users/3731a3a452204dba9437be7303bc6617/roles/237966442f05411191105bf336bc14f3 => generated 0 bytes in 35 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 02 10:16:51.650020 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97bcf905-556b-4c19-887d-5686a6fd2444 None None] REQUEST_METHOD: `POST` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:16:51.650392 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97bcf905-556b-4c19-887d-5686a6fd2444 None None] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:16:51.650772 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97bcf905-556b-4c19-887d-5686a6fd2444 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:16:51.679665 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.auth.core [None req-97bcf905-556b-4c19-887d-5686a6fd2444 None None] MFA Rules not processed for user `3731a3a452204dba9437be7303bc6617`. Rule list: `[]` (Enabled: `True`). {{(pid=80378) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 02 10:16:51.680905 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.fernet_utils [None req-97bcf905-556b-4c19-887d-5686a6fd2444 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80378) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 10:16:51.750760 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 931/1864] 10.222.0.82 () {58 vars in 958 bytes} [Tue Aug 2 10:16:51 2022] POST /identity/v3/auth/tokens => generated 3313 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 02 10:16:51.763714 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5224a8bd-05e2-497e-ab43-6bad4df90300 admin admin] Authenticating user token {{(pid=80377) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:16:51.765742 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.fernet_utils [None req-5224a8bd-05e2-497e-ab43-6bad4df90300 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80377) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 10:16:51.771322 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG dbcounter [-] [80377] Writing DB stats keystone:SELECT=92,keystone:INSERT=8 {{(pid=80377) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 10:16:51.832946 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5224a8bd-05e2-497e-ab43-6bad4df90300 admin admin] Validating token access rules against request {{(pid=80377) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:16:51.836999 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0eb65c4f-91ac-4089-a942-89e24a72ee84 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1023de281ae74e9d8a6d6c312c1c97a7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f1b38ebb804c41c69772c998aae72c41', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80377) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:16:51.837843 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0eb65c4f-91ac-4089-a942-89e24a72ee84 service cinder] REQUEST_METHOD: `GET` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:16:51.838136 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0eb65c4f-91ac-4089-a942-89e24a72ee84 service cinder] SCRIPT_NAME: `/identity` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:16:51.838413 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0eb65c4f-91ac-4089-a942-89e24a72ee84 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:16:51.858063 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0eb65c4f-91ac-4089-a942-89e24a72ee84 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:16:51.859538 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0eb65c4f-91ac-4089-a942-89e24a72ee84 service cinder] RBAC: Authorization granted {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:16:51.871267 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: [pid: 80377|app: 0|req: 934/1865] 10.222.0.82 () {60 vars in 1365 bytes} [Tue Aug 2 10:16:51 2022] GET /identity/v3/auth/tokens => generated 3313 bytes in 109 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 02 10:16:51.920221 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97bcf905-556b-4c19-887d-5686a6fd2444 None None] Authenticating user token {{(pid=80378) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:16:51.921718 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.fernet_utils [None req-97bcf905-556b-4c19-887d-5686a6fd2444 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80378) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 10:16:51.937587 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97bcf905-556b-4c19-887d-5686a6fd2444 None None] Validating token access rules against request {{(pid=80378) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:16:51.941409 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c550eb43-cd96-4df1-a7af-86544b5d33fa service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1023de281ae74e9d8a6d6c312c1c97a7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f1b38ebb804c41c69772c998aae72c41', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80378) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:16:51.942090 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c550eb43-cd96-4df1-a7af-86544b5d33fa service cinder] REQUEST_METHOD: `GET` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:16:51.942378 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c550eb43-cd96-4df1-a7af-86544b5d33fa service cinder] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:16:51.942698 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c550eb43-cd96-4df1-a7af-86544b5d33fa service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:16:52.004765 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c550eb43-cd96-4df1-a7af-86544b5d33fa service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:16:52.006355 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c550eb43-cd96-4df1-a7af-86544b5d33fa service cinder] RBAC: Authorization granted {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:16:52.017042 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 932/1866] 10.222.0.82 () {60 vars in 1365 bytes} [Tue Aug 2 10:16:51 2022] GET /identity/v3/auth/tokens => generated 3246 bytes in 98 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 02 10:16:55.520860 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0eb65c4f-91ac-4089-a942-89e24a72ee84 service cinder] Authenticating user token {{(pid=80377) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:16:55.585663 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0eb65c4f-91ac-4089-a942-89e24a72ee84 service cinder] Validating token access rules against request {{(pid=80377) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:16:55.593269 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d384722d-9975-480f-9b12-b9494e5098d8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '891005fa5ff74041b75747d16cc66798', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7047912c5acd4adeba5d87c3bf895e39', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80377) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:16:55.593969 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d384722d-9975-480f-9b12-b9494e5098d8 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:16:55.594255 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d384722d-9975-480f-9b12-b9494e5098d8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:16:55.594536 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d384722d-9975-480f-9b12-b9494e5098d8 admin admin] PATH_INFO: `/v3/users/56e4e985a7d24ab5888e0726265735dd` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:16:55.595945 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d384722d-9975-480f-9b12-b9494e5098d8 admin admin] RBAC: Authorizing `identity:delete_user(user_id=56e4e985a7d24ab5888e0726265735dd)` {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:16:55.597909 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d384722d-9975-480f-9b12-b9494e5098d8 admin admin] RBAC: Authorization granted {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:16:55.652582 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.notifications [None req-d384722d-9975-480f-9b12-b9494e5098d8 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '56e4e985a7d24ab5888e0726265735dd'} {{(pid=80377) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 10:16:55.663380 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.notifications [None req-d384722d-9975-480f-9b12-b9494e5098d8 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '56e4e985a7d24ab5888e0726265735dd'} {{(pid=80377) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 10:16:55.672645 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.notifications [None req-d384722d-9975-480f-9b12-b9494e5098d8 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '56e4e985a7d24ab5888e0726265735dd'} {{(pid=80377) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 10:16:55.673351 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.notifications [None req-d384722d-9975-480f-9b12-b9494e5098d8 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '56e4e985a7d24ab5888e0726265735dd'} {{(pid=80377) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 10:16:55.691015 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.notifications [None req-d384722d-9975-480f-9b12-b9494e5098d8 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '56e4e985a7d24ab5888e0726265735dd'} {{(pid=80377) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 10:16:55.697525 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: [pid: 80377|app: 0|req: 935/1867] 10.222.0.82 () {58 vars in 1224 bytes} [Tue Aug 2 10:16:55 2022] DELETE /identity/v3/users/56e4e985a7d24ab5888e0726265735dd => generated 0 bytes in 177 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 02 10:16:55.702088 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f843863-d1a5-48e1-abac-ce15a6e85c55 None None] REQUEST_METHOD: `POST` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:16:55.702585 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f843863-d1a5-48e1-abac-ce15a6e85c55 None None] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:16:55.702820 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f843863-d1a5-48e1-abac-ce15a6e85c55 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:16:55.724745 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.auth.core [None req-7f843863-d1a5-48e1-abac-ce15a6e85c55 None None] MFA Rules not processed for user `891005fa5ff74041b75747d16cc66798`. Rule list: `[]` (Enabled: `True`). {{(pid=80378) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 02 10:16:55.726122 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.fernet_utils [None req-7f843863-d1a5-48e1-abac-ce15a6e85c55 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80378) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 10:16:55.784937 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG dbcounter [-] [80378] Writing DB stats keystone:SELECT=100 {{(pid=80378) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 10:16:55.799575 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 933/1868] 10.222.0.82 () {58 vars in 958 bytes} [Tue Aug 2 10:16:55 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 02 10:16:55.810938 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d384722d-9975-480f-9b12-b9494e5098d8 admin admin] Authenticating user token {{(pid=80377) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:16:55.812551 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.fernet_utils [None req-d384722d-9975-480f-9b12-b9494e5098d8 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80377) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 10:16:55.842976 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG dbcounter [-] [80377] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80377) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 10:16:55.876440 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d384722d-9975-480f-9b12-b9494e5098d8 admin admin] Validating token access rules against request {{(pid=80377) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:16:55.880074 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b711678e-79f5-4986-a892-42ee62eb5a54 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '33b87170f0634b67bce60750847b5cbe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f1b38ebb804c41c69772c998aae72c41', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80377) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:16:55.880710 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b711678e-79f5-4986-a892-42ee62eb5a54 service neutron] REQUEST_METHOD: `GET` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:16:55.880996 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b711678e-79f5-4986-a892-42ee62eb5a54 service neutron] SCRIPT_NAME: `/identity` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:16:55.881275 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b711678e-79f5-4986-a892-42ee62eb5a54 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:16:55.890893 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b711678e-79f5-4986-a892-42ee62eb5a54 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:16:55.892367 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b711678e-79f5-4986-a892-42ee62eb5a54 service neutron] RBAC: Authorization granted {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:16:55.901941 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: [pid: 80377|app: 0|req: 936/1869] 10.222.0.82 () {60 vars in 1365 bytes} [Tue Aug 2 10:16:55 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 93 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 02 10:16:56.183569 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f843863-d1a5-48e1-abac-ce15a6e85c55 None None] Authenticating user token {{(pid=80378) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:16:56.185031 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.fernet_utils [None req-7f843863-d1a5-48e1-abac-ce15a6e85c55 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80378) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 10:16:56.198431 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f843863-d1a5-48e1-abac-ce15a6e85c55 None None] Validating token access rules against request {{(pid=80378) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:16:56.206969 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4176d7de-c2f9-4614-9138-b95d91408f30 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '891005fa5ff74041b75747d16cc66798', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7047912c5acd4adeba5d87c3bf895e39', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80378) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:16:56.207647 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4176d7de-c2f9-4614-9138-b95d91408f30 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:16:56.208227 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4176d7de-c2f9-4614-9138-b95d91408f30 admin admin] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:16:56.208513 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4176d7de-c2f9-4614-9138-b95d91408f30 admin admin] PATH_INFO: `/v3/projects/1ca1b599936540508bd567f10db6581c` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:16:56.210285 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4176d7de-c2f9-4614-9138-b95d91408f30 admin admin] RBAC: Authorizing `identity:delete_project(project_id=1ca1b599936540508bd567f10db6581c)` {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:16:56.212374 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4176d7de-c2f9-4614-9138-b95d91408f30 admin admin] RBAC: Authorization granted {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:16:56.278398 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.notifications [None req-4176d7de-c2f9-4614-9138-b95d91408f30 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '1ca1b599936540508bd567f10db6581c', 'request_id': 'req-4176d7de-c2f9-4614-9138-b95d91408f30'} {{(pid=80378) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 10:16:56.285372 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.notifications [None req-4176d7de-c2f9-4614-9138-b95d91408f30 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '1ca1b599936540508bd567f10db6581c', 'request_id': 'req-4176d7de-c2f9-4614-9138-b95d91408f30'} {{(pid=80378) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 10:16:56.292417 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.notifications [None req-4176d7de-c2f9-4614-9138-b95d91408f30 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '1ca1b599936540508bd567f10db6581c', 'request_id': 'req-4176d7de-c2f9-4614-9138-b95d91408f30'} {{(pid=80378) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 10:16:56.305283 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.notifications [None req-4176d7de-c2f9-4614-9138-b95d91408f30 admin admin] The token cache is being invalidate because project 1ca1b599936540508bd567f10db6581c was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80378) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 02 10:16:56.305575 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.notifications [None req-4176d7de-c2f9-4614-9138-b95d91408f30 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80378) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 10:16:56.306606 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.notifications [None req-4176d7de-c2f9-4614-9138-b95d91408f30 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80378) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 10:16:56.308170 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 934/1870] 10.222.0.82 () {58 vars in 1233 bytes} [Tue Aug 2 10:16:56 2022] DELETE /identity/v3/projects/1ca1b599936540508bd567f10db6581c => generated 0 bytes in 125 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 02 10:16:56.312284 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b711678e-79f5-4986-a892-42ee62eb5a54 service neutron] Authenticating user token {{(pid=80377) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:16:56.322516 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b711678e-79f5-4986-a892-42ee62eb5a54 service neutron] Validating token access rules against request {{(pid=80377) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:16:56.382248 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7ce737e-7cdd-4d93-8522-2178427b507c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '891005fa5ff74041b75747d16cc66798', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7047912c5acd4adeba5d87c3bf895e39', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80377) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:16:56.382878 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7ce737e-7cdd-4d93-8522-2178427b507c admin admin] REQUEST_METHOD: `DELETE` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:16:56.383159 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7ce737e-7cdd-4d93-8522-2178427b507c admin admin] SCRIPT_NAME: `/identity` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:16:56.383431 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7ce737e-7cdd-4d93-8522-2178427b507c admin admin] PATH_INFO: `/v3/users/3731a3a452204dba9437be7303bc6617` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:16:56.384799 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7ce737e-7cdd-4d93-8522-2178427b507c admin admin] RBAC: Authorizing `identity:delete_user(user_id=3731a3a452204dba9437be7303bc6617)` {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:16:56.386750 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7ce737e-7cdd-4d93-8522-2178427b507c admin admin] RBAC: Authorization granted {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:16:56.445314 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.notifications [None req-d7ce737e-7cdd-4d93-8522-2178427b507c admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '3731a3a452204dba9437be7303bc6617'} {{(pid=80377) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 10:16:56.455428 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.notifications [None req-d7ce737e-7cdd-4d93-8522-2178427b507c admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '3731a3a452204dba9437be7303bc6617'} {{(pid=80377) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 10:16:56.463893 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.notifications [None req-d7ce737e-7cdd-4d93-8522-2178427b507c admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '3731a3a452204dba9437be7303bc6617'} {{(pid=80377) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 10:16:56.464659 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.notifications [None req-d7ce737e-7cdd-4d93-8522-2178427b507c admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '3731a3a452204dba9437be7303bc6617'} {{(pid=80377) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 10:16:56.478486 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.notifications [None req-d7ce737e-7cdd-4d93-8522-2178427b507c admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '3731a3a452204dba9437be7303bc6617'} {{(pid=80377) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 10:16:56.485221 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: [pid: 80377|app: 0|req: 937/1871] 10.222.0.82 () {58 vars in 1224 bytes} [Tue Aug 2 10:16:56 2022] DELETE /identity/v3/users/3731a3a452204dba9437be7303bc6617 => generated 0 bytes in 173 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 02 10:16:56.791724 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4176d7de-c2f9-4614-9138-b95d91408f30 admin admin] Authenticating user token {{(pid=80378) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:16:56.793177 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.fernet_utils [None req-4176d7de-c2f9-4614-9138-b95d91408f30 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80378) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 10:16:56.838471 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4176d7de-c2f9-4614-9138-b95d91408f30 admin admin] Validating token access rules against request {{(pid=80378) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:16:56.922839 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e050431a-39bf-4456-a95c-73a981c15454 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '891005fa5ff74041b75747d16cc66798', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7047912c5acd4adeba5d87c3bf895e39', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80378) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:16:56.923501 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e050431a-39bf-4456-a95c-73a981c15454 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:16:56.923787 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e050431a-39bf-4456-a95c-73a981c15454 admin admin] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:16:56.924065 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e050431a-39bf-4456-a95c-73a981c15454 admin admin] PATH_INFO: `/v3/projects/bc024990583a4b63aa458eb2d75d595e` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:16:56.925250 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e050431a-39bf-4456-a95c-73a981c15454 admin admin] RBAC: Authorizing `identity:delete_project(project_id=bc024990583a4b63aa458eb2d75d595e)` {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:16:56.927206 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e050431a-39bf-4456-a95c-73a981c15454 admin admin] RBAC: Authorization granted {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:16:56.978328 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.notifications [None req-e050431a-39bf-4456-a95c-73a981c15454 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'bc024990583a4b63aa458eb2d75d595e', 'request_id': 'req-e050431a-39bf-4456-a95c-73a981c15454'} {{(pid=80378) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 10:16:56.985197 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.notifications [None req-e050431a-39bf-4456-a95c-73a981c15454 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'bc024990583a4b63aa458eb2d75d595e', 'request_id': 'req-e050431a-39bf-4456-a95c-73a981c15454'} {{(pid=80378) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 10:16:56.991785 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.notifications [None req-e050431a-39bf-4456-a95c-73a981c15454 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'bc024990583a4b63aa458eb2d75d595e', 'request_id': 'req-e050431a-39bf-4456-a95c-73a981c15454'} {{(pid=80378) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 10:16:57.006212 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.notifications [None req-e050431a-39bf-4456-a95c-73a981c15454 admin admin] The token cache is being invalidate because project bc024990583a4b63aa458eb2d75d595e was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80378) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 02 10:16:57.007021 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.notifications [None req-e050431a-39bf-4456-a95c-73a981c15454 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80378) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 10:16:57.007704 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.notifications [None req-e050431a-39bf-4456-a95c-73a981c15454 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80378) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 10:16:57.009250 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 935/1872] 10.222.0.82 () {58 vars in 1233 bytes} [Tue Aug 2 10:16:56 2022] DELETE /identity/v3/projects/bc024990583a4b63aa458eb2d75d595e => generated 0 bytes in 219 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 02 10:16:57.015257 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd2f52bf-484c-4427-bd8b-d579d2ad59ba None None] REQUEST_METHOD: `POST` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:16:57.015763 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd2f52bf-484c-4427-bd8b-d579d2ad59ba None None] SCRIPT_NAME: `/identity` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:16:57.015881 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd2f52bf-484c-4427-bd8b-d579d2ad59ba None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:16:57.038844 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.auth.core [None req-dd2f52bf-484c-4427-bd8b-d579d2ad59ba None None] MFA Rules not processed for user `891005fa5ff74041b75747d16cc66798`. Rule list: `[]` (Enabled: `True`). {{(pid=80377) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 02 10:16:57.040053 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.fernet_utils [None req-dd2f52bf-484c-4427-bd8b-d579d2ad59ba None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80377) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 10:16:57.083057 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG dbcounter [-] [80377] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80377) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 10:16:57.102456 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: [pid: 80377|app: 0|req: 938/1873] 10.222.0.82 () {58 vars in 958 bytes} [Tue Aug 2 10:16:57 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 02 10:16:57.113460 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d8fb852-965b-4876-a1d0-90b830e0fa5e None None] REQUEST_METHOD: `POST` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:16:57.113862 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d8fb852-965b-4876-a1d0-90b830e0fa5e None None] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:16:57.114178 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d8fb852-965b-4876-a1d0-90b830e0fa5e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:16:57.140495 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.auth.core [None req-6d8fb852-965b-4876-a1d0-90b830e0fa5e None None] MFA Rules not processed for user `891005fa5ff74041b75747d16cc66798`. Rule list: `[]` (Enabled: `True`). {{(pid=80378) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 02 10:16:57.141740 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.fernet_utils [None req-6d8fb852-965b-4876-a1d0-90b830e0fa5e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80378) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 10:16:57.149675 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 936/1874] 10.222.0.82 () {58 vars in 958 bytes} [Tue Aug 2 10:16:57 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 38 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 02 10:16:57.153969 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd2f52bf-484c-4427-bd8b-d579d2ad59ba None None] Authenticating user token {{(pid=80377) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:16:57.164004 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd2f52bf-484c-4427-bd8b-d579d2ad59ba None None] Validating token access rules against request {{(pid=80377) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:16:57.168856 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86e83df9-1ab9-4e67-83fc-c7188e3e17df admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '891005fa5ff74041b75747d16cc66798', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7047912c5acd4adeba5d87c3bf895e39', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80377) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:16:57.170016 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86e83df9-1ab9-4e67-83fc-c7188e3e17df admin admin] REQUEST_METHOD: `GET` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:16:57.170300 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86e83df9-1ab9-4e67-83fc-c7188e3e17df admin admin] SCRIPT_NAME: `/identity` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:16:57.170576 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86e83df9-1ab9-4e67-83fc-c7188e3e17df admin admin] PATH_INFO: `/v3/domains` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:16:57.171276 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86e83df9-1ab9-4e67-83fc-c7188e3e17df admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80377) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 02 10:16:57.171585 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86e83df9-1ab9-4e67-83fc-c7188e3e17df admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:16:57.173183 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86e83df9-1ab9-4e67-83fc-c7188e3e17df admin admin] RBAC: Authorization granted {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:16:57.185814 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: [pid: 80377|app: 0|req: 939/1875] 10.222.0.82 () {58 vars in 1153 bytes} [Tue Aug 2 10:16:57 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 02 10:16:57.190493 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d8fb852-965b-4876-a1d0-90b830e0fa5e None None] Authenticating user token {{(pid=80378) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:16:57.194987 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG dbcounter [-] [80378] Writing DB stats keystone:SELECT=84,keystone:DELETE=14,keystone:INSERT=2 {{(pid=80378) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 10:16:57.201460 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d8fb852-965b-4876-a1d0-90b830e0fa5e None None] Validating token access rules against request {{(pid=80378) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:16:57.207055 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f9a3c3a-fbf0-4964-8698-59c52d181bdc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '891005fa5ff74041b75747d16cc66798', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7047912c5acd4adeba5d87c3bf895e39', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80378) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:16:57.207667 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f9a3c3a-fbf0-4964-8698-59c52d181bdc admin admin] REQUEST_METHOD: `POST` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:16:57.207950 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f9a3c3a-fbf0-4964-8698-59c52d181bdc admin admin] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:16:57.208223 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f9a3c3a-fbf0-4964-8698-59c52d181bdc admin admin] PATH_INFO: `/v3/projects` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:16:57.208900 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f9a3c3a-fbf0-4964-8698-59c52d181bdc admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:16:57.210927 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f9a3c3a-fbf0-4964-8698-59c52d181bdc admin admin] RBAC: Authorization granted {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:16:57.239536 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 937/1876] 10.222.0.82 () {60 vars in 1153 bytes} [Tue Aug 2 10:16:57 2022] POST /identity/v3/projects => generated 387 bytes in 50 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 02 10:16:57.243768 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86e83df9-1ab9-4e67-83fc-c7188e3e17df admin admin] Authenticating user token {{(pid=80377) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:16:57.254263 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86e83df9-1ab9-4e67-83fc-c7188e3e17df admin admin] Validating token access rules against request {{(pid=80377) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:16:57.326438 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58af6790-107a-426a-8ffd-d53ee55e8483 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '891005fa5ff74041b75747d16cc66798', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7047912c5acd4adeba5d87c3bf895e39', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80377) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:16:57.327078 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58af6790-107a-426a-8ffd-d53ee55e8483 admin admin] REQUEST_METHOD: `POST` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:16:57.327364 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58af6790-107a-426a-8ffd-d53ee55e8483 admin admin] SCRIPT_NAME: `/identity` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:16:57.327642 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58af6790-107a-426a-8ffd-d53ee55e8483 admin admin] PATH_INFO: `/v3/users` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:16:57.328363 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58af6790-107a-426a-8ffd-d53ee55e8483 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:16:57.330327 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58af6790-107a-426a-8ffd-d53ee55e8483 admin admin] RBAC: Authorization granted {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:16:57.360221 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: [pid: 80377|app: 0|req: 940/1877] 10.222.0.82 () {60 vars in 1144 bytes} [Tue Aug 2 10:16:57 2022] POST /identity/v3/users => generated 294 bytes in 115 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 02 10:16:57.363376 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f9a3c3a-fbf0-4964-8698-59c52d181bdc admin admin] Authenticating user token {{(pid=80378) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:16:57.374732 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f9a3c3a-fbf0-4964-8698-59c52d181bdc admin admin] Validating token access rules against request {{(pid=80378) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:16:57.379440 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3291d7f1-46d0-4b9d-a7f9-6c8716e9e376 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '891005fa5ff74041b75747d16cc66798', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7047912c5acd4adeba5d87c3bf895e39', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80378) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:16:57.380191 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3291d7f1-46d0-4b9d-a7f9-6c8716e9e376 admin admin] REQUEST_METHOD: `GET` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:16:57.380475 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3291d7f1-46d0-4b9d-a7f9-6c8716e9e376 admin admin] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:16:57.380744 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3291d7f1-46d0-4b9d-a7f9-6c8716e9e376 admin admin] PATH_INFO: `/v3/roles` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:16:57.381442 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3291d7f1-46d0-4b9d-a7f9-6c8716e9e376 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:16:57.383094 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3291d7f1-46d0-4b9d-a7f9-6c8716e9e376 admin admin] RBAC: Authorization granted {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:16:57.396411 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 938/1878] 10.222.0.82 () {58 vars in 1122 bytes} [Tue Aug 2 10:16:57 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 02 10:16:57.398292 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58af6790-107a-426a-8ffd-d53ee55e8483 admin admin] Authenticating user token {{(pid=80377) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:16:57.409502 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58af6790-107a-426a-8ffd-d53ee55e8483 admin admin] Validating token access rules against request {{(pid=80377) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:16:57.414863 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-464c25ed-3485-467e-9c07-6e939a0be2c7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '891005fa5ff74041b75747d16cc66798', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7047912c5acd4adeba5d87c3bf895e39', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80377) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:16:57.415546 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-464c25ed-3485-467e-9c07-6e939a0be2c7 admin admin] REQUEST_METHOD: `GET` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:16:57.415978 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-464c25ed-3485-467e-9c07-6e939a0be2c7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:16:57.416310 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-464c25ed-3485-467e-9c07-6e939a0be2c7 admin admin] PATH_INFO: `/v3/roles` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:16:57.417082 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-464c25ed-3485-467e-9c07-6e939a0be2c7 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:16:57.418830 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-464c25ed-3485-467e-9c07-6e939a0be2c7 admin admin] RBAC: Authorization granted {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:16:57.433171 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: [pid: 80377|app: 0|req: 941/1879] 10.222.0.82 () {58 vars in 1122 bytes} [Tue Aug 2 10:16:57 2022] GET /identity/v3/roles => generated 1420 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 02 10:16:57.437613 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3291d7f1-46d0-4b9d-a7f9-6c8716e9e376 admin admin] Authenticating user token {{(pid=80378) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:16:57.447961 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3291d7f1-46d0-4b9d-a7f9-6c8716e9e376 admin admin] Validating token access rules against request {{(pid=80378) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:16:57.453545 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e969cb55-d4aa-48e0-a534-41857314c91f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '891005fa5ff74041b75747d16cc66798', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7047912c5acd4adeba5d87c3bf895e39', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80378) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:16:57.454260 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e969cb55-d4aa-48e0-a534-41857314c91f admin admin] REQUEST_METHOD: `PUT` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:16:57.454546 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e969cb55-d4aa-48e0-a534-41857314c91f admin admin] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:16:57.454824 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e969cb55-d4aa-48e0-a534-41857314c91f admin admin] PATH_INFO: `/v3/projects/8816c13a4da74d3191285b9a4c2361f2/users/dc8bee754f9a43af9f047fe10308200e/roles/e1c04b19fc4543b8b54f58bbcffbc598` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:16:57.470879 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e969cb55-d4aa-48e0-a534-41857314c91f admin admin] RBAC: Authorizing `identity:create_grant(project_id=8816c13a4da74d3191285b9a4c2361f2, user_id=dc8bee754f9a43af9f047fe10308200e, role_id=e1c04b19fc4543b8b54f58bbcffbc598)` {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:16:57.473566 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e969cb55-d4aa-48e0-a534-41857314c91f admin admin] RBAC: Authorization granted {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:16:57.488820 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 939/1880] 10.222.0.82 () {60 vars in 1483 bytes} [Tue Aug 2 10:16:57 2022] PUT /identity/v3/projects/8816c13a4da74d3191285b9a4c2361f2/users/dc8bee754f9a43af9f047fe10308200e/roles/e1c04b19fc4543b8b54f58bbcffbc598 => generated 0 bytes in 52 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 02 10:16:57.499457 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d48c700-18b0-452c-be27-b0d56a3f3f03 None None] REQUEST_METHOD: `POST` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:16:57.499801 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d48c700-18b0-452c-be27-b0d56a3f3f03 None None] SCRIPT_NAME: `/identity` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:16:57.500087 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d48c700-18b0-452c-be27-b0d56a3f3f03 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:16:57.533533 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.auth.core [None req-6d48c700-18b0-452c-be27-b0d56a3f3f03 None None] MFA Rules not processed for user `dc8bee754f9a43af9f047fe10308200e`. Rule list: `[]` (Enabled: `True`). {{(pid=80377) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 02 10:16:57.534832 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.fernet_utils [None req-6d48c700-18b0-452c-be27-b0d56a3f3f03 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80377) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 10:16:57.599732 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: [pid: 80377|app: 0|req: 942/1881] 10.222.0.82 () {58 vars in 958 bytes} [Tue Aug 2 10:16:57 2022] POST /identity/v3/auth/tokens => generated 3264 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 02 10:16:57.604275 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e969cb55-d4aa-48e0-a534-41857314c91f admin admin] Authenticating user token {{(pid=80378) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:16:57.667952 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e969cb55-d4aa-48e0-a534-41857314c91f admin admin] Validating token access rules against request {{(pid=80378) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:16:57.673198 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c89aab6-63e4-407d-a2c0-53837997fee8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '891005fa5ff74041b75747d16cc66798', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7047912c5acd4adeba5d87c3bf895e39', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80378) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:16:57.673843 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c89aab6-63e4-407d-a2c0-53837997fee8 admin admin] REQUEST_METHOD: `POST` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:16:57.674136 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c89aab6-63e4-407d-a2c0-53837997fee8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:16:57.674427 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c89aab6-63e4-407d-a2c0-53837997fee8 admin admin] PATH_INFO: `/v3/projects` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:16:57.675103 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c89aab6-63e4-407d-a2c0-53837997fee8 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:16:57.677018 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c89aab6-63e4-407d-a2c0-53837997fee8 admin admin] RBAC: Authorization granted {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:16:57.703627 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 940/1882] 10.222.0.82 () {60 vars in 1153 bytes} [Tue Aug 2 10:16:57 2022] POST /identity/v3/projects => generated 385 bytes in 100 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 02 10:16:57.707930 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d48c700-18b0-452c-be27-b0d56a3f3f03 None None] Authenticating user token {{(pid=80377) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:16:57.719963 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d48c700-18b0-452c-be27-b0d56a3f3f03 None None] Validating token access rules against request {{(pid=80377) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:16:57.759859 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG dbcounter [-] [80377] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80377) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 10:16:57.777496 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85b79783-c0db-4078-b527-89638580c098 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '891005fa5ff74041b75747d16cc66798', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7047912c5acd4adeba5d87c3bf895e39', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80377) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:16:57.778172 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85b79783-c0db-4078-b527-89638580c098 admin admin] REQUEST_METHOD: `POST` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:16:57.779050 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85b79783-c0db-4078-b527-89638580c098 admin admin] SCRIPT_NAME: `/identity` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:16:57.779338 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85b79783-c0db-4078-b527-89638580c098 admin admin] PATH_INFO: `/v3/users` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:16:57.780026 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85b79783-c0db-4078-b527-89638580c098 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:16:57.782226 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85b79783-c0db-4078-b527-89638580c098 admin admin] RBAC: Authorization granted {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:16:57.809215 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: [pid: 80377|app: 0|req: 943/1883] 10.222.0.82 () {60 vars in 1144 bytes} [Tue Aug 2 10:16:57 2022] POST /identity/v3/users => generated 299 bytes in 102 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 02 10:16:57.813293 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c89aab6-63e4-407d-a2c0-53837997fee8 admin admin] Authenticating user token {{(pid=80378) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:16:57.824446 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c89aab6-63e4-407d-a2c0-53837997fee8 admin admin] Validating token access rules against request {{(pid=80378) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:16:57.830141 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dee6b9f5-9a6b-4274-b803-8546b10f4316 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '891005fa5ff74041b75747d16cc66798', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7047912c5acd4adeba5d87c3bf895e39', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80378) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:16:57.830820 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dee6b9f5-9a6b-4274-b803-8546b10f4316 admin admin] REQUEST_METHOD: `GET` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:16:57.831233 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dee6b9f5-9a6b-4274-b803-8546b10f4316 admin admin] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:16:57.831515 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dee6b9f5-9a6b-4274-b803-8546b10f4316 admin admin] PATH_INFO: `/v3/roles` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:16:57.832224 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dee6b9f5-9a6b-4274-b803-8546b10f4316 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:16:57.833893 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dee6b9f5-9a6b-4274-b803-8546b10f4316 admin admin] RBAC: Authorization granted {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:16:57.848492 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 941/1884] 10.222.0.82 () {58 vars in 1122 bytes} [Tue Aug 2 10:16:57 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 02 10:16:57.849625 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85b79783-c0db-4078-b527-89638580c098 admin admin] Authenticating user token {{(pid=80377) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:16:57.860466 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85b79783-c0db-4078-b527-89638580c098 admin admin] Validating token access rules against request {{(pid=80377) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:16:57.865290 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f8fb05c-06ad-404f-8c42-6668ee6489b1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '891005fa5ff74041b75747d16cc66798', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7047912c5acd4adeba5d87c3bf895e39', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80377) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:16:57.866000 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f8fb05c-06ad-404f-8c42-6668ee6489b1 admin admin] REQUEST_METHOD: `PUT` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:16:57.866288 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f8fb05c-06ad-404f-8c42-6668ee6489b1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:16:57.866567 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f8fb05c-06ad-404f-8c42-6668ee6489b1 admin admin] PATH_INFO: `/v3/projects/89608e25cc4e4acaab453433b8b926f6/users/99d3ab27ed8f4332b5c97882c47fe14a/roles/237966442f05411191105bf336bc14f3` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:16:57.881337 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f8fb05c-06ad-404f-8c42-6668ee6489b1 admin admin] RBAC: Authorizing `identity:create_grant(project_id=89608e25cc4e4acaab453433b8b926f6, user_id=99d3ab27ed8f4332b5c97882c47fe14a, role_id=237966442f05411191105bf336bc14f3)` {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:16:57.884739 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f8fb05c-06ad-404f-8c42-6668ee6489b1 admin admin] RBAC: Authorization granted {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:16:57.898341 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: [pid: 80377|app: 0|req: 944/1885] 10.222.0.82 () {60 vars in 1483 bytes} [Tue Aug 2 10:16:57 2022] PUT /identity/v3/projects/89608e25cc4e4acaab453433b8b926f6/users/99d3ab27ed8f4332b5c97882c47fe14a/roles/237966442f05411191105bf336bc14f3 => generated 0 bytes in 49 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 02 10:16:57.902341 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dee6b9f5-9a6b-4274-b803-8546b10f4316 admin admin] Authenticating user token {{(pid=80378) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:16:57.912787 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dee6b9f5-9a6b-4274-b803-8546b10f4316 admin admin] Validating token access rules against request {{(pid=80378) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:16:57.982166 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff1a539c-bbc6-4759-8eae-3f3c539b6c7a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '891005fa5ff74041b75747d16cc66798', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7047912c5acd4adeba5d87c3bf895e39', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80378) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:16:57.982790 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff1a539c-bbc6-4759-8eae-3f3c539b6c7a admin admin] REQUEST_METHOD: `GET` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:16:57.983081 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff1a539c-bbc6-4759-8eae-3f3c539b6c7a admin admin] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:16:57.983357 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff1a539c-bbc6-4759-8eae-3f3c539b6c7a admin admin] PATH_INFO: `/v3/roles` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:16:57.984058 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff1a539c-bbc6-4759-8eae-3f3c539b6c7a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:16:57.985657 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff1a539c-bbc6-4759-8eae-3f3c539b6c7a admin admin] RBAC: Authorization granted {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:16:57.996088 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 942/1886] 10.222.0.82 () {58 vars in 1122 bytes} [Tue Aug 2 10:16:57 2022] GET /identity/v3/roles => generated 1420 bytes in 94 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 02 10:16:58.000094 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f8fb05c-06ad-404f-8c42-6668ee6489b1 admin admin] Authenticating user token {{(pid=80377) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:16:58.010560 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f8fb05c-06ad-404f-8c42-6668ee6489b1 admin admin] Validating token access rules against request {{(pid=80377) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:16:58.016164 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ab77ca9-5fec-43af-ab51-98136dd851a6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '891005fa5ff74041b75747d16cc66798', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7047912c5acd4adeba5d87c3bf895e39', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80377) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:16:58.016841 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ab77ca9-5fec-43af-ab51-98136dd851a6 admin admin] REQUEST_METHOD: `PUT` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:16:58.017108 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ab77ca9-5fec-43af-ab51-98136dd851a6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:16:58.017382 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ab77ca9-5fec-43af-ab51-98136dd851a6 admin admin] PATH_INFO: `/v3/projects/89608e25cc4e4acaab453433b8b926f6/users/99d3ab27ed8f4332b5c97882c47fe14a/roles/237966442f05411191105bf336bc14f3` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:16:58.019567 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ab77ca9-5fec-43af-ab51-98136dd851a6 admin admin] RBAC: Authorizing `identity:create_grant(project_id=89608e25cc4e4acaab453433b8b926f6, user_id=99d3ab27ed8f4332b5c97882c47fe14a, role_id=237966442f05411191105bf336bc14f3)` {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:16:58.022247 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ab77ca9-5fec-43af-ab51-98136dd851a6 admin admin] RBAC: Authorization granted {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:16:58.055205 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: [pid: 80377|app: 0|req: 945/1887] 10.222.0.82 () {60 vars in 1483 bytes} [Tue Aug 2 10:16:57 2022] PUT /identity/v3/projects/89608e25cc4e4acaab453433b8b926f6/users/99d3ab27ed8f4332b5c97882c47fe14a/roles/237966442f05411191105bf336bc14f3 => generated 0 bytes in 47 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 02 10:16:58.058750 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf1ca9ca-a8f7-4b8a-ba05-bb01a32f85b4 None None] REQUEST_METHOD: `POST` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:16:58.059047 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf1ca9ca-a8f7-4b8a-ba05-bb01a32f85b4 None None] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:16:58.059348 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf1ca9ca-a8f7-4b8a-ba05-bb01a32f85b4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:16:58.094491 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.auth.core [None req-cf1ca9ca-a8f7-4b8a-ba05-bb01a32f85b4 None None] MFA Rules not processed for user `99d3ab27ed8f4332b5c97882c47fe14a`. Rule list: `[]` (Enabled: `True`). {{(pid=80378) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 02 10:16:58.095721 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.fernet_utils [None req-cf1ca9ca-a8f7-4b8a-ba05-bb01a32f85b4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80378) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 10:16:58.101116 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG dbcounter [-] [80378] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80378) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 10:16:58.187931 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 943/1888] 10.222.0.82 () {58 vars in 958 bytes} [Tue Aug 2 10:16:58 2022] POST /identity/v3/auth/tokens => generated 3329 bytes in 130 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 02 10:16:58.199494 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ab77ca9-5fec-43af-ab51-98136dd851a6 admin admin] Authenticating user token {{(pid=80377) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:16:58.203030 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.fernet_utils [None req-5ab77ca9-5fec-43af-ab51-98136dd851a6 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80377) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 10:16:58.272896 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ab77ca9-5fec-43af-ab51-98136dd851a6 admin admin] Validating token access rules against request {{(pid=80377) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:16:58.277122 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9d986fe-5e6c-4133-a407-6ef452170dd1 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1023de281ae74e9d8a6d6c312c1c97a7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f1b38ebb804c41c69772c998aae72c41', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80377) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:16:58.277755 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9d986fe-5e6c-4133-a407-6ef452170dd1 service cinder] REQUEST_METHOD: `GET` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:16:58.278239 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9d986fe-5e6c-4133-a407-6ef452170dd1 service cinder] SCRIPT_NAME: `/identity` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:16:58.278377 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9d986fe-5e6c-4133-a407-6ef452170dd1 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:16:58.288145 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9d986fe-5e6c-4133-a407-6ef452170dd1 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:16:58.289581 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9d986fe-5e6c-4133-a407-6ef452170dd1 service cinder] RBAC: Authorization granted {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:16:58.300500 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: [pid: 80377|app: 0|req: 946/1889] 10.222.0.82 () {60 vars in 1365 bytes} [Tue Aug 2 10:16:58 2022] GET /identity/v3/auth/tokens => generated 3329 bytes in 102 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 02 10:16:58.437753 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf1ca9ca-a8f7-4b8a-ba05-bb01a32f85b4 None None] Authenticating user token {{(pid=80378) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:16:58.499066 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf1ca9ca-a8f7-4b8a-ba05-bb01a32f85b4 None None] Validating token access rules against request {{(pid=80378) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:16:58.503550 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-402294c9-5238-4154-9d98-c33ee3e51218 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '891005fa5ff74041b75747d16cc66798', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7047912c5acd4adeba5d87c3bf895e39', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80378) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:16:58.504176 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-402294c9-5238-4154-9d98-c33ee3e51218 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:16:58.504458 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-402294c9-5238-4154-9d98-c33ee3e51218 admin admin] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:16:58.504733 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-402294c9-5238-4154-9d98-c33ee3e51218 admin admin] PATH_INFO: `/v3/users/dc8bee754f9a43af9f047fe10308200e` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:16:58.505928 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-402294c9-5238-4154-9d98-c33ee3e51218 admin admin] RBAC: Authorizing `identity:delete_user(user_id=dc8bee754f9a43af9f047fe10308200e)` {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:16:58.507855 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-402294c9-5238-4154-9d98-c33ee3e51218 admin admin] RBAC: Authorization granted {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:16:58.568876 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.notifications [None req-402294c9-5238-4154-9d98-c33ee3e51218 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'dc8bee754f9a43af9f047fe10308200e'} {{(pid=80378) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 10:16:58.577192 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.notifications [None req-402294c9-5238-4154-9d98-c33ee3e51218 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'dc8bee754f9a43af9f047fe10308200e'} {{(pid=80378) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 10:16:58.584475 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.notifications [None req-402294c9-5238-4154-9d98-c33ee3e51218 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'dc8bee754f9a43af9f047fe10308200e'} {{(pid=80378) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 10:16:58.585158 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.notifications [None req-402294c9-5238-4154-9d98-c33ee3e51218 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'dc8bee754f9a43af9f047fe10308200e'} {{(pid=80378) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 10:16:58.598657 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.notifications [None req-402294c9-5238-4154-9d98-c33ee3e51218 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'dc8bee754f9a43af9f047fe10308200e'} {{(pid=80378) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 10:16:58.605195 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 944/1890] 10.222.0.82 () {58 vars in 1224 bytes} [Tue Aug 2 10:16:58 2022] DELETE /identity/v3/users/dc8bee754f9a43af9f047fe10308200e => generated 0 bytes in 168 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 02 10:16:58.610626 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cef346e8-d5e0-4764-b73e-cc90b49c1758 None None] REQUEST_METHOD: `POST` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:16:58.610946 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cef346e8-d5e0-4764-b73e-cc90b49c1758 None None] SCRIPT_NAME: `/identity` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:16:58.611212 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cef346e8-d5e0-4764-b73e-cc90b49c1758 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:16:58.633249 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.auth.core [None req-cef346e8-d5e0-4764-b73e-cc90b49c1758 None None] MFA Rules not processed for user `891005fa5ff74041b75747d16cc66798`. Rule list: `[]` (Enabled: `True`). {{(pid=80377) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 02 10:16:58.634515 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.fernet_utils [None req-cef346e8-d5e0-4764-b73e-cc90b49c1758 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80377) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 10:16:58.700149 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: [pid: 80377|app: 0|req: 947/1891] 10.222.0.82 () {58 vars in 958 bytes} [Tue Aug 2 10:16:58 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 91 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 02 10:16:58.710717 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-402294c9-5238-4154-9d98-c33ee3e51218 admin admin] Authenticating user token {{(pid=80378) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:16:58.712339 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.fernet_utils [None req-402294c9-5238-4154-9d98-c33ee3e51218 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80378) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 10:16:58.735760 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG dbcounter [-] [80378] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80378) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 10:16:58.785225 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-402294c9-5238-4154-9d98-c33ee3e51218 admin admin] Validating token access rules against request {{(pid=80378) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:16:58.788748 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2413cbb-b5f8-4de9-9676-3fb9ce6cdb0a service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '33b87170f0634b67bce60750847b5cbe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f1b38ebb804c41c69772c998aae72c41', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80378) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:16:58.789369 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2413cbb-b5f8-4de9-9676-3fb9ce6cdb0a service neutron] REQUEST_METHOD: `GET` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:16:58.789647 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2413cbb-b5f8-4de9-9676-3fb9ce6cdb0a service neutron] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:16:58.789958 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2413cbb-b5f8-4de9-9676-3fb9ce6cdb0a service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:16:58.799621 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2413cbb-b5f8-4de9-9676-3fb9ce6cdb0a service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:16:58.801057 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2413cbb-b5f8-4de9-9676-3fb9ce6cdb0a service neutron] RBAC: Authorization granted {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:16:58.811499 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 945/1892] 10.222.0.82 () {60 vars in 1365 bytes} [Tue Aug 2 10:16:58 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 101 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 02 10:16:59.091858 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cef346e8-d5e0-4764-b73e-cc90b49c1758 None None] Authenticating user token {{(pid=80377) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:16:59.093760 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.fernet_utils [None req-cef346e8-d5e0-4764-b73e-cc90b49c1758 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80377) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 10:16:59.105868 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cef346e8-d5e0-4764-b73e-cc90b49c1758 None None] Validating token access rules against request {{(pid=80377) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:16:59.111040 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e67e3d53-6ae2-4f74-b60c-7b0735a790fd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '891005fa5ff74041b75747d16cc66798', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7047912c5acd4adeba5d87c3bf895e39', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80377) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:16:59.111696 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e67e3d53-6ae2-4f74-b60c-7b0735a790fd admin admin] REQUEST_METHOD: `DELETE` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:16:59.111975 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e67e3d53-6ae2-4f74-b60c-7b0735a790fd admin admin] SCRIPT_NAME: `/identity` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:16:59.112250 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e67e3d53-6ae2-4f74-b60c-7b0735a790fd admin admin] PATH_INFO: `/v3/projects/8816c13a4da74d3191285b9a4c2361f2` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:16:59.113465 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e67e3d53-6ae2-4f74-b60c-7b0735a790fd admin admin] RBAC: Authorizing `identity:delete_project(project_id=8816c13a4da74d3191285b9a4c2361f2)` {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:16:59.116010 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e67e3d53-6ae2-4f74-b60c-7b0735a790fd admin admin] RBAC: Authorization granted {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:16:59.154415 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG dbcounter [-] [80377] Writing DB stats keystone:SELECT=93,keystone:INSERT=5,keystone:DELETE=2 {{(pid=80377) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 10:16:59.171079 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.notifications [None req-e67e3d53-6ae2-4f74-b60c-7b0735a790fd admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '8816c13a4da74d3191285b9a4c2361f2', 'request_id': 'req-e67e3d53-6ae2-4f74-b60c-7b0735a790fd'} {{(pid=80377) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 10:16:59.178519 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.notifications [None req-e67e3d53-6ae2-4f74-b60c-7b0735a790fd admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '8816c13a4da74d3191285b9a4c2361f2', 'request_id': 'req-e67e3d53-6ae2-4f74-b60c-7b0735a790fd'} {{(pid=80377) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 10:16:59.186792 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.notifications [None req-e67e3d53-6ae2-4f74-b60c-7b0735a790fd admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '8816c13a4da74d3191285b9a4c2361f2', 'request_id': 'req-e67e3d53-6ae2-4f74-b60c-7b0735a790fd'} {{(pid=80377) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 10:16:59.202955 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.notifications [None req-e67e3d53-6ae2-4f74-b60c-7b0735a790fd admin admin] The token cache is being invalidate because project 8816c13a4da74d3191285b9a4c2361f2 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80377) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 02 10:16:59.203322 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.notifications [None req-e67e3d53-6ae2-4f74-b60c-7b0735a790fd admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80377) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 10:16:59.203983 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.notifications [None req-e67e3d53-6ae2-4f74-b60c-7b0735a790fd admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80377) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 10:16:59.208679 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: [pid: 80377|app: 0|req: 948/1893] 10.222.0.82 () {58 vars in 1233 bytes} [Tue Aug 2 10:16:59 2022] DELETE /identity/v3/projects/8816c13a4da74d3191285b9a4c2361f2 => generated 0 bytes in 114 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 02 10:16:59.209600 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2413cbb-b5f8-4de9-9676-3fb9ce6cdb0a service neutron] Authenticating user token {{(pid=80378) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:16:59.219942 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2413cbb-b5f8-4de9-9676-3fb9ce6cdb0a service neutron] Validating token access rules against request {{(pid=80378) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:16:59.282024 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe860ae2-02db-4a8d-83ff-901f94d2e259 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '891005fa5ff74041b75747d16cc66798', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7047912c5acd4adeba5d87c3bf895e39', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80378) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:16:59.282215 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe860ae2-02db-4a8d-83ff-901f94d2e259 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:16:59.282414 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe860ae2-02db-4a8d-83ff-901f94d2e259 admin admin] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:16:59.282691 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe860ae2-02db-4a8d-83ff-901f94d2e259 admin admin] PATH_INFO: `/v3/users/99d3ab27ed8f4332b5c97882c47fe14a` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:16:59.283922 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe860ae2-02db-4a8d-83ff-901f94d2e259 admin admin] RBAC: Authorizing `identity:delete_user(user_id=99d3ab27ed8f4332b5c97882c47fe14a)` {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:16:59.285919 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe860ae2-02db-4a8d-83ff-901f94d2e259 admin admin] RBAC: Authorization granted {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:16:59.354672 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.notifications [None req-fe860ae2-02db-4a8d-83ff-901f94d2e259 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '99d3ab27ed8f4332b5c97882c47fe14a'} {{(pid=80378) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 10:16:59.363582 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.notifications [None req-fe860ae2-02db-4a8d-83ff-901f94d2e259 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '99d3ab27ed8f4332b5c97882c47fe14a'} {{(pid=80378) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 10:16:59.373542 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.notifications [None req-fe860ae2-02db-4a8d-83ff-901f94d2e259 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '99d3ab27ed8f4332b5c97882c47fe14a'} {{(pid=80378) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 10:16:59.374432 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.notifications [None req-fe860ae2-02db-4a8d-83ff-901f94d2e259 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '99d3ab27ed8f4332b5c97882c47fe14a'} {{(pid=80378) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 10:16:59.388393 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.notifications [None req-fe860ae2-02db-4a8d-83ff-901f94d2e259 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '99d3ab27ed8f4332b5c97882c47fe14a'} {{(pid=80378) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 10:16:59.394859 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 946/1894] 10.222.0.82 () {58 vars in 1224 bytes} [Tue Aug 2 10:16:59 2022] DELETE /identity/v3/users/99d3ab27ed8f4332b5c97882c47fe14a => generated 0 bytes in 186 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 02 10:16:59.678137 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e67e3d53-6ae2-4f74-b60c-7b0735a790fd admin admin] Authenticating user token {{(pid=80377) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:16:59.679671 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.fernet_utils [None req-e67e3d53-6ae2-4f74-b60c-7b0735a790fd admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80377) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 10:16:59.693298 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e67e3d53-6ae2-4f74-b60c-7b0735a790fd admin admin] Validating token access rules against request {{(pid=80377) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:16:59.804027 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f67d1ef-7e13-4e5c-a4b1-588bebeda7cc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '891005fa5ff74041b75747d16cc66798', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7047912c5acd4adeba5d87c3bf895e39', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80377) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:16:59.809042 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f67d1ef-7e13-4e5c-a4b1-588bebeda7cc admin admin] REQUEST_METHOD: `DELETE` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:16:59.809616 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f67d1ef-7e13-4e5c-a4b1-588bebeda7cc admin admin] SCRIPT_NAME: `/identity` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:16:59.811919 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f67d1ef-7e13-4e5c-a4b1-588bebeda7cc admin admin] PATH_INFO: `/v3/projects/89608e25cc4e4acaab453433b8b926f6` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:16:59.815612 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f67d1ef-7e13-4e5c-a4b1-588bebeda7cc admin admin] RBAC: Authorizing `identity:delete_project(project_id=89608e25cc4e4acaab453433b8b926f6)` {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:16:59.820894 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f67d1ef-7e13-4e5c-a4b1-588bebeda7cc admin admin] RBAC: Authorization granted {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:16:59.877957 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.notifications [None req-5f67d1ef-7e13-4e5c-a4b1-588bebeda7cc admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '89608e25cc4e4acaab453433b8b926f6', 'request_id': 'req-5f67d1ef-7e13-4e5c-a4b1-588bebeda7cc'} {{(pid=80377) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 10:16:59.888535 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.notifications [None req-5f67d1ef-7e13-4e5c-a4b1-588bebeda7cc admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '89608e25cc4e4acaab453433b8b926f6', 'request_id': 'req-5f67d1ef-7e13-4e5c-a4b1-588bebeda7cc'} {{(pid=80377) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 10:16:59.895410 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.notifications [None req-5f67d1ef-7e13-4e5c-a4b1-588bebeda7cc admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '89608e25cc4e4acaab453433b8b926f6', 'request_id': 'req-5f67d1ef-7e13-4e5c-a4b1-588bebeda7cc'} {{(pid=80377) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 10:16:59.911986 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.notifications [None req-5f67d1ef-7e13-4e5c-a4b1-588bebeda7cc admin admin] The token cache is being invalidate because project 89608e25cc4e4acaab453433b8b926f6 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80377) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 02 10:16:59.912447 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.notifications [None req-5f67d1ef-7e13-4e5c-a4b1-588bebeda7cc admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80377) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 10:16:59.913217 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.notifications [None req-5f67d1ef-7e13-4e5c-a4b1-588bebeda7cc admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80377) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 10:16:59.914960 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: [pid: 80377|app: 0|req: 949/1895] 10.222.0.82 () {58 vars in 1233 bytes} [Tue Aug 2 10:16:59 2022] DELETE /identity/v3/projects/89608e25cc4e4acaab453433b8b926f6 => generated 0 bytes in 238 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 02 10:16:59.920639 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba394266-8091-4cd0-b444-363edbb01dde None None] REQUEST_METHOD: `POST` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:16:59.921103 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba394266-8091-4cd0-b444-363edbb01dde None None] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:16:59.921532 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba394266-8091-4cd0-b444-363edbb01dde None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:16:59.946444 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.auth.core [None req-ba394266-8091-4cd0-b444-363edbb01dde None None] MFA Rules not processed for user `891005fa5ff74041b75747d16cc66798`. Rule list: `[]` (Enabled: `True`). {{(pid=80378) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 02 10:16:59.947667 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.fernet_utils [None req-ba394266-8091-4cd0-b444-363edbb01dde None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80378) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 10:16:59.988197 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG dbcounter [-] [80378] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80378) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 10:17:00.015096 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 947/1896] 10.222.0.82 () {58 vars in 958 bytes} [Tue Aug 2 10:16:59 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 02 10:17:00.023058 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3b9052c-7c70-41ff-93dd-94dfd557c604 None None] REQUEST_METHOD: `POST` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:17:00.023473 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3b9052c-7c70-41ff-93dd-94dfd557c604 None None] SCRIPT_NAME: `/identity` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:17:00.023748 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3b9052c-7c70-41ff-93dd-94dfd557c604 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:17:00.044373 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.auth.core [None req-b3b9052c-7c70-41ff-93dd-94dfd557c604 None None] MFA Rules not processed for user `891005fa5ff74041b75747d16cc66798`. Rule list: `[]` (Enabled: `True`). {{(pid=80377) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 02 10:17:00.045594 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.fernet_utils [None req-b3b9052c-7c70-41ff-93dd-94dfd557c604 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80377) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 10:17:00.053886 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: [pid: 80377|app: 0|req: 950/1897] 10.222.0.82 () {58 vars in 958 bytes} [Tue Aug 2 10:17:00 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 02 10:17:00.058345 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba394266-8091-4cd0-b444-363edbb01dde None None] Authenticating user token {{(pid=80378) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:17:00.070089 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba394266-8091-4cd0-b444-363edbb01dde None None] Validating token access rules against request {{(pid=80378) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:17:00.076997 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-413094da-9f8a-4d61-a1cb-4c534d8d5370 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '891005fa5ff74041b75747d16cc66798', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7047912c5acd4adeba5d87c3bf895e39', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80378) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:17:00.077644 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-413094da-9f8a-4d61-a1cb-4c534d8d5370 admin admin] REQUEST_METHOD: `GET` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:17:00.077909 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-413094da-9f8a-4d61-a1cb-4c534d8d5370 admin admin] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:17:00.078180 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-413094da-9f8a-4d61-a1cb-4c534d8d5370 admin admin] PATH_INFO: `/v3/domains` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:17:00.078884 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-413094da-9f8a-4d61-a1cb-4c534d8d5370 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80378) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 02 10:17:00.079186 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-413094da-9f8a-4d61-a1cb-4c534d8d5370 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:17:00.081038 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-413094da-9f8a-4d61-a1cb-4c534d8d5370 admin admin] RBAC: Authorization granted {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:17:00.107867 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 948/1898] 10.222.0.82 () {58 vars in 1153 bytes} [Tue Aug 2 10:17:00 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 50 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 02 10:17:00.108729 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3b9052c-7c70-41ff-93dd-94dfd557c604 None None] Authenticating user token {{(pid=80377) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:17:00.119274 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3b9052c-7c70-41ff-93dd-94dfd557c604 None None] Validating token access rules against request {{(pid=80377) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:17:00.124434 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7c77df1-249c-4cb5-a802-21a7fc683e0e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '891005fa5ff74041b75747d16cc66798', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7047912c5acd4adeba5d87c3bf895e39', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80377) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:17:00.125189 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7c77df1-249c-4cb5-a802-21a7fc683e0e admin admin] REQUEST_METHOD: `POST` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:17:00.125999 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7c77df1-249c-4cb5-a802-21a7fc683e0e admin admin] SCRIPT_NAME: `/identity` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:17:00.126381 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7c77df1-249c-4cb5-a802-21a7fc683e0e admin admin] PATH_INFO: `/v3/projects` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:17:00.127165 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7c77df1-249c-4cb5-a802-21a7fc683e0e admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:17:00.129246 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7c77df1-249c-4cb5-a802-21a7fc683e0e admin admin] RBAC: Authorization granted {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:17:00.165832 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: [pid: 80377|app: 0|req: 951/1899] 10.222.0.82 () {60 vars in 1153 bytes} [Tue Aug 2 10:17:00 2022] POST /identity/v3/projects => generated 365 bytes in 57 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 02 10:17:00.169283 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-413094da-9f8a-4d61-a1cb-4c534d8d5370 admin admin] Authenticating user token {{(pid=80378) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:17:00.180702 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-413094da-9f8a-4d61-a1cb-4c534d8d5370 admin admin] Validating token access rules against request {{(pid=80378) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:17:00.248823 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24279838-5a43-4e59-b515-19309908b2ba admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '891005fa5ff74041b75747d16cc66798', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7047912c5acd4adeba5d87c3bf895e39', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80378) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:17:00.249578 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24279838-5a43-4e59-b515-19309908b2ba admin admin] REQUEST_METHOD: `POST` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:17:00.249990 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24279838-5a43-4e59-b515-19309908b2ba admin admin] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:17:00.250397 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24279838-5a43-4e59-b515-19309908b2ba admin admin] PATH_INFO: `/v3/users` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:17:00.252255 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24279838-5a43-4e59-b515-19309908b2ba admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:17:00.254401 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24279838-5a43-4e59-b515-19309908b2ba admin admin] RBAC: Authorization granted {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:17:00.291839 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 949/1900] 10.222.0.82 () {60 vars in 1144 bytes} [Tue Aug 2 10:17:00 2022] POST /identity/v3/users => generated 283 bytes in 120 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 02 10:17:00.293139 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7c77df1-249c-4cb5-a802-21a7fc683e0e admin admin] Authenticating user token {{(pid=80377) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:17:00.304998 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7c77df1-249c-4cb5-a802-21a7fc683e0e admin admin] Validating token access rules against request {{(pid=80377) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:17:00.310569 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32aaa5e5-81f3-4f57-aac9-02fbe7c9b186 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '891005fa5ff74041b75747d16cc66798', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7047912c5acd4adeba5d87c3bf895e39', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80377) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:17:00.311882 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32aaa5e5-81f3-4f57-aac9-02fbe7c9b186 admin admin] REQUEST_METHOD: `GET` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:17:00.312236 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32aaa5e5-81f3-4f57-aac9-02fbe7c9b186 admin admin] SCRIPT_NAME: `/identity` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:17:00.312584 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32aaa5e5-81f3-4f57-aac9-02fbe7c9b186 admin admin] PATH_INFO: `/v3/roles` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:17:00.313361 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32aaa5e5-81f3-4f57-aac9-02fbe7c9b186 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:17:00.315087 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32aaa5e5-81f3-4f57-aac9-02fbe7c9b186 admin admin] RBAC: Authorization granted {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:17:00.327744 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: [pid: 80377|app: 0|req: 952/1901] 10.222.0.82 () {58 vars in 1122 bytes} [Tue Aug 2 10:17:00 2022] GET /identity/v3/roles => generated 1420 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 02 10:17:00.333190 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24279838-5a43-4e59-b515-19309908b2ba admin admin] Authenticating user token {{(pid=80378) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:17:00.353450 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24279838-5a43-4e59-b515-19309908b2ba admin admin] Validating token access rules against request {{(pid=80378) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:17:00.362554 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e92a2a6-724a-431e-9fe4-f09fd87252fa admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '891005fa5ff74041b75747d16cc66798', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7047912c5acd4adeba5d87c3bf895e39', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80378) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:17:00.363195 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e92a2a6-724a-431e-9fe4-f09fd87252fa admin admin] REQUEST_METHOD: `GET` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:17:00.363488 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e92a2a6-724a-431e-9fe4-f09fd87252fa admin admin] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:17:00.363944 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e92a2a6-724a-431e-9fe4-f09fd87252fa admin admin] PATH_INFO: `/v3/roles` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:17:00.365947 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e92a2a6-724a-431e-9fe4-f09fd87252fa admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:17:00.366364 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e92a2a6-724a-431e-9fe4-f09fd87252fa admin admin] RBAC: Authorization granted {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:17:00.381855 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 950/1902] 10.222.0.82 () {58 vars in 1122 bytes} [Tue Aug 2 10:17:00 2022] GET /identity/v3/roles => generated 1420 bytes in 49 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 02 10:17:00.382887 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32aaa5e5-81f3-4f57-aac9-02fbe7c9b186 admin admin] Authenticating user token {{(pid=80377) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:17:00.395188 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32aaa5e5-81f3-4f57-aac9-02fbe7c9b186 admin admin] Validating token access rules against request {{(pid=80377) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:17:00.400609 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f5af84e-3631-4cf0-946b-368420d0da5e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '891005fa5ff74041b75747d16cc66798', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7047912c5acd4adeba5d87c3bf895e39', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80377) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:17:00.401399 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f5af84e-3631-4cf0-946b-368420d0da5e admin admin] REQUEST_METHOD: `PUT` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:17:00.401813 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f5af84e-3631-4cf0-946b-368420d0da5e admin admin] SCRIPT_NAME: `/identity` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:17:00.402196 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f5af84e-3631-4cf0-946b-368420d0da5e admin admin] PATH_INFO: `/v3/projects/aea8ccf0fd71475da7205bd1db84c609/users/eb2783502f874dfaa315ea300113932c/roles/e1c04b19fc4543b8b54f58bbcffbc598` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:17:00.412590 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG dbcounter [-] [80377] Writing DB stats keystone:SELECT=85,keystone:DELETE=12,keystone:INSERT=3 {{(pid=80377) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 10:17:00.423158 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f5af84e-3631-4cf0-946b-368420d0da5e admin admin] RBAC: Authorizing `identity:create_grant(project_id=aea8ccf0fd71475da7205bd1db84c609, user_id=eb2783502f874dfaa315ea300113932c, role_id=e1c04b19fc4543b8b54f58bbcffbc598)` {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:17:00.426336 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f5af84e-3631-4cf0-946b-368420d0da5e admin admin] RBAC: Authorization granted {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:17:00.442677 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: [pid: 80377|app: 0|req: 953/1903] 10.222.0.82 () {60 vars in 1483 bytes} [Tue Aug 2 10:17:00 2022] PUT /identity/v3/projects/aea8ccf0fd71475da7205bd1db84c609/users/eb2783502f874dfaa315ea300113932c/roles/e1c04b19fc4543b8b54f58bbcffbc598 => generated 0 bytes in 60 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 02 10:17:00.453915 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63478242-16a8-4df6-bc88-a7fb043fe71d None None] REQUEST_METHOD: `POST` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:17:00.454216 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63478242-16a8-4df6-bc88-a7fb043fe71d None None] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:17:00.454496 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63478242-16a8-4df6-bc88-a7fb043fe71d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:17:00.487043 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.auth.core [None req-63478242-16a8-4df6-bc88-a7fb043fe71d None None] MFA Rules not processed for user `eb2783502f874dfaa315ea300113932c`. Rule list: `[]` (Enabled: `True`). {{(pid=80378) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 02 10:17:00.488288 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.fernet_utils [None req-63478242-16a8-4df6-bc88-a7fb043fe71d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80378) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 10:17:00.561641 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 951/1904] 10.222.0.82 () {58 vars in 958 bytes} [Tue Aug 2 10:17:00 2022] POST /identity/v3/auth/tokens => generated 3242 bytes in 109 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 02 10:17:00.566157 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f5af84e-3631-4cf0-946b-368420d0da5e admin admin] Authenticating user token {{(pid=80377) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:17:00.639390 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f5af84e-3631-4cf0-946b-368420d0da5e admin admin] Validating token access rules against request {{(pid=80377) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:17:00.644644 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1de4a34c-1db7-4f56-b06b-bd7fe67ed9b1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '891005fa5ff74041b75747d16cc66798', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7047912c5acd4adeba5d87c3bf895e39', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80377) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:17:00.645538 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1de4a34c-1db7-4f56-b06b-bd7fe67ed9b1 admin admin] REQUEST_METHOD: `POST` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:17:00.645847 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1de4a34c-1db7-4f56-b06b-bd7fe67ed9b1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:17:00.646130 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1de4a34c-1db7-4f56-b06b-bd7fe67ed9b1 admin admin] PATH_INFO: `/v3/projects` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:17:00.646814 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1de4a34c-1db7-4f56-b06b-bd7fe67ed9b1 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:17:00.649716 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1de4a34c-1db7-4f56-b06b-bd7fe67ed9b1 admin admin] RBAC: Authorization granted {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:17:00.688120 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: [pid: 80377|app: 0|req: 954/1905] 10.222.0.82 () {60 vars in 1153 bytes} [Tue Aug 2 10:17:00 2022] POST /identity/v3/projects => generated 367 bytes in 122 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 02 10:17:00.691931 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63478242-16a8-4df6-bc88-a7fb043fe71d None None] Authenticating user token {{(pid=80378) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:17:00.708937 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63478242-16a8-4df6-bc88-a7fb043fe71d None None] Validating token access rules against request {{(pid=80378) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:17:00.754950 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG dbcounter [-] [80378] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80378) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 10:17:00.804524 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87e4a880-9a10-4db3-9126-ccf9a0a0972e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '891005fa5ff74041b75747d16cc66798', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7047912c5acd4adeba5d87c3bf895e39', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80378) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:17:00.805459 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87e4a880-9a10-4db3-9126-ccf9a0a0972e admin admin] REQUEST_METHOD: `POST` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:17:00.806085 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87e4a880-9a10-4db3-9126-ccf9a0a0972e admin admin] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:17:00.806511 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87e4a880-9a10-4db3-9126-ccf9a0a0972e admin admin] PATH_INFO: `/v3/users` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:17:00.807510 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87e4a880-9a10-4db3-9126-ccf9a0a0972e admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:17:00.812291 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87e4a880-9a10-4db3-9126-ccf9a0a0972e admin admin] RBAC: Authorization granted {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:17:00.863212 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 952/1906] 10.222.0.82 () {60 vars in 1144 bytes} [Tue Aug 2 10:17:00 2022] POST /identity/v3/users => generated 290 bytes in 169 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 02 10:17:00.864109 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1de4a34c-1db7-4f56-b06b-bd7fe67ed9b1 admin admin] Authenticating user token {{(pid=80377) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:17:00.894379 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1de4a34c-1db7-4f56-b06b-bd7fe67ed9b1 admin admin] Validating token access rules against request {{(pid=80377) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:17:00.903838 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-538289dc-0218-4bce-b7ee-7d50dacf1cc5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '891005fa5ff74041b75747d16cc66798', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7047912c5acd4adeba5d87c3bf895e39', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80377) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:17:00.904774 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-538289dc-0218-4bce-b7ee-7d50dacf1cc5 admin admin] REQUEST_METHOD: `GET` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:17:00.905176 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-538289dc-0218-4bce-b7ee-7d50dacf1cc5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:17:00.905527 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-538289dc-0218-4bce-b7ee-7d50dacf1cc5 admin admin] PATH_INFO: `/v3/roles` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:17:00.907097 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-538289dc-0218-4bce-b7ee-7d50dacf1cc5 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:17:00.908915 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-538289dc-0218-4bce-b7ee-7d50dacf1cc5 admin admin] RBAC: Authorization granted {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:17:00.932391 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: [pid: 80377|app: 0|req: 955/1907] 10.222.0.82 () {58 vars in 1122 bytes} [Tue Aug 2 10:17:00 2022] GET /identity/v3/roles => generated 1420 bytes in 69 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 02 10:17:00.937407 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87e4a880-9a10-4db3-9126-ccf9a0a0972e admin admin] Authenticating user token {{(pid=80378) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:17:00.964581 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87e4a880-9a10-4db3-9126-ccf9a0a0972e admin admin] Validating token access rules against request {{(pid=80378) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:17:00.971474 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8af199cb-5dd2-4103-8155-11f99eb65eab admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '891005fa5ff74041b75747d16cc66798', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7047912c5acd4adeba5d87c3bf895e39', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80378) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:17:00.972486 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8af199cb-5dd2-4103-8155-11f99eb65eab admin admin] REQUEST_METHOD: `PUT` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:17:00.973366 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8af199cb-5dd2-4103-8155-11f99eb65eab admin admin] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:17:00.973988 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8af199cb-5dd2-4103-8155-11f99eb65eab admin admin] PATH_INFO: `/v3/projects/5723260254ef4af6a96055d5cfabe5d5/users/45ab63261e53456cb3af5553d167a93f/roles/237966442f05411191105bf336bc14f3` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:17:00.998478 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8af199cb-5dd2-4103-8155-11f99eb65eab admin admin] RBAC: Authorizing `identity:create_grant(project_id=5723260254ef4af6a96055d5cfabe5d5, user_id=45ab63261e53456cb3af5553d167a93f, role_id=237966442f05411191105bf336bc14f3)` {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:17:01.004494 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8af199cb-5dd2-4103-8155-11f99eb65eab admin admin] RBAC: Authorization granted {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:17:01.025807 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 953/1908] 10.222.0.82 () {60 vars in 1483 bytes} [Tue Aug 2 10:17:00 2022] PUT /identity/v3/projects/5723260254ef4af6a96055d5cfabe5d5/users/45ab63261e53456cb3af5553d167a93f/roles/237966442f05411191105bf336bc14f3 => generated 0 bytes in 86 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 02 10:17:01.026784 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-538289dc-0218-4bce-b7ee-7d50dacf1cc5 admin admin] Authenticating user token {{(pid=80377) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:17:01.048541 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-538289dc-0218-4bce-b7ee-7d50dacf1cc5 admin admin] Validating token access rules against request {{(pid=80377) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:17:01.148869 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37217e77-c5c7-492f-9922-a995f5396895 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '891005fa5ff74041b75747d16cc66798', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7047912c5acd4adeba5d87c3bf895e39', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80377) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:17:01.148869 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37217e77-c5c7-492f-9922-a995f5396895 admin admin] REQUEST_METHOD: `GET` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:17:01.148869 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37217e77-c5c7-492f-9922-a995f5396895 admin admin] SCRIPT_NAME: `/identity` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:17:01.148869 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37217e77-c5c7-492f-9922-a995f5396895 admin admin] PATH_INFO: `/v3/roles` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:17:01.149754 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37217e77-c5c7-492f-9922-a995f5396895 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:17:01.151080 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37217e77-c5c7-492f-9922-a995f5396895 admin admin] RBAC: Authorization granted {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:17:01.168032 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8af199cb-5dd2-4103-8155-11f99eb65eab admin admin] Authenticating user token {{(pid=80378) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:17:01.176850 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: [pid: 80377|app: 0|req: 956/1909] 10.222.0.82 () {58 vars in 1122 bytes} [Tue Aug 2 10:17:01 2022] GET /identity/v3/roles => generated 1420 bytes in 145 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 02 10:17:01.184141 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8af199cb-5dd2-4103-8155-11f99eb65eab admin admin] Validating token access rules against request {{(pid=80378) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:17:01.189178 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1326ef1-5a73-4c4e-897e-d08c227fcf4b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '891005fa5ff74041b75747d16cc66798', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7047912c5acd4adeba5d87c3bf895e39', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80378) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:17:01.189987 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1326ef1-5a73-4c4e-897e-d08c227fcf4b admin admin] REQUEST_METHOD: `PUT` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:17:01.190340 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1326ef1-5a73-4c4e-897e-d08c227fcf4b admin admin] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:17:01.190682 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1326ef1-5a73-4c4e-897e-d08c227fcf4b admin admin] PATH_INFO: `/v3/projects/5723260254ef4af6a96055d5cfabe5d5/users/45ab63261e53456cb3af5553d167a93f/roles/237966442f05411191105bf336bc14f3` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:17:01.192916 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1326ef1-5a73-4c4e-897e-d08c227fcf4b admin admin] RBAC: Authorizing `identity:create_grant(project_id=5723260254ef4af6a96055d5cfabe5d5, user_id=45ab63261e53456cb3af5553d167a93f, role_id=237966442f05411191105bf336bc14f3)` {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:17:01.196601 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1326ef1-5a73-4c4e-897e-d08c227fcf4b admin admin] RBAC: Authorization granted {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:17:01.209300 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 954/1910] 10.222.0.82 () {60 vars in 1483 bytes} [Tue Aug 2 10:17:01 2022] PUT /identity/v3/projects/5723260254ef4af6a96055d5cfabe5d5/users/45ab63261e53456cb3af5553d167a93f/roles/237966442f05411191105bf336bc14f3 => generated 0 bytes in 42 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 02 10:17:01.223822 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0450a1d8-e6f2-45e3-be4e-0686f18c8d14 None None] REQUEST_METHOD: `POST` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:17:01.223822 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0450a1d8-e6f2-45e3-be4e-0686f18c8d14 None None] SCRIPT_NAME: `/identity` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:17:01.223822 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0450a1d8-e6f2-45e3-be4e-0686f18c8d14 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:17:01.257442 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.auth.core [None req-0450a1d8-e6f2-45e3-be4e-0686f18c8d14 None None] MFA Rules not processed for user `45ab63261e53456cb3af5553d167a93f`. Rule list: `[]` (Enabled: `True`). {{(pid=80377) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 02 10:17:01.258979 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.fernet_utils [None req-0450a1d8-e6f2-45e3-be4e-0686f18c8d14 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80377) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 10:17:01.345856 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG dbcounter [-] [80377] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80377) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 10:17:01.400894 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: [pid: 80377|app: 0|req: 957/1911] 10.222.0.82 () {58 vars in 958 bytes} [Tue Aug 2 10:17:01 2022] POST /identity/v3/auth/tokens => generated 3311 bytes in 180 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 02 10:17:01.414553 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1326ef1-5a73-4c4e-897e-d08c227fcf4b admin admin] Authenticating user token {{(pid=80378) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:17:01.417701 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.fernet_utils [None req-a1326ef1-5a73-4c4e-897e-d08c227fcf4b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80378) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 10:17:01.535319 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1326ef1-5a73-4c4e-897e-d08c227fcf4b admin admin] Validating token access rules against request {{(pid=80378) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:17:01.541680 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-363a1eda-6f75-45c8-9af2-025e35dbf16a service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1023de281ae74e9d8a6d6c312c1c97a7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f1b38ebb804c41c69772c998aae72c41', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80378) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:17:01.542351 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-363a1eda-6f75-45c8-9af2-025e35dbf16a service cinder] REQUEST_METHOD: `GET` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:17:01.542637 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-363a1eda-6f75-45c8-9af2-025e35dbf16a service cinder] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:17:01.542914 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-363a1eda-6f75-45c8-9af2-025e35dbf16a service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:17:01.557834 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-363a1eda-6f75-45c8-9af2-025e35dbf16a service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:17:01.559337 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-363a1eda-6f75-45c8-9af2-025e35dbf16a service cinder] RBAC: Authorization granted {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:17:01.570869 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 955/1912] 10.222.0.82 () {60 vars in 1365 bytes} [Tue Aug 2 10:17:01 2022] GET /identity/v3/auth/tokens => generated 3311 bytes in 157 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 02 10:17:01.674414 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0450a1d8-e6f2-45e3-be4e-0686f18c8d14 None None] Authenticating user token {{(pid=80377) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:17:01.692480 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0450a1d8-e6f2-45e3-be4e-0686f18c8d14 None None] Validating token access rules against request {{(pid=80377) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:17:01.700086 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5cbadb25-7337-468b-933d-274b1a34cbd8 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1023de281ae74e9d8a6d6c312c1c97a7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f1b38ebb804c41c69772c998aae72c41', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80377) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:17:01.701651 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cbadb25-7337-468b-933d-274b1a34cbd8 service cinder] REQUEST_METHOD: `GET` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:17:01.702057 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cbadb25-7337-468b-933d-274b1a34cbd8 service cinder] SCRIPT_NAME: `/identity` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:17:01.702443 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cbadb25-7337-468b-933d-274b1a34cbd8 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:17:01.809682 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5cbadb25-7337-468b-933d-274b1a34cbd8 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:17:01.812622 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5cbadb25-7337-468b-933d-274b1a34cbd8 service cinder] RBAC: Authorization granted {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:17:01.826125 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: [pid: 80377|app: 0|req: 958/1913] 10.222.0.82 () {60 vars in 1365 bytes} [Tue Aug 2 10:17:01 2022] GET /identity/v3/auth/tokens => generated 3242 bytes in 152 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 02 10:17:02.723579 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-363a1eda-6f75-45c8-9af2-025e35dbf16a service cinder] Authenticating user token {{(pid=80378) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:17:02.725024 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.fernet_utils [None req-363a1eda-6f75-45c8-9af2-025e35dbf16a service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80378) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 10:17:02.805252 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-363a1eda-6f75-45c8-9af2-025e35dbf16a service cinder] Validating token access rules against request {{(pid=80378) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:17:02.810897 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e34b22c0-dab9-48db-b06d-3fd12ac1b032 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '891005fa5ff74041b75747d16cc66798', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7047912c5acd4adeba5d87c3bf895e39', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80378) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:17:02.811567 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e34b22c0-dab9-48db-b06d-3fd12ac1b032 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:17:02.811857 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e34b22c0-dab9-48db-b06d-3fd12ac1b032 admin admin] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:17:02.812253 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e34b22c0-dab9-48db-b06d-3fd12ac1b032 admin admin] PATH_INFO: `/v3/users/3808aca2179747538f6ee19cf9b1c817` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:17:02.813501 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e34b22c0-dab9-48db-b06d-3fd12ac1b032 admin admin] RBAC: Authorizing `identity:delete_user(user_id=3808aca2179747538f6ee19cf9b1c817)` {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:17:02.815493 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e34b22c0-dab9-48db-b06d-3fd12ac1b032 admin admin] RBAC: Authorization granted {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:17:02.866119 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG dbcounter [-] [80378] Writing DB stats keystone:SELECT=90,keystone:INSERT=5,keystone:DELETE=5 {{(pid=80378) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 10:17:02.876398 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf392c43-2fa9-4121-affb-131512c15634 None None] REQUEST_METHOD: `GET` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:17:02.877726 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf392c43-2fa9-4121-affb-131512c15634 None None] SCRIPT_NAME: `/identity` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:17:02.878332 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf392c43-2fa9-4121-affb-131512c15634 None None] PATH_INFO: `/` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:17:02.886021 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.notifications [None req-e34b22c0-dab9-48db-b06d-3fd12ac1b032 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '3808aca2179747538f6ee19cf9b1c817'} {{(pid=80378) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 10:17:02.886466 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: [pid: 80377|app: 0|req: 959/1914] 10.222.0.82 () {54 vars in 921 bytes} [Tue Aug 2 10:17:02 2022] GET /identity => generated 270 bytes in 11 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 02 10:17:02.898961 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-126c6204-19de-4851-a8b9-ba3a7fc9ad21 None None] REQUEST_METHOD: `POST` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:17:02.899251 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-126c6204-19de-4851-a8b9-ba3a7fc9ad21 None None] SCRIPT_NAME: `/identity` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:17:02.899587 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-126c6204-19de-4851-a8b9-ba3a7fc9ad21 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:17:02.910168 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.notifications [None req-e34b22c0-dab9-48db-b06d-3fd12ac1b032 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '3808aca2179747538f6ee19cf9b1c817'} {{(pid=80378) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 10:17:02.922957 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.notifications [None req-e34b22c0-dab9-48db-b06d-3fd12ac1b032 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '3808aca2179747538f6ee19cf9b1c817'} {{(pid=80378) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 10:17:02.923722 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.notifications [None req-e34b22c0-dab9-48db-b06d-3fd12ac1b032 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '3808aca2179747538f6ee19cf9b1c817'} {{(pid=80378) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 10:17:02.928018 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.auth.core [None req-126c6204-19de-4851-a8b9-ba3a7fc9ad21 None None] MFA Rules not processed for user `88af518196674fe593ad88d30111fb7d`. Rule list: `[]` (Enabled: `True`). {{(pid=80377) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 02 10:17:02.929242 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.fernet_utils [None req-126c6204-19de-4851-a8b9-ba3a7fc9ad21 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80377) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 10:17:02.942378 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.notifications [None req-e34b22c0-dab9-48db-b06d-3fd12ac1b032 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '3808aca2179747538f6ee19cf9b1c817'} {{(pid=80378) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 10:17:02.949726 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 956/1915] 10.222.0.82 () {58 vars in 1224 bytes} [Tue Aug 2 10:17:02 2022] DELETE /identity/v3/users/3808aca2179747538f6ee19cf9b1c817 => generated 0 bytes in 227 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 02 10:17:02.954199 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c974ab70-8803-40ed-87ca-56fc5f5e6923 None None] REQUEST_METHOD: `POST` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:17:02.954607 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c974ab70-8803-40ed-87ca-56fc5f5e6923 None None] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:17:02.954987 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c974ab70-8803-40ed-87ca-56fc5f5e6923 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:17:02.976949 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.auth.core [None req-c974ab70-8803-40ed-87ca-56fc5f5e6923 None None] MFA Rules not processed for user `891005fa5ff74041b75747d16cc66798`. Rule list: `[]` (Enabled: `True`). {{(pid=80378) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 02 10:17:02.978363 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.fernet_utils [None req-c974ab70-8803-40ed-87ca-56fc5f5e6923 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80378) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 10:17:03.010999 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: [pid: 80377|app: 0|req: 960/1916] 10.222.0.82 () {58 vars in 1018 bytes} [Tue Aug 2 10:17:02 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 113 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 02 10:17:03.023631 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-126c6204-19de-4851-a8b9-ba3a7fc9ad21 None None] Authenticating user token {{(pid=80377) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:17:03.025327 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.fernet_utils [None req-126c6204-19de-4851-a8b9-ba3a7fc9ad21 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80377) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 10:17:03.038776 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-126c6204-19de-4851-a8b9-ba3a7fc9ad21 None None] Validating token access rules against request {{(pid=80377) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:17:03.044779 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 957/1917] 10.222.0.82 () {58 vars in 958 bytes} [Tue Aug 2 10:17:02 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 91 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 02 10:17:03.054794 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c974ab70-8803-40ed-87ca-56fc5f5e6923 None None] Authenticating user token {{(pid=80378) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:17:03.056538 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.fernet_utils [None req-c974ab70-8803-40ed-87ca-56fc5f5e6923 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80378) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 10:17:03.107065 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15e80412-8417-4c31-82c2-785d26aa5298 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88af518196674fe593ad88d30111fb7d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f1b38ebb804c41c69772c998aae72c41', 'project_domain_id': 'default', 'roles': ['admin', 'ResellerAdmin', 'reader', 'service', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80377) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:17:03.107607 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15e80412-8417-4c31-82c2-785d26aa5298 service nova] REQUEST_METHOD: `GET` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:17:03.107891 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15e80412-8417-4c31-82c2-785d26aa5298 service nova] SCRIPT_NAME: `/identity` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:17:03.108166 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15e80412-8417-4c31-82c2-785d26aa5298 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:17:03.114304 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c974ab70-8803-40ed-87ca-56fc5f5e6923 None None] Validating token access rules against request {{(pid=80378) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:17:03.117210 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15e80412-8417-4c31-82c2-785d26aa5298 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:17:03.118283 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27c63ec7-59b8-4734-908c-d7755d7bd297 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '33b87170f0634b67bce60750847b5cbe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f1b38ebb804c41c69772c998aae72c41', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80378) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:17:03.119111 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15e80412-8417-4c31-82c2-785d26aa5298 service nova] RBAC: Authorization granted {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:17:03.119234 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27c63ec7-59b8-4734-908c-d7755d7bd297 service neutron] REQUEST_METHOD: `GET` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:17:03.119343 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27c63ec7-59b8-4734-908c-d7755d7bd297 service neutron] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:17:03.119558 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27c63ec7-59b8-4734-908c-d7755d7bd297 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:17:03.123835 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG dbcounter [-] [80377] Writing DB stats keystone:SELECT=100 {{(pid=80377) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 10:17:03.128989 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27c63ec7-59b8-4734-908c-d7755d7bd297 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:17:03.130461 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27c63ec7-59b8-4734-908c-d7755d7bd297 service neutron] RBAC: Authorization granted {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:17:03.134464 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: [pid: 80377|app: 0|req: 961/1918] 10.222.0.82 () {60 vars in 1365 bytes} [Tue Aug 2 10:17:03 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 111 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 02 10:17:03.141382 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 958/1919] 10.222.0.82 () {60 vars in 1365 bytes} [Tue Aug 2 10:17:03 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 87 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 02 10:17:03.454504 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15e80412-8417-4c31-82c2-785d26aa5298 service nova] Authenticating user token {{(pid=80377) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:17:03.456191 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.fernet_utils [None req-15e80412-8417-4c31-82c2-785d26aa5298 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80377) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 10:17:03.477337 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15e80412-8417-4c31-82c2-785d26aa5298 service nova] Validating token access rules against request {{(pid=80377) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:17:03.494911 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a53df760-ce02-415c-a046-af24282d1268 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '891005fa5ff74041b75747d16cc66798', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7047912c5acd4adeba5d87c3bf895e39', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80377) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:17:03.495657 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a53df760-ce02-415c-a046-af24282d1268 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:17:03.495950 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a53df760-ce02-415c-a046-af24282d1268 admin admin] SCRIPT_NAME: `/identity` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:17:03.496273 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a53df760-ce02-415c-a046-af24282d1268 admin admin] PATH_INFO: `/v3/projects/cd52f29ebc21465197ac524eaf664d73` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:17:03.498790 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a53df760-ce02-415c-a046-af24282d1268 admin admin] RBAC: Authorizing `identity:delete_project(project_id=cd52f29ebc21465197ac524eaf664d73)` {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:17:03.500939 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a53df760-ce02-415c-a046-af24282d1268 admin admin] RBAC: Authorization granted {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:17:03.560780 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.notifications [None req-a53df760-ce02-415c-a046-af24282d1268 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'cd52f29ebc21465197ac524eaf664d73', 'request_id': 'req-a53df760-ce02-415c-a046-af24282d1268'} {{(pid=80377) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 10:17:03.569245 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.notifications [None req-a53df760-ce02-415c-a046-af24282d1268 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'cd52f29ebc21465197ac524eaf664d73', 'request_id': 'req-a53df760-ce02-415c-a046-af24282d1268'} {{(pid=80377) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 10:17:03.578005 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.notifications [None req-a53df760-ce02-415c-a046-af24282d1268 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'cd52f29ebc21465197ac524eaf664d73', 'request_id': 'req-a53df760-ce02-415c-a046-af24282d1268'} {{(pid=80377) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 10:17:03.593835 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.notifications [None req-a53df760-ce02-415c-a046-af24282d1268 admin admin] The token cache is being invalidate because project cd52f29ebc21465197ac524eaf664d73 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80377) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 02 10:17:03.594294 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.notifications [None req-a53df760-ce02-415c-a046-af24282d1268 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80377) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 10:17:03.595431 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.notifications [None req-a53df760-ce02-415c-a046-af24282d1268 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80377) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 10:17:03.597413 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: [pid: 80377|app: 0|req: 962/1920] 10.222.0.82 () {58 vars in 1233 bytes} [Tue Aug 2 10:17:03 2022] DELETE /identity/v3/projects/cd52f29ebc21465197ac524eaf664d73 => generated 0 bytes in 144 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 02 10:17:03.602256 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27c63ec7-59b8-4734-908c-d7755d7bd297 service neutron] Authenticating user token {{(pid=80378) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:17:03.615896 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27c63ec7-59b8-4734-908c-d7755d7bd297 service neutron] Validating token access rules against request {{(pid=80378) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:17:03.684873 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG dbcounter [-] [80378] Writing DB stats keystone:SELECT=94,keystone:DELETE=5,keystone:INSERT=1 {{(pid=80378) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 10:17:03.695412 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89b02062-058a-4e55-abe3-b49de39324bc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '891005fa5ff74041b75747d16cc66798', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7047912c5acd4adeba5d87c3bf895e39', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80378) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:17:03.697271 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89b02062-058a-4e55-abe3-b49de39324bc admin admin] REQUEST_METHOD: `DELETE` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:17:03.698123 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89b02062-058a-4e55-abe3-b49de39324bc admin admin] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:17:03.699559 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89b02062-058a-4e55-abe3-b49de39324bc admin admin] PATH_INFO: `/v3/users/582a6da97d03445bad651118d5b215e2` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:17:03.702738 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89b02062-058a-4e55-abe3-b49de39324bc admin admin] RBAC: Authorizing `identity:delete_user(user_id=582a6da97d03445bad651118d5b215e2)` {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:17:03.705722 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89b02062-058a-4e55-abe3-b49de39324bc admin admin] RBAC: Authorization granted {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:17:03.788983 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.notifications [None req-89b02062-058a-4e55-abe3-b49de39324bc admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '582a6da97d03445bad651118d5b215e2'} {{(pid=80378) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 10:17:03.802480 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.notifications [None req-89b02062-058a-4e55-abe3-b49de39324bc admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '582a6da97d03445bad651118d5b215e2'} {{(pid=80378) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 10:17:03.824937 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.notifications [None req-89b02062-058a-4e55-abe3-b49de39324bc admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '582a6da97d03445bad651118d5b215e2'} {{(pid=80378) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 10:17:03.826800 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.notifications [None req-89b02062-058a-4e55-abe3-b49de39324bc admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '582a6da97d03445bad651118d5b215e2'} {{(pid=80378) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 10:17:03.846657 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.notifications [None req-89b02062-058a-4e55-abe3-b49de39324bc admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '582a6da97d03445bad651118d5b215e2'} {{(pid=80378) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 10:17:03.853680 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 959/1921] 10.222.0.82 () {58 vars in 1224 bytes} [Tue Aug 2 10:17:03 2022] DELETE /identity/v3/users/582a6da97d03445bad651118d5b215e2 => generated 0 bytes in 252 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 02 10:17:04.248189 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a53df760-ce02-415c-a046-af24282d1268 admin admin] Authenticating user token {{(pid=80377) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:17:04.250051 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.fernet_utils [None req-a53df760-ce02-415c-a046-af24282d1268 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80377) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 10:17:04.272582 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a53df760-ce02-415c-a046-af24282d1268 admin admin] Validating token access rules against request {{(pid=80377) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:17:04.341734 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20df214a-885f-4671-b990-51d7b4624fca admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '891005fa5ff74041b75747d16cc66798', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7047912c5acd4adeba5d87c3bf895e39', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80377) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:17:04.342417 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20df214a-885f-4671-b990-51d7b4624fca admin admin] REQUEST_METHOD: `DELETE` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:17:04.342709 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20df214a-885f-4671-b990-51d7b4624fca admin admin] SCRIPT_NAME: `/identity` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:17:04.342987 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20df214a-885f-4671-b990-51d7b4624fca admin admin] PATH_INFO: `/v3/projects/eecc188f6ae346b4aa105725a443d8c4` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:17:04.344242 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20df214a-885f-4671-b990-51d7b4624fca admin admin] RBAC: Authorizing `identity:delete_project(project_id=eecc188f6ae346b4aa105725a443d8c4)` {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:17:04.346543 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20df214a-885f-4671-b990-51d7b4624fca admin admin] RBAC: Authorization granted {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:17:04.408137 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.notifications [None req-20df214a-885f-4671-b990-51d7b4624fca admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'eecc188f6ae346b4aa105725a443d8c4', 'request_id': 'req-20df214a-885f-4671-b990-51d7b4624fca'} {{(pid=80377) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 10:17:04.414231 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.notifications [None req-20df214a-885f-4671-b990-51d7b4624fca admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'eecc188f6ae346b4aa105725a443d8c4', 'request_id': 'req-20df214a-885f-4671-b990-51d7b4624fca'} {{(pid=80377) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 10:17:04.421026 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.notifications [None req-20df214a-885f-4671-b990-51d7b4624fca admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'eecc188f6ae346b4aa105725a443d8c4', 'request_id': 'req-20df214a-885f-4671-b990-51d7b4624fca'} {{(pid=80377) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 10:17:04.435524 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.notifications [None req-20df214a-885f-4671-b990-51d7b4624fca admin admin] The token cache is being invalidate because project eecc188f6ae346b4aa105725a443d8c4 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80377) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 02 10:17:04.436071 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.notifications [None req-20df214a-885f-4671-b990-51d7b4624fca admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80377) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 10:17:04.436946 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.notifications [None req-20df214a-885f-4671-b990-51d7b4624fca admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80377) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 10:17:04.439064 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: [pid: 80377|app: 0|req: 963/1922] 10.222.0.82 () {58 vars in 1233 bytes} [Tue Aug 2 10:17:04 2022] DELETE /identity/v3/projects/eecc188f6ae346b4aa105725a443d8c4 => generated 0 bytes in 192 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 02 10:17:04.446142 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bec2ca73-0c08-49fc-a31b-3078fc104481 None None] REQUEST_METHOD: `POST` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:17:04.446432 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bec2ca73-0c08-49fc-a31b-3078fc104481 None None] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:17:04.446709 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bec2ca73-0c08-49fc-a31b-3078fc104481 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:17:04.467015 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.auth.core [None req-bec2ca73-0c08-49fc-a31b-3078fc104481 None None] MFA Rules not processed for user `891005fa5ff74041b75747d16cc66798`. Rule list: `[]` (Enabled: `True`). {{(pid=80378) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 02 10:17:04.468226 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.fernet_utils [None req-bec2ca73-0c08-49fc-a31b-3078fc104481 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80378) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 10:17:04.539557 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 960/1923] 10.222.0.82 () {58 vars in 958 bytes} [Tue Aug 2 10:17:04 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 02 10:17:04.550447 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f72a85b7-b43e-4414-9fad-b98e82f65038 None None] REQUEST_METHOD: `POST` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:17:04.550728 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f72a85b7-b43e-4414-9fad-b98e82f65038 None None] SCRIPT_NAME: `/identity` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:17:04.551005 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f72a85b7-b43e-4414-9fad-b98e82f65038 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:17:04.574588 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.auth.core [None req-f72a85b7-b43e-4414-9fad-b98e82f65038 None None] MFA Rules not processed for user `891005fa5ff74041b75747d16cc66798`. Rule list: `[]` (Enabled: `True`). {{(pid=80377) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 02 10:17:04.575887 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.fernet_utils [None req-f72a85b7-b43e-4414-9fad-b98e82f65038 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80377) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 10:17:04.589126 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: [pid: 80377|app: 0|req: 964/1924] 10.222.0.82 () {58 vars in 958 bytes} [Tue Aug 2 10:17:04 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 37 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 02 10:17:04.590893 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bec2ca73-0c08-49fc-a31b-3078fc104481 None None] Authenticating user token {{(pid=80378) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:17:04.605805 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bec2ca73-0c08-49fc-a31b-3078fc104481 None None] Validating token access rules against request {{(pid=80378) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:17:04.611510 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d02a9e96-5430-4b4f-b9f6-b510624f7ea2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '891005fa5ff74041b75747d16cc66798', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7047912c5acd4adeba5d87c3bf895e39', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80378) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:17:04.612124 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d02a9e96-5430-4b4f-b9f6-b510624f7ea2 admin admin] REQUEST_METHOD: `GET` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:17:04.612438 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d02a9e96-5430-4b4f-b9f6-b510624f7ea2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:17:04.612705 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d02a9e96-5430-4b4f-b9f6-b510624f7ea2 admin admin] PATH_INFO: `/v3/domains` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:17:04.613446 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d02a9e96-5430-4b4f-b9f6-b510624f7ea2 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80378) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 02 10:17:04.613735 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d02a9e96-5430-4b4f-b9f6-b510624f7ea2 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:17:04.615375 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d02a9e96-5430-4b4f-b9f6-b510624f7ea2 admin admin] RBAC: Authorization granted {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:17:04.634089 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 961/1925] 10.222.0.82 () {58 vars in 1153 bytes} [Tue Aug 2 10:17:04 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 44 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 02 10:17:04.643338 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f72a85b7-b43e-4414-9fad-b98e82f65038 None None] Authenticating user token {{(pid=80377) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:17:04.652990 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG dbcounter [-] [80377] Writing DB stats keystone:SELECT=84,keystone:DELETE=14,keystone:INSERT=2 {{(pid=80377) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 10:17:04.657138 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f72a85b7-b43e-4414-9fad-b98e82f65038 None None] Validating token access rules against request {{(pid=80377) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:17:04.670674 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c121f129-b35c-45fe-b91f-cc11d778c886 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '891005fa5ff74041b75747d16cc66798', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7047912c5acd4adeba5d87c3bf895e39', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80377) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:17:04.671635 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c121f129-b35c-45fe-b91f-cc11d778c886 admin admin] REQUEST_METHOD: `POST` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:17:04.672531 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c121f129-b35c-45fe-b91f-cc11d778c886 admin admin] SCRIPT_NAME: `/identity` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:17:04.673265 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c121f129-b35c-45fe-b91f-cc11d778c886 admin admin] PATH_INFO: `/v3/projects` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:17:04.673598 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c121f129-b35c-45fe-b91f-cc11d778c886 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:17:04.675689 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c121f129-b35c-45fe-b91f-cc11d778c886 admin admin] RBAC: Authorization granted {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:17:04.712399 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: [pid: 80377|app: 0|req: 965/1926] 10.222.0.82 () {60 vars in 1153 bytes} [Tue Aug 2 10:17:04 2022] POST /identity/v3/projects => generated 387 bytes in 67 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 02 10:17:04.713867 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d02a9e96-5430-4b4f-b9f6-b510624f7ea2 admin admin] Authenticating user token {{(pid=80378) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:17:04.725010 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d02a9e96-5430-4b4f-b9f6-b510624f7ea2 admin admin] Validating token access rules against request {{(pid=80378) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:17:04.811043 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1cf9fa8-60f8-4f0d-9280-63d9078377ca admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '891005fa5ff74041b75747d16cc66798', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7047912c5acd4adeba5d87c3bf895e39', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80378) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:17:04.811790 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1cf9fa8-60f8-4f0d-9280-63d9078377ca admin admin] REQUEST_METHOD: `POST` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:17:04.812399 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1cf9fa8-60f8-4f0d-9280-63d9078377ca admin admin] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:17:04.812669 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1cf9fa8-60f8-4f0d-9280-63d9078377ca admin admin] PATH_INFO: `/v3/users` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:17:04.813355 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1cf9fa8-60f8-4f0d-9280-63d9078377ca admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:17:04.816021 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1cf9fa8-60f8-4f0d-9280-63d9078377ca admin admin] RBAC: Authorization granted {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:17:04.828138 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG dbcounter [-] [80378] Writing DB stats keystone:SELECT=88,keystone:DELETE=10,keystone:INSERT=2 {{(pid=80378) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 10:17:04.854830 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c121f129-b35c-45fe-b91f-cc11d778c886 admin admin] Authenticating user token {{(pid=80377) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:17:04.857878 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 962/1927] 10.222.0.82 () {60 vars in 1144 bytes} [Tue Aug 2 10:17:04 2022] POST /identity/v3/users => generated 294 bytes in 142 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 02 10:17:04.869585 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c121f129-b35c-45fe-b91f-cc11d778c886 admin admin] Validating token access rules against request {{(pid=80377) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:17:04.879053 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3ebf3a2-7342-4f89-ab01-24b45521ecd4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '891005fa5ff74041b75747d16cc66798', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7047912c5acd4adeba5d87c3bf895e39', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80377) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:17:04.879053 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3ebf3a2-7342-4f89-ab01-24b45521ecd4 admin admin] REQUEST_METHOD: `GET` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:17:04.879053 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3ebf3a2-7342-4f89-ab01-24b45521ecd4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:17:04.879053 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3ebf3a2-7342-4f89-ab01-24b45521ecd4 admin admin] PATH_INFO: `/v3/roles` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:17:04.879483 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3ebf3a2-7342-4f89-ab01-24b45521ecd4 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:17:04.879483 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3ebf3a2-7342-4f89-ab01-24b45521ecd4 admin admin] RBAC: Authorization granted {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:17:04.892886 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: [pid: 80377|app: 0|req: 966/1928] 10.222.0.82 () {58 vars in 1122 bytes} [Tue Aug 2 10:17:04 2022] GET /identity/v3/roles => generated 1420 bytes in 39 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 02 10:17:04.897072 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1cf9fa8-60f8-4f0d-9280-63d9078377ca admin admin] Authenticating user token {{(pid=80378) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:17:04.909147 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1cf9fa8-60f8-4f0d-9280-63d9078377ca admin admin] Validating token access rules against request {{(pid=80378) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:17:04.916024 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6715b638-81f1-491c-86a1-15effef303b1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '891005fa5ff74041b75747d16cc66798', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7047912c5acd4adeba5d87c3bf895e39', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80378) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:17:04.916525 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6715b638-81f1-491c-86a1-15effef303b1 admin admin] REQUEST_METHOD: `GET` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:17:04.916812 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6715b638-81f1-491c-86a1-15effef303b1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:17:04.917095 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6715b638-81f1-491c-86a1-15effef303b1 admin admin] PATH_INFO: `/v3/roles` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:17:04.917896 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6715b638-81f1-491c-86a1-15effef303b1 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:17:04.919586 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6715b638-81f1-491c-86a1-15effef303b1 admin admin] RBAC: Authorization granted {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:17:04.935186 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 963/1929] 10.222.0.82 () {58 vars in 1122 bytes} [Tue Aug 2 10:17:04 2022] GET /identity/v3/roles => generated 1420 bytes in 39 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 02 10:17:04.940207 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3ebf3a2-7342-4f89-ab01-24b45521ecd4 admin admin] Authenticating user token {{(pid=80377) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:17:04.962314 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3ebf3a2-7342-4f89-ab01-24b45521ecd4 admin admin] Validating token access rules against request {{(pid=80377) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:17:04.967821 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de28faee-47da-46c2-8389-1d8ab35851ef admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '891005fa5ff74041b75747d16cc66798', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7047912c5acd4adeba5d87c3bf895e39', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80377) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:17:04.968504 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de28faee-47da-46c2-8389-1d8ab35851ef admin admin] REQUEST_METHOD: `PUT` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:17:04.969094 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de28faee-47da-46c2-8389-1d8ab35851ef admin admin] SCRIPT_NAME: `/identity` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:17:04.969439 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de28faee-47da-46c2-8389-1d8ab35851ef admin admin] PATH_INFO: `/v3/projects/094bdaea97554859bfb556a99a1e92f6/users/1477e6cc48f44609b3d2e1eb59c34185/roles/e1c04b19fc4543b8b54f58bbcffbc598` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:17:04.994250 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de28faee-47da-46c2-8389-1d8ab35851ef admin admin] RBAC: Authorizing `identity:create_grant(project_id=094bdaea97554859bfb556a99a1e92f6, user_id=1477e6cc48f44609b3d2e1eb59c34185, role_id=e1c04b19fc4543b8b54f58bbcffbc598)` {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:17:04.998056 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de28faee-47da-46c2-8389-1d8ab35851ef admin admin] RBAC: Authorization granted {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:17:05.024212 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: [pid: 80377|app: 0|req: 967/1930] 10.222.0.82 () {60 vars in 1483 bytes} [Tue Aug 2 10:17:04 2022] PUT /identity/v3/projects/094bdaea97554859bfb556a99a1e92f6/users/1477e6cc48f44609b3d2e1eb59c34185/roles/e1c04b19fc4543b8b54f58bbcffbc598 => generated 0 bytes in 84 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 02 10:17:05.035733 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccef5175-cb9b-44da-96a9-ba977e2f6b6a None None] REQUEST_METHOD: `POST` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:17:05.036091 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccef5175-cb9b-44da-96a9-ba977e2f6b6a None None] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:17:05.036363 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccef5175-cb9b-44da-96a9-ba977e2f6b6a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:17:05.066701 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.auth.core [None req-ccef5175-cb9b-44da-96a9-ba977e2f6b6a None None] MFA Rules not processed for user `1477e6cc48f44609b3d2e1eb59c34185`. Rule list: `[]` (Enabled: `True`). {{(pid=80378) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 02 10:17:05.067937 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.fernet_utils [None req-ccef5175-cb9b-44da-96a9-ba977e2f6b6a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80378) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 10:17:05.141078 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 964/1931] 10.222.0.82 () {58 vars in 958 bytes} [Tue Aug 2 10:17:05 2022] POST /identity/v3/auth/tokens => generated 3264 bytes in 106 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 02 10:17:05.145474 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de28faee-47da-46c2-8389-1d8ab35851ef admin admin] Authenticating user token {{(pid=80377) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:17:05.213173 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de28faee-47da-46c2-8389-1d8ab35851ef admin admin] Validating token access rules against request {{(pid=80377) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:17:05.218654 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-435c176f-d6c8-4545-89cc-26f60aa29430 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '891005fa5ff74041b75747d16cc66798', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7047912c5acd4adeba5d87c3bf895e39', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80377) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:17:05.219351 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-435c176f-d6c8-4545-89cc-26f60aa29430 admin admin] REQUEST_METHOD: `POST` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:17:05.219698 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-435c176f-d6c8-4545-89cc-26f60aa29430 admin admin] SCRIPT_NAME: `/identity` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:17:05.220036 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-435c176f-d6c8-4545-89cc-26f60aa29430 admin admin] PATH_INFO: `/v3/projects` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:17:05.220772 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-435c176f-d6c8-4545-89cc-26f60aa29430 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:17:05.222793 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-435c176f-d6c8-4545-89cc-26f60aa29430 admin admin] RBAC: Authorization granted {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:17:05.256271 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: [pid: 80377|app: 0|req: 968/1932] 10.222.0.82 () {60 vars in 1153 bytes} [Tue Aug 2 10:17:05 2022] POST /identity/v3/projects => generated 387 bytes in 108 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 02 10:17:05.257989 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ccef5175-cb9b-44da-96a9-ba977e2f6b6a None None] Authenticating user token {{(pid=80378) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:17:05.270029 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ccef5175-cb9b-44da-96a9-ba977e2f6b6a None None] Validating token access rules against request {{(pid=80378) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:17:05.345042 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85565158-a823-417c-bf0d-ea725de39f3a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '891005fa5ff74041b75747d16cc66798', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7047912c5acd4adeba5d87c3bf895e39', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80378) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:17:05.345681 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85565158-a823-417c-bf0d-ea725de39f3a admin admin] REQUEST_METHOD: `POST` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:17:05.345997 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85565158-a823-417c-bf0d-ea725de39f3a admin admin] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:17:05.346278 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85565158-a823-417c-bf0d-ea725de39f3a admin admin] PATH_INFO: `/v3/users` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:17:05.346955 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85565158-a823-417c-bf0d-ea725de39f3a admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:17:05.348913 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85565158-a823-417c-bf0d-ea725de39f3a admin admin] RBAC: Authorization granted {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:17:05.371639 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 965/1933] 10.222.0.82 () {60 vars in 1144 bytes} [Tue Aug 2 10:17:05 2022] POST /identity/v3/users => generated 300 bytes in 114 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 02 10:17:05.378254 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-435c176f-d6c8-4545-89cc-26f60aa29430 admin admin] Authenticating user token {{(pid=80377) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:17:05.389726 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-435c176f-d6c8-4545-89cc-26f60aa29430 admin admin] Validating token access rules against request {{(pid=80377) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:17:05.394699 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f92cb07e-4f14-48d4-a074-c71a0e60390d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '891005fa5ff74041b75747d16cc66798', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7047912c5acd4adeba5d87c3bf895e39', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80377) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:17:05.395252 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f92cb07e-4f14-48d4-a074-c71a0e60390d admin admin] REQUEST_METHOD: `GET` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:17:05.395531 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f92cb07e-4f14-48d4-a074-c71a0e60390d admin admin] SCRIPT_NAME: `/identity` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:17:05.395805 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f92cb07e-4f14-48d4-a074-c71a0e60390d admin admin] PATH_INFO: `/v3/roles` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:17:05.397395 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f92cb07e-4f14-48d4-a074-c71a0e60390d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:17:05.402807 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f92cb07e-4f14-48d4-a074-c71a0e60390d admin admin] RBAC: Authorization granted {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:17:05.418928 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: [pid: 80377|app: 0|req: 969/1934] 10.222.0.82 () {58 vars in 1122 bytes} [Tue Aug 2 10:17:05 2022] GET /identity/v3/roles => generated 1420 bytes in 41 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 02 10:17:05.423219 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85565158-a823-417c-bf0d-ea725de39f3a admin admin] Authenticating user token {{(pid=80378) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:17:05.434608 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85565158-a823-417c-bf0d-ea725de39f3a admin admin] Validating token access rules against request {{(pid=80378) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:17:05.441898 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c77e7151-dff7-437c-a16f-fff3ee142892 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '891005fa5ff74041b75747d16cc66798', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7047912c5acd4adeba5d87c3bf895e39', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80378) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:17:05.443195 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c77e7151-dff7-437c-a16f-fff3ee142892 admin admin] REQUEST_METHOD: `PUT` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:17:05.443523 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c77e7151-dff7-437c-a16f-fff3ee142892 admin admin] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:17:05.444040 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c77e7151-dff7-437c-a16f-fff3ee142892 admin admin] PATH_INFO: `/v3/projects/a9c7a9cc95944c7aa3b94b4d7492a77c/users/3604c4b85615466b984c1fb93bf4e81a/roles/237966442f05411191105bf336bc14f3` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:17:05.474806 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c77e7151-dff7-437c-a16f-fff3ee142892 admin admin] RBAC: Authorizing `identity:create_grant(project_id=a9c7a9cc95944c7aa3b94b4d7492a77c, user_id=3604c4b85615466b984c1fb93bf4e81a, role_id=237966442f05411191105bf336bc14f3)` {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:17:05.478998 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c77e7151-dff7-437c-a16f-fff3ee142892 admin admin] RBAC: Authorization granted {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:17:05.501505 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 966/1935] 10.222.0.82 () {60 vars in 1483 bytes} [Tue Aug 2 10:17:05 2022] PUT /identity/v3/projects/a9c7a9cc95944c7aa3b94b4d7492a77c/users/3604c4b85615466b984c1fb93bf4e81a/roles/237966442f05411191105bf336bc14f3 => generated 0 bytes in 79 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 02 10:17:05.507419 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f92cb07e-4f14-48d4-a074-c71a0e60390d admin admin] Authenticating user token {{(pid=80377) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:17:05.522970 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f92cb07e-4f14-48d4-a074-c71a0e60390d admin admin] Validating token access rules against request {{(pid=80377) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:17:05.579129 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4d20fd0-7989-40ee-a3f3-6b57e52f0a8c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '891005fa5ff74041b75747d16cc66798', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7047912c5acd4adeba5d87c3bf895e39', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80377) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:17:05.579751 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4d20fd0-7989-40ee-a3f3-6b57e52f0a8c admin admin] REQUEST_METHOD: `GET` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:17:05.580035 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4d20fd0-7989-40ee-a3f3-6b57e52f0a8c admin admin] SCRIPT_NAME: `/identity` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:17:05.580308 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4d20fd0-7989-40ee-a3f3-6b57e52f0a8c admin admin] PATH_INFO: `/v3/roles` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:17:05.580999 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4d20fd0-7989-40ee-a3f3-6b57e52f0a8c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:17:05.582634 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4d20fd0-7989-40ee-a3f3-6b57e52f0a8c admin admin] RBAC: Authorization granted {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:17:05.593523 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: [pid: 80377|app: 0|req: 970/1936] 10.222.0.82 () {58 vars in 1122 bytes} [Tue Aug 2 10:17:05 2022] GET /identity/v3/roles => generated 1420 bytes in 87 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 02 10:17:05.597698 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c77e7151-dff7-437c-a16f-fff3ee142892 admin admin] Authenticating user token {{(pid=80378) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:17:05.609088 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c77e7151-dff7-437c-a16f-fff3ee142892 admin admin] Validating token access rules against request {{(pid=80378) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:17:05.615037 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac20d496-3e86-4b1d-95a9-794b50ef8e06 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '891005fa5ff74041b75747d16cc66798', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7047912c5acd4adeba5d87c3bf895e39', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80378) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:17:05.615859 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac20d496-3e86-4b1d-95a9-794b50ef8e06 admin admin] REQUEST_METHOD: `PUT` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:17:05.616151 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac20d496-3e86-4b1d-95a9-794b50ef8e06 admin admin] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:17:05.616434 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac20d496-3e86-4b1d-95a9-794b50ef8e06 admin admin] PATH_INFO: `/v3/projects/a9c7a9cc95944c7aa3b94b4d7492a77c/users/3604c4b85615466b984c1fb93bf4e81a/roles/237966442f05411191105bf336bc14f3` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:17:05.618470 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac20d496-3e86-4b1d-95a9-794b50ef8e06 admin admin] RBAC: Authorizing `identity:create_grant(project_id=a9c7a9cc95944c7aa3b94b4d7492a77c, user_id=3604c4b85615466b984c1fb93bf4e81a, role_id=237966442f05411191105bf336bc14f3)` {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:17:05.621139 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac20d496-3e86-4b1d-95a9-794b50ef8e06 admin admin] RBAC: Authorization granted {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:17:05.631805 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 967/1937] 10.222.0.82 () {60 vars in 1483 bytes} [Tue Aug 2 10:17:05 2022] PUT /identity/v3/projects/a9c7a9cc95944c7aa3b94b4d7492a77c/users/3604c4b85615466b984c1fb93bf4e81a/roles/237966442f05411191105bf336bc14f3 => generated 0 bytes in 35 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 02 10:17:05.643768 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89438923-e2f3-418e-80a9-5307a3b8b07c None None] REQUEST_METHOD: `POST` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:17:05.644064 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89438923-e2f3-418e-80a9-5307a3b8b07c None None] SCRIPT_NAME: `/identity` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:17:05.644346 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89438923-e2f3-418e-80a9-5307a3b8b07c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:17:05.674120 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.auth.core [None req-89438923-e2f3-418e-80a9-5307a3b8b07c None None] MFA Rules not processed for user `3604c4b85615466b984c1fb93bf4e81a`. Rule list: `[]` (Enabled: `True`). {{(pid=80377) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 02 10:17:05.675242 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.fernet_utils [None req-89438923-e2f3-418e-80a9-5307a3b8b07c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80377) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 10:17:05.680397 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG dbcounter [-] [80377] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80377) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 10:17:05.746274 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: [pid: 80377|app: 0|req: 971/1938] 10.222.0.82 () {58 vars in 958 bytes} [Tue Aug 2 10:17:05 2022] POST /identity/v3/auth/tokens => generated 3331 bytes in 104 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 02 10:17:05.758150 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac20d496-3e86-4b1d-95a9-794b50ef8e06 admin admin] Authenticating user token {{(pid=80378) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:17:05.759859 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.fernet_utils [None req-ac20d496-3e86-4b1d-95a9-794b50ef8e06 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80378) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 10:17:05.786547 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG dbcounter [-] [80378] Writing DB stats keystone:INSERT=7,keystone:SELECT=93 {{(pid=80378) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 10:17:05.819097 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac20d496-3e86-4b1d-95a9-794b50ef8e06 admin admin] Validating token access rules against request {{(pid=80378) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:17:05.822933 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eec20d97-fc52-4f29-a987-c0b36b690975 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1023de281ae74e9d8a6d6c312c1c97a7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f1b38ebb804c41c69772c998aae72c41', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80378) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:17:05.823550 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eec20d97-fc52-4f29-a987-c0b36b690975 service cinder] REQUEST_METHOD: `GET` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:17:05.823831 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eec20d97-fc52-4f29-a987-c0b36b690975 service cinder] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:17:05.824105 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eec20d97-fc52-4f29-a987-c0b36b690975 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:17:05.833981 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eec20d97-fc52-4f29-a987-c0b36b690975 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:17:05.835403 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eec20d97-fc52-4f29-a987-c0b36b690975 service cinder] RBAC: Authorization granted {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:17:05.844999 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 968/1939] 10.222.0.82 () {60 vars in 1365 bytes} [Tue Aug 2 10:17:05 2022] GET /identity/v3/auth/tokens => generated 3331 bytes in 87 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 02 10:17:05.874349 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89438923-e2f3-418e-80a9-5307a3b8b07c None None] Authenticating user token {{(pid=80377) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:17:05.951555 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89438923-e2f3-418e-80a9-5307a3b8b07c None None] Validating token access rules against request {{(pid=80377) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:17:05.951749 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d891d94-45d2-417d-af25-0471610c87fd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '891005fa5ff74041b75747d16cc66798', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7047912c5acd4adeba5d87c3bf895e39', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80377) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:17:05.952263 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d891d94-45d2-417d-af25-0471610c87fd admin admin] REQUEST_METHOD: `DELETE` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:17:05.952544 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d891d94-45d2-417d-af25-0471610c87fd admin admin] SCRIPT_NAME: `/identity` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:17:05.952819 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d891d94-45d2-417d-af25-0471610c87fd admin admin] PATH_INFO: `/v3/users/1477e6cc48f44609b3d2e1eb59c34185` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:17:05.954067 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d891d94-45d2-417d-af25-0471610c87fd admin admin] RBAC: Authorizing `identity:delete_user(user_id=1477e6cc48f44609b3d2e1eb59c34185)` {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:17:05.956026 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d891d94-45d2-417d-af25-0471610c87fd admin admin] RBAC: Authorization granted {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:17:06.003501 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.notifications [None req-9d891d94-45d2-417d-af25-0471610c87fd admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '1477e6cc48f44609b3d2e1eb59c34185'} {{(pid=80377) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 10:17:06.010415 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.notifications [None req-9d891d94-45d2-417d-af25-0471610c87fd admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '1477e6cc48f44609b3d2e1eb59c34185'} {{(pid=80377) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 10:17:06.018925 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.notifications [None req-9d891d94-45d2-417d-af25-0471610c87fd admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '1477e6cc48f44609b3d2e1eb59c34185'} {{(pid=80377) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 10:17:06.019636 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.notifications [None req-9d891d94-45d2-417d-af25-0471610c87fd admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '1477e6cc48f44609b3d2e1eb59c34185'} {{(pid=80377) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 10:17:06.036468 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.notifications [None req-9d891d94-45d2-417d-af25-0471610c87fd admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '1477e6cc48f44609b3d2e1eb59c34185'} {{(pid=80377) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 10:17:06.048169 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: [pid: 80377|app: 0|req: 972/1940] 10.222.0.82 () {58 vars in 1224 bytes} [Tue Aug 2 10:17:05 2022] DELETE /identity/v3/users/1477e6cc48f44609b3d2e1eb59c34185 => generated 0 bytes in 174 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 02 10:17:06.053147 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e3e4dde-27a5-4016-8a6e-20164bfd430c None None] REQUEST_METHOD: `POST` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:17:06.053550 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e3e4dde-27a5-4016-8a6e-20164bfd430c None None] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:17:06.053974 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e3e4dde-27a5-4016-8a6e-20164bfd430c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:17:06.074122 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.auth.core [None req-3e3e4dde-27a5-4016-8a6e-20164bfd430c None None] MFA Rules not processed for user `891005fa5ff74041b75747d16cc66798`. Rule list: `[]` (Enabled: `True`). {{(pid=80378) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 02 10:17:06.075776 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.fernet_utils [None req-3e3e4dde-27a5-4016-8a6e-20164bfd430c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80378) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 10:17:06.139757 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 969/1941] 10.222.0.82 () {58 vars in 958 bytes} [Tue Aug 2 10:17:06 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 02 10:17:06.149354 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d891d94-45d2-417d-af25-0471610c87fd admin admin] Authenticating user token {{(pid=80377) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:17:06.151560 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.fernet_utils [None req-9d891d94-45d2-417d-af25-0471610c87fd admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80377) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 10:17:06.185559 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG dbcounter [-] [80377] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80377) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 10:17:06.224835 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d891d94-45d2-417d-af25-0471610c87fd admin admin] Validating token access rules against request {{(pid=80377) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:17:06.228468 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d351132-31c6-48ea-8617-b05aad94895a service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '33b87170f0634b67bce60750847b5cbe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f1b38ebb804c41c69772c998aae72c41', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80377) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:17:06.229101 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d351132-31c6-48ea-8617-b05aad94895a service neutron] REQUEST_METHOD: `GET` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:17:06.229377 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d351132-31c6-48ea-8617-b05aad94895a service neutron] SCRIPT_NAME: `/identity` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:17:06.229646 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d351132-31c6-48ea-8617-b05aad94895a service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:17:06.240825 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d351132-31c6-48ea-8617-b05aad94895a service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:17:06.242497 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d351132-31c6-48ea-8617-b05aad94895a service neutron] RBAC: Authorization granted {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:17:06.253245 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: [pid: 80377|app: 0|req: 973/1942] 10.222.0.82 () {60 vars in 1365 bytes} [Tue Aug 2 10:17:06 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 104 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 02 10:17:06.557544 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e3e4dde-27a5-4016-8a6e-20164bfd430c None None] Authenticating user token {{(pid=80378) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:17:06.560221 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.fernet_utils [None req-3e3e4dde-27a5-4016-8a6e-20164bfd430c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80378) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 10:17:06.580817 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e3e4dde-27a5-4016-8a6e-20164bfd430c None None] Validating token access rules against request {{(pid=80378) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:17:06.593333 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68e5e1d9-d94d-4772-8ea1-a4ec5a11a6d6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '891005fa5ff74041b75747d16cc66798', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7047912c5acd4adeba5d87c3bf895e39', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80378) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:17:06.594045 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68e5e1d9-d94d-4772-8ea1-a4ec5a11a6d6 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:17:06.594333 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68e5e1d9-d94d-4772-8ea1-a4ec5a11a6d6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:17:06.594609 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68e5e1d9-d94d-4772-8ea1-a4ec5a11a6d6 admin admin] PATH_INFO: `/v3/projects/094bdaea97554859bfb556a99a1e92f6` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:17:06.595848 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68e5e1d9-d94d-4772-8ea1-a4ec5a11a6d6 admin admin] RBAC: Authorizing `identity:delete_project(project_id=094bdaea97554859bfb556a99a1e92f6)` {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:17:06.597741 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68e5e1d9-d94d-4772-8ea1-a4ec5a11a6d6 admin admin] RBAC: Authorization granted {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:17:06.649977 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.notifications [None req-68e5e1d9-d94d-4772-8ea1-a4ec5a11a6d6 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '094bdaea97554859bfb556a99a1e92f6', 'request_id': 'req-68e5e1d9-d94d-4772-8ea1-a4ec5a11a6d6'} {{(pid=80378) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 10:17:06.656834 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.notifications [None req-68e5e1d9-d94d-4772-8ea1-a4ec5a11a6d6 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '094bdaea97554859bfb556a99a1e92f6', 'request_id': 'req-68e5e1d9-d94d-4772-8ea1-a4ec5a11a6d6'} {{(pid=80378) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 10:17:06.662396 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.notifications [None req-68e5e1d9-d94d-4772-8ea1-a4ec5a11a6d6 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '094bdaea97554859bfb556a99a1e92f6', 'request_id': 'req-68e5e1d9-d94d-4772-8ea1-a4ec5a11a6d6'} {{(pid=80378) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 10:17:06.675897 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.notifications [None req-68e5e1d9-d94d-4772-8ea1-a4ec5a11a6d6 admin admin] The token cache is being invalidate because project 094bdaea97554859bfb556a99a1e92f6 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80378) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 02 10:17:06.676170 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.notifications [None req-68e5e1d9-d94d-4772-8ea1-a4ec5a11a6d6 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80378) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 10:17:06.676861 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.notifications [None req-68e5e1d9-d94d-4772-8ea1-a4ec5a11a6d6 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80378) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 10:17:06.678712 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 970/1943] 10.222.0.82 () {58 vars in 1233 bytes} [Tue Aug 2 10:17:06 2022] DELETE /identity/v3/projects/094bdaea97554859bfb556a99a1e92f6 => generated 0 bytes in 122 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 02 10:17:06.682828 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d351132-31c6-48ea-8617-b05aad94895a service neutron] Authenticating user token {{(pid=80377) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:17:06.693085 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d351132-31c6-48ea-8617-b05aad94895a service neutron] Validating token access rules against request {{(pid=80377) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:17:06.752125 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4b51c2a-7a6f-465e-8e25-3606aae3b6e2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '891005fa5ff74041b75747d16cc66798', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7047912c5acd4adeba5d87c3bf895e39', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80377) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:17:06.752705 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4b51c2a-7a6f-465e-8e25-3606aae3b6e2 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:17:06.752988 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4b51c2a-7a6f-465e-8e25-3606aae3b6e2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:17:06.753266 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4b51c2a-7a6f-465e-8e25-3606aae3b6e2 admin admin] PATH_INFO: `/v3/users/3604c4b85615466b984c1fb93bf4e81a` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:17:06.754475 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4b51c2a-7a6f-465e-8e25-3606aae3b6e2 admin admin] RBAC: Authorizing `identity:delete_user(user_id=3604c4b85615466b984c1fb93bf4e81a)` {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:17:06.756406 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4b51c2a-7a6f-465e-8e25-3606aae3b6e2 admin admin] RBAC: Authorization granted {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:17:06.802140 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.notifications [None req-f4b51c2a-7a6f-465e-8e25-3606aae3b6e2 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '3604c4b85615466b984c1fb93bf4e81a'} {{(pid=80377) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 10:17:06.809922 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.notifications [None req-f4b51c2a-7a6f-465e-8e25-3606aae3b6e2 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '3604c4b85615466b984c1fb93bf4e81a'} {{(pid=80377) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 10:17:06.816830 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.notifications [None req-f4b51c2a-7a6f-465e-8e25-3606aae3b6e2 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '3604c4b85615466b984c1fb93bf4e81a'} {{(pid=80377) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 10:17:06.817471 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.notifications [None req-f4b51c2a-7a6f-465e-8e25-3606aae3b6e2 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '3604c4b85615466b984c1fb93bf4e81a'} {{(pid=80377) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 10:17:06.829847 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.notifications [None req-f4b51c2a-7a6f-465e-8e25-3606aae3b6e2 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '3604c4b85615466b984c1fb93bf4e81a'} {{(pid=80377) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 10:17:06.835866 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: [pid: 80377|app: 0|req: 974/1944] 10.222.0.82 () {58 vars in 1224 bytes} [Tue Aug 2 10:17:06 2022] DELETE /identity/v3/users/3604c4b85615466b984c1fb93bf4e81a => generated 0 bytes in 154 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 02 10:17:07.125283 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68e5e1d9-d94d-4772-8ea1-a4ec5a11a6d6 admin admin] Authenticating user token {{(pid=80378) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:17:07.127756 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.fernet_utils [None req-68e5e1d9-d94d-4772-8ea1-a4ec5a11a6d6 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80378) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 10:17:07.143808 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68e5e1d9-d94d-4772-8ea1-a4ec5a11a6d6 admin admin] Validating token access rules against request {{(pid=80378) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:17:07.196505 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG dbcounter [-] [80378] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80378) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 10:17:07.211222 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ccdce33a-757b-4307-b6b8-36e38ea66950 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '891005fa5ff74041b75747d16cc66798', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7047912c5acd4adeba5d87c3bf895e39', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80378) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:17:07.212155 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccdce33a-757b-4307-b6b8-36e38ea66950 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:17:07.212437 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccdce33a-757b-4307-b6b8-36e38ea66950 admin admin] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:17:07.212713 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccdce33a-757b-4307-b6b8-36e38ea66950 admin admin] PATH_INFO: `/v3/projects/a9c7a9cc95944c7aa3b94b4d7492a77c` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:17:07.213931 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ccdce33a-757b-4307-b6b8-36e38ea66950 admin admin] RBAC: Authorizing `identity:delete_project(project_id=a9c7a9cc95944c7aa3b94b4d7492a77c)` {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:17:07.215851 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ccdce33a-757b-4307-b6b8-36e38ea66950 admin admin] RBAC: Authorization granted {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:17:07.261355 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.notifications [None req-ccdce33a-757b-4307-b6b8-36e38ea66950 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'a9c7a9cc95944c7aa3b94b4d7492a77c', 'request_id': 'req-ccdce33a-757b-4307-b6b8-36e38ea66950'} {{(pid=80378) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 10:17:07.270931 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.notifications [None req-ccdce33a-757b-4307-b6b8-36e38ea66950 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'a9c7a9cc95944c7aa3b94b4d7492a77c', 'request_id': 'req-ccdce33a-757b-4307-b6b8-36e38ea66950'} {{(pid=80378) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 10:17:07.277299 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.notifications [None req-ccdce33a-757b-4307-b6b8-36e38ea66950 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'a9c7a9cc95944c7aa3b94b4d7492a77c', 'request_id': 'req-ccdce33a-757b-4307-b6b8-36e38ea66950'} {{(pid=80378) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 10:17:07.290218 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.notifications [None req-ccdce33a-757b-4307-b6b8-36e38ea66950 admin admin] The token cache is being invalidate because project a9c7a9cc95944c7aa3b94b4d7492a77c was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80378) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 02 10:17:07.290510 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.notifications [None req-ccdce33a-757b-4307-b6b8-36e38ea66950 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80378) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 10:17:07.291172 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.notifications [None req-ccdce33a-757b-4307-b6b8-36e38ea66950 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80378) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 10:17:07.292668 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 971/1945] 10.222.0.82 () {58 vars in 1233 bytes} [Tue Aug 2 10:17:07 2022] DELETE /identity/v3/projects/a9c7a9cc95944c7aa3b94b4d7492a77c => generated 0 bytes in 169 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 02 10:17:07.297599 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26934001-c45d-4f37-b14d-1128bcdda8bb None None] REQUEST_METHOD: `POST` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:17:07.297862 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26934001-c45d-4f37-b14d-1128bcdda8bb None None] SCRIPT_NAME: `/identity` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:17:07.298143 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26934001-c45d-4f37-b14d-1128bcdda8bb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:17:07.318623 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.auth.core [None req-26934001-c45d-4f37-b14d-1128bcdda8bb None None] MFA Rules not processed for user `891005fa5ff74041b75747d16cc66798`. Rule list: `[]` (Enabled: `True`). {{(pid=80377) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 02 10:17:07.319829 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.fernet_utils [None req-26934001-c45d-4f37-b14d-1128bcdda8bb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80377) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 10:17:07.362929 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG dbcounter [-] [80377] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80377) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 10:17:07.389076 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: [pid: 80377|app: 0|req: 975/1946] 10.222.0.82 () {58 vars in 958 bytes} [Tue Aug 2 10:17:07 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 92 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 02 10:17:07.398152 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db981d17-f585-45a3-8f69-b46141b2121b None None] REQUEST_METHOD: `POST` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:17:07.398534 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db981d17-f585-45a3-8f69-b46141b2121b None None] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:17:07.398929 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db981d17-f585-45a3-8f69-b46141b2121b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:17:07.424317 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.auth.core [None req-db981d17-f585-45a3-8f69-b46141b2121b None None] MFA Rules not processed for user `891005fa5ff74041b75747d16cc66798`. Rule list: `[]` (Enabled: `True`). {{(pid=80378) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 02 10:17:07.425501 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.fernet_utils [None req-db981d17-f585-45a3-8f69-b46141b2121b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80378) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 10:17:07.434294 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 972/1947] 10.222.0.82 () {58 vars in 958 bytes} [Tue Aug 2 10:17:07 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 38 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 02 10:17:07.439161 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26934001-c45d-4f37-b14d-1128bcdda8bb None None] Authenticating user token {{(pid=80377) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:17:07.449317 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26934001-c45d-4f37-b14d-1128bcdda8bb None None] Validating token access rules against request {{(pid=80377) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:17:07.454592 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-598e8bd7-29e7-4513-975e-10c1836696d5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '891005fa5ff74041b75747d16cc66798', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7047912c5acd4adeba5d87c3bf895e39', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80377) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:17:07.455223 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-598e8bd7-29e7-4513-975e-10c1836696d5 admin admin] REQUEST_METHOD: `GET` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:17:07.455541 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-598e8bd7-29e7-4513-975e-10c1836696d5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:17:07.455821 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-598e8bd7-29e7-4513-975e-10c1836696d5 admin admin] PATH_INFO: `/v3/domains` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:17:07.456530 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-598e8bd7-29e7-4513-975e-10c1836696d5 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80377) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 02 10:17:07.456842 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-598e8bd7-29e7-4513-975e-10c1836696d5 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:17:07.458528 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-598e8bd7-29e7-4513-975e-10c1836696d5 admin admin] RBAC: Authorization granted {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:17:07.471953 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: [pid: 80377|app: 0|req: 976/1948] 10.222.0.82 () {58 vars in 1153 bytes} [Tue Aug 2 10:17:07 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 02 10:17:07.476198 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db981d17-f585-45a3-8f69-b46141b2121b None None] Authenticating user token {{(pid=80378) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:17:07.485920 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db981d17-f585-45a3-8f69-b46141b2121b None None] Validating token access rules against request {{(pid=80378) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:17:07.490688 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fc614a3-35c5-4ca0-b27e-73ad2325c1e5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '891005fa5ff74041b75747d16cc66798', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7047912c5acd4adeba5d87c3bf895e39', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80378) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:17:07.491345 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fc614a3-35c5-4ca0-b27e-73ad2325c1e5 admin admin] REQUEST_METHOD: `POST` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:17:07.491628 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fc614a3-35c5-4ca0-b27e-73ad2325c1e5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:17:07.491904 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fc614a3-35c5-4ca0-b27e-73ad2325c1e5 admin admin] PATH_INFO: `/v3/projects` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:17:07.492747 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9fc614a3-35c5-4ca0-b27e-73ad2325c1e5 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:17:07.494782 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9fc614a3-35c5-4ca0-b27e-73ad2325c1e5 admin admin] RBAC: Authorization granted {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:17:07.527643 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 973/1949] 10.222.0.82 () {60 vars in 1153 bytes} [Tue Aug 2 10:17:07 2022] POST /identity/v3/projects => generated 401 bytes in 49 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 02 10:17:07.528800 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-598e8bd7-29e7-4513-975e-10c1836696d5 admin admin] Authenticating user token {{(pid=80377) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:17:07.539315 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-598e8bd7-29e7-4513-975e-10c1836696d5 admin admin] Validating token access rules against request {{(pid=80377) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:17:07.616632 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d3e67a7-c87c-4d2e-bcf6-8ba8c82f7f09 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '891005fa5ff74041b75747d16cc66798', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7047912c5acd4adeba5d87c3bf895e39', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80377) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:17:07.617677 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d3e67a7-c87c-4d2e-bcf6-8ba8c82f7f09 admin admin] REQUEST_METHOD: `POST` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:17:07.618218 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d3e67a7-c87c-4d2e-bcf6-8ba8c82f7f09 admin admin] SCRIPT_NAME: `/identity` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:17:07.618630 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d3e67a7-c87c-4d2e-bcf6-8ba8c82f7f09 admin admin] PATH_INFO: `/v3/users` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:17:07.619793 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d3e67a7-c87c-4d2e-bcf6-8ba8c82f7f09 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:17:07.623873 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d3e67a7-c87c-4d2e-bcf6-8ba8c82f7f09 admin admin] RBAC: Authorization granted {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:17:07.651380 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: [pid: 80377|app: 0|req: 977/1950] 10.222.0.82 () {60 vars in 1144 bytes} [Tue Aug 2 10:17:07 2022] POST /identity/v3/users => generated 301 bytes in 123 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 02 10:17:07.655486 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fc614a3-35c5-4ca0-b27e-73ad2325c1e5 admin admin] Authenticating user token {{(pid=80378) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:17:07.666793 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fc614a3-35c5-4ca0-b27e-73ad2325c1e5 admin admin] Validating token access rules against request {{(pid=80378) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:17:07.672300 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae4160e0-0dee-4cab-983d-059f527a8916 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '891005fa5ff74041b75747d16cc66798', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7047912c5acd4adeba5d87c3bf895e39', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80378) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:17:07.672964 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae4160e0-0dee-4cab-983d-059f527a8916 admin admin] REQUEST_METHOD: `GET` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:17:07.673253 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae4160e0-0dee-4cab-983d-059f527a8916 admin admin] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:17:07.673534 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae4160e0-0dee-4cab-983d-059f527a8916 admin admin] PATH_INFO: `/v3/roles` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:17:07.674286 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae4160e0-0dee-4cab-983d-059f527a8916 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:17:07.675953 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae4160e0-0dee-4cab-983d-059f527a8916 admin admin] RBAC: Authorization granted {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:17:07.688059 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 974/1951] 10.222.0.82 () {58 vars in 1122 bytes} [Tue Aug 2 10:17:07 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 02 10:17:07.692583 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d3e67a7-c87c-4d2e-bcf6-8ba8c82f7f09 admin admin] Authenticating user token {{(pid=80377) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:17:07.705121 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d3e67a7-c87c-4d2e-bcf6-8ba8c82f7f09 admin admin] Validating token access rules against request {{(pid=80377) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:17:07.708244 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b845f0f1-89b7-4302-b0ff-f41f4824a9ce admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '891005fa5ff74041b75747d16cc66798', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7047912c5acd4adeba5d87c3bf895e39', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80377) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:17:07.708900 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b845f0f1-89b7-4302-b0ff-f41f4824a9ce admin admin] REQUEST_METHOD: `GET` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:17:07.709187 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b845f0f1-89b7-4302-b0ff-f41f4824a9ce admin admin] SCRIPT_NAME: `/identity` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:17:07.709467 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b845f0f1-89b7-4302-b0ff-f41f4824a9ce admin admin] PATH_INFO: `/v3/roles` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:17:07.710216 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b845f0f1-89b7-4302-b0ff-f41f4824a9ce admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:17:07.711829 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b845f0f1-89b7-4302-b0ff-f41f4824a9ce admin admin] RBAC: Authorization granted {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:17:07.723653 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: [pid: 80377|app: 0|req: 978/1952] 10.222.0.82 () {58 vars in 1122 bytes} [Tue Aug 2 10:17:07 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 02 10:17:07.727942 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae4160e0-0dee-4cab-983d-059f527a8916 admin admin] Authenticating user token {{(pid=80378) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:17:07.738669 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae4160e0-0dee-4cab-983d-059f527a8916 admin admin] Validating token access rules against request {{(pid=80378) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:17:07.743829 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-681d8e7f-1252-49a3-b0f7-48255d3fd0ee admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '891005fa5ff74041b75747d16cc66798', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7047912c5acd4adeba5d87c3bf895e39', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80378) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:17:07.744520 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-681d8e7f-1252-49a3-b0f7-48255d3fd0ee admin admin] REQUEST_METHOD: `PUT` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:17:07.744795 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-681d8e7f-1252-49a3-b0f7-48255d3fd0ee admin admin] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:17:07.745070 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-681d8e7f-1252-49a3-b0f7-48255d3fd0ee admin admin] PATH_INFO: `/v3/projects/6eb831630f7c4594bbfa41ef86ae84df/users/8a976d24cbfd407eb9da97e21e37a733/roles/e1c04b19fc4543b8b54f58bbcffbc598` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:17:07.771125 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-681d8e7f-1252-49a3-b0f7-48255d3fd0ee admin admin] RBAC: Authorizing `identity:create_grant(project_id=6eb831630f7c4594bbfa41ef86ae84df, user_id=8a976d24cbfd407eb9da97e21e37a733, role_id=e1c04b19fc4543b8b54f58bbcffbc598)` {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:17:07.771125 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-681d8e7f-1252-49a3-b0f7-48255d3fd0ee admin admin] RBAC: Authorization granted {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:17:07.786097 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 975/1953] 10.222.0.82 () {60 vars in 1483 bytes} [Tue Aug 2 10:17:07 2022] PUT /identity/v3/projects/6eb831630f7c4594bbfa41ef86ae84df/users/8a976d24cbfd407eb9da97e21e37a733/roles/e1c04b19fc4543b8b54f58bbcffbc598 => generated 0 bytes in 59 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 02 10:17:07.796453 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d073dd5c-134d-402c-b9d5-7454f14c61df None None] REQUEST_METHOD: `POST` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:17:07.796745 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d073dd5c-134d-402c-b9d5-7454f14c61df None None] SCRIPT_NAME: `/identity` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:17:07.797027 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d073dd5c-134d-402c-b9d5-7454f14c61df None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:17:07.827233 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.auth.core [None req-d073dd5c-134d-402c-b9d5-7454f14c61df None None] MFA Rules not processed for user `8a976d24cbfd407eb9da97e21e37a733`. Rule list: `[]` (Enabled: `True`). {{(pid=80377) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 02 10:17:07.828381 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.fernet_utils [None req-d073dd5c-134d-402c-b9d5-7454f14c61df None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80377) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 10:17:07.905919 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: [pid: 80377|app: 0|req: 979/1954] 10.222.0.82 () {58 vars in 958 bytes} [Tue Aug 2 10:17:07 2022] POST /identity/v3/auth/tokens => generated 3278 bytes in 111 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 02 10:17:07.910379 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-681d8e7f-1252-49a3-b0f7-48255d3fd0ee admin admin] Authenticating user token {{(pid=80378) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:17:07.973839 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-681d8e7f-1252-49a3-b0f7-48255d3fd0ee admin admin] Validating token access rules against request {{(pid=80378) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:17:07.978841 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af048f6b-7492-479d-ae40-2ac49226e066 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '891005fa5ff74041b75747d16cc66798', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7047912c5acd4adeba5d87c3bf895e39', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80378) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:17:07.979471 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af048f6b-7492-479d-ae40-2ac49226e066 admin admin] REQUEST_METHOD: `POST` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:17:07.979755 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af048f6b-7492-479d-ae40-2ac49226e066 admin admin] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:17:07.980030 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af048f6b-7492-479d-ae40-2ac49226e066 admin admin] PATH_INFO: `/v3/projects` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:17:07.980720 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af048f6b-7492-479d-ae40-2ac49226e066 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:17:07.982746 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af048f6b-7492-479d-ae40-2ac49226e066 admin admin] RBAC: Authorization granted {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:17:08.005899 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG dbcounter [-] [80378] Writing DB stats keystone:SELECT=89,keystone:DELETE=7,keystone:INSERT=4 {{(pid=80378) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 10:17:08.014664 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 976/1955] 10.222.0.82 () {60 vars in 1153 bytes} [Tue Aug 2 10:17:07 2022] POST /identity/v3/projects => generated 401 bytes in 104 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 02 10:17:08.018125 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d073dd5c-134d-402c-b9d5-7454f14c61df None None] Authenticating user token {{(pid=80377) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:17:08.029713 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d073dd5c-134d-402c-b9d5-7454f14c61df None None] Validating token access rules against request {{(pid=80377) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:17:08.068731 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG dbcounter [-] [80377] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80377) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 10:17:08.088526 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a17827f0-d12d-493e-9239-038e88427812 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '891005fa5ff74041b75747d16cc66798', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7047912c5acd4adeba5d87c3bf895e39', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80377) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:17:08.090311 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a17827f0-d12d-493e-9239-038e88427812 admin admin] REQUEST_METHOD: `POST` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:17:08.090311 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a17827f0-d12d-493e-9239-038e88427812 admin admin] SCRIPT_NAME: `/identity` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:17:08.090311 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a17827f0-d12d-493e-9239-038e88427812 admin admin] PATH_INFO: `/v3/users` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:17:08.090311 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a17827f0-d12d-493e-9239-038e88427812 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:17:08.092414 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a17827f0-d12d-493e-9239-038e88427812 admin admin] RBAC: Authorization granted {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:17:08.114031 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: [pid: 80377|app: 0|req: 980/1956] 10.222.0.82 () {60 vars in 1144 bytes} [Tue Aug 2 10:17:08 2022] POST /identity/v3/users => generated 307 bytes in 96 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 02 10:17:08.118617 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af048f6b-7492-479d-ae40-2ac49226e066 admin admin] Authenticating user token {{(pid=80378) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:17:08.129832 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af048f6b-7492-479d-ae40-2ac49226e066 admin admin] Validating token access rules against request {{(pid=80378) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:17:08.134673 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18abaf7e-5dc9-4356-afb4-e82d48eeee28 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '891005fa5ff74041b75747d16cc66798', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7047912c5acd4adeba5d87c3bf895e39', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80378) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:17:08.135299 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18abaf7e-5dc9-4356-afb4-e82d48eeee28 admin admin] REQUEST_METHOD: `GET` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:17:08.135584 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18abaf7e-5dc9-4356-afb4-e82d48eeee28 admin admin] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:17:08.135891 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18abaf7e-5dc9-4356-afb4-e82d48eeee28 admin admin] PATH_INFO: `/v3/roles` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:17:08.136592 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18abaf7e-5dc9-4356-afb4-e82d48eeee28 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:17:08.139075 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18abaf7e-5dc9-4356-afb4-e82d48eeee28 admin admin] RBAC: Authorization granted {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:17:08.151534 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 977/1957] 10.222.0.82 () {58 vars in 1122 bytes} [Tue Aug 2 10:17:08 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 02 10:17:08.156198 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a17827f0-d12d-493e-9239-038e88427812 admin admin] Authenticating user token {{(pid=80377) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:17:08.168986 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a17827f0-d12d-493e-9239-038e88427812 admin admin] Validating token access rules against request {{(pid=80377) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:17:08.173676 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-637203e0-203e-4e32-99e4-de6718b5361d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '891005fa5ff74041b75747d16cc66798', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7047912c5acd4adeba5d87c3bf895e39', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80377) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:17:08.174397 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-637203e0-203e-4e32-99e4-de6718b5361d admin admin] REQUEST_METHOD: `PUT` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:17:08.174677 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-637203e0-203e-4e32-99e4-de6718b5361d admin admin] SCRIPT_NAME: `/identity` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:17:08.174953 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-637203e0-203e-4e32-99e4-de6718b5361d admin admin] PATH_INFO: `/v3/projects/84e78936aca74b67b7fd11f28a67c947/users/4fa9d12062364181a53ac83f9f700a33/roles/237966442f05411191105bf336bc14f3` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:17:08.192834 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-637203e0-203e-4e32-99e4-de6718b5361d admin admin] RBAC: Authorizing `identity:create_grant(project_id=84e78936aca74b67b7fd11f28a67c947, user_id=4fa9d12062364181a53ac83f9f700a33, role_id=237966442f05411191105bf336bc14f3)` {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:17:08.195680 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-637203e0-203e-4e32-99e4-de6718b5361d admin admin] RBAC: Authorization granted {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:17:08.218849 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: [pid: 80377|app: 0|req: 981/1958] 10.222.0.82 () {60 vars in 1483 bytes} [Tue Aug 2 10:17:08 2022] PUT /identity/v3/projects/84e78936aca74b67b7fd11f28a67c947/users/4fa9d12062364181a53ac83f9f700a33/roles/237966442f05411191105bf336bc14f3 => generated 0 bytes in 63 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 02 10:17:08.222871 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18abaf7e-5dc9-4356-afb4-e82d48eeee28 admin admin] Authenticating user token {{(pid=80378) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:17:08.233486 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18abaf7e-5dc9-4356-afb4-e82d48eeee28 admin admin] Validating token access rules against request {{(pid=80378) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:17:08.290439 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3124eba-5576-4ee2-882f-04775609de15 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '891005fa5ff74041b75747d16cc66798', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7047912c5acd4adeba5d87c3bf895e39', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80378) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:17:08.291080 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3124eba-5576-4ee2-882f-04775609de15 admin admin] REQUEST_METHOD: `GET` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:17:08.291498 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3124eba-5576-4ee2-882f-04775609de15 admin admin] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:17:08.291805 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3124eba-5576-4ee2-882f-04775609de15 admin admin] PATH_INFO: `/v3/roles` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:17:08.292523 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3124eba-5576-4ee2-882f-04775609de15 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:17:08.294160 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3124eba-5576-4ee2-882f-04775609de15 admin admin] RBAC: Authorization granted {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:17:08.304495 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 978/1959] 10.222.0.82 () {58 vars in 1122 bytes} [Tue Aug 2 10:17:08 2022] GET /identity/v3/roles => generated 1420 bytes in 82 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 02 10:17:08.308955 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-637203e0-203e-4e32-99e4-de6718b5361d admin admin] Authenticating user token {{(pid=80377) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:17:08.322650 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-637203e0-203e-4e32-99e4-de6718b5361d admin admin] Validating token access rules against request {{(pid=80377) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:17:08.328235 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdaf9a20-c47a-45bc-8aa1-036a6d26e7f6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '891005fa5ff74041b75747d16cc66798', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7047912c5acd4adeba5d87c3bf895e39', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80377) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:17:08.328914 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdaf9a20-c47a-45bc-8aa1-036a6d26e7f6 admin admin] REQUEST_METHOD: `PUT` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:17:08.329199 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdaf9a20-c47a-45bc-8aa1-036a6d26e7f6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:17:08.329479 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdaf9a20-c47a-45bc-8aa1-036a6d26e7f6 admin admin] PATH_INFO: `/v3/projects/84e78936aca74b67b7fd11f28a67c947/users/4fa9d12062364181a53ac83f9f700a33/roles/237966442f05411191105bf336bc14f3` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:17:08.331465 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cdaf9a20-c47a-45bc-8aa1-036a6d26e7f6 admin admin] RBAC: Authorizing `identity:create_grant(project_id=84e78936aca74b67b7fd11f28a67c947, user_id=4fa9d12062364181a53ac83f9f700a33, role_id=237966442f05411191105bf336bc14f3)` {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:17:08.334130 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cdaf9a20-c47a-45bc-8aa1-036a6d26e7f6 admin admin] RBAC: Authorization granted {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:17:08.345566 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: [pid: 80377|app: 0|req: 982/1960] 10.222.0.82 () {60 vars in 1483 bytes} [Tue Aug 2 10:17:08 2022] PUT /identity/v3/projects/84e78936aca74b67b7fd11f28a67c947/users/4fa9d12062364181a53ac83f9f700a33/roles/237966442f05411191105bf336bc14f3 => generated 0 bytes in 37 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 02 10:17:08.357369 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b8e5bce-2738-480a-9936-69ef2e782466 None None] REQUEST_METHOD: `POST` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:17:08.357702 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b8e5bce-2738-480a-9936-69ef2e782466 None None] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:17:08.357979 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b8e5bce-2738-480a-9936-69ef2e782466 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:17:08.386260 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.auth.core [None req-1b8e5bce-2738-480a-9936-69ef2e782466 None None] MFA Rules not processed for user `4fa9d12062364181a53ac83f9f700a33`. Rule list: `[]` (Enabled: `True`). {{(pid=80378) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 02 10:17:08.387512 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.fernet_utils [None req-1b8e5bce-2738-480a-9936-69ef2e782466 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80378) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 10:17:08.461209 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 979/1961] 10.222.0.82 () {58 vars in 958 bytes} [Tue Aug 2 10:17:08 2022] POST /identity/v3/auth/tokens => generated 3345 bytes in 105 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 02 10:17:08.473194 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdaf9a20-c47a-45bc-8aa1-036a6d26e7f6 admin admin] Authenticating user token {{(pid=80377) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:17:08.474936 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.fernet_utils [None req-cdaf9a20-c47a-45bc-8aa1-036a6d26e7f6 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80377) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 10:17:08.536476 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdaf9a20-c47a-45bc-8aa1-036a6d26e7f6 admin admin] Validating token access rules against request {{(pid=80377) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:17:08.540205 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9379971-a7df-4056-a11d-2fa72334fd33 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1023de281ae74e9d8a6d6c312c1c97a7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f1b38ebb804c41c69772c998aae72c41', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80377) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:17:08.540828 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9379971-a7df-4056-a11d-2fa72334fd33 service cinder] REQUEST_METHOD: `GET` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:17:08.541113 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9379971-a7df-4056-a11d-2fa72334fd33 service cinder] SCRIPT_NAME: `/identity` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:17:08.541391 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9379971-a7df-4056-a11d-2fa72334fd33 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:17:08.550411 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9379971-a7df-4056-a11d-2fa72334fd33 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:17:08.551847 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9379971-a7df-4056-a11d-2fa72334fd33 service cinder] RBAC: Authorization granted {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:17:08.561522 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: [pid: 80377|app: 0|req: 983/1962] 10.222.0.82 () {60 vars in 1365 bytes} [Tue Aug 2 10:17:08 2022] GET /identity/v3/auth/tokens => generated 3345 bytes in 89 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 02 10:17:08.643098 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b8e5bce-2738-480a-9936-69ef2e782466 None None] Authenticating user token {{(pid=80378) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:17:08.652764 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b8e5bce-2738-480a-9936-69ef2e782466 None None] Validating token access rules against request {{(pid=80378) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:17:08.657178 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8248cac-9dd1-4e35-aa92-da837fcb016f service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1023de281ae74e9d8a6d6c312c1c97a7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f1b38ebb804c41c69772c998aae72c41', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80378) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:17:08.657943 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8248cac-9dd1-4e35-aa92-da837fcb016f service cinder] REQUEST_METHOD: `GET` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:17:08.658291 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8248cac-9dd1-4e35-aa92-da837fcb016f service cinder] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:17:08.658638 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8248cac-9dd1-4e35-aa92-da837fcb016f service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:17:08.719627 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8248cac-9dd1-4e35-aa92-da837fcb016f service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:17:08.721688 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8248cac-9dd1-4e35-aa92-da837fcb016f service cinder] RBAC: Authorization granted {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:17:08.730785 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 980/1963] 10.222.0.82 () {60 vars in 1365 bytes} [Tue Aug 2 10:17:08 2022] GET /identity/v3/auth/tokens => generated 3278 bytes in 88 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 02 10:17:09.080795 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9379971-a7df-4056-a11d-2fa72334fd33 service cinder] Authenticating user token {{(pid=80377) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:17:09.082374 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.fernet_utils [None req-e9379971-a7df-4056-a11d-2fa72334fd33 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80377) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 10:17:09.172503 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9379971-a7df-4056-a11d-2fa72334fd33 service cinder] Validating token access rules against request {{(pid=80377) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:17:09.186102 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-748e7f61-baf7-4fa2-9da3-40fbf50f0f3d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '891005fa5ff74041b75747d16cc66798', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7047912c5acd4adeba5d87c3bf895e39', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80377) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:17:09.187162 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-748e7f61-baf7-4fa2-9da3-40fbf50f0f3d admin admin] REQUEST_METHOD: `DELETE` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:17:09.187531 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-748e7f61-baf7-4fa2-9da3-40fbf50f0f3d admin admin] SCRIPT_NAME: `/identity` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:17:09.187806 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-748e7f61-baf7-4fa2-9da3-40fbf50f0f3d admin admin] PATH_INFO: `/v3/users/7e8d3d65b49440fab43627ca92c509ca` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:17:09.189715 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-748e7f61-baf7-4fa2-9da3-40fbf50f0f3d admin admin] RBAC: Authorizing `identity:delete_user(user_id=7e8d3d65b49440fab43627ca92c509ca)` {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:17:09.191733 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-748e7f61-baf7-4fa2-9da3-40fbf50f0f3d admin admin] RBAC: Authorization granted {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:17:09.259185 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG dbcounter [-] [80377] Writing DB stats keystone:SELECT=90,keystone:INSERT=5,keystone:DELETE=5 {{(pid=80377) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 10:17:09.281416 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.notifications [None req-748e7f61-baf7-4fa2-9da3-40fbf50f0f3d admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '7e8d3d65b49440fab43627ca92c509ca'} {{(pid=80377) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 10:17:09.299400 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.notifications [None req-748e7f61-baf7-4fa2-9da3-40fbf50f0f3d admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '7e8d3d65b49440fab43627ca92c509ca'} {{(pid=80377) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 10:17:09.310849 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.notifications [None req-748e7f61-baf7-4fa2-9da3-40fbf50f0f3d admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '7e8d3d65b49440fab43627ca92c509ca'} {{(pid=80377) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 10:17:09.313892 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.notifications [None req-748e7f61-baf7-4fa2-9da3-40fbf50f0f3d admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '7e8d3d65b49440fab43627ca92c509ca'} {{(pid=80377) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 10:17:09.328284 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.notifications [None req-748e7f61-baf7-4fa2-9da3-40fbf50f0f3d admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '7e8d3d65b49440fab43627ca92c509ca'} {{(pid=80377) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 10:17:09.335903 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: [pid: 80377|app: 0|req: 984/1964] 10.222.0.82 () {58 vars in 1224 bytes} [Tue Aug 2 10:17:09 2022] DELETE /identity/v3/users/7e8d3d65b49440fab43627ca92c509ca => generated 0 bytes in 256 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 02 10:17:09.340535 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52ef870e-e9a0-4699-ba14-1f5fe6a1ae1b None None] REQUEST_METHOD: `POST` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:17:09.341235 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52ef870e-e9a0-4699-ba14-1f5fe6a1ae1b None None] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:17:09.341476 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52ef870e-e9a0-4699-ba14-1f5fe6a1ae1b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:17:09.362193 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.auth.core [None req-52ef870e-e9a0-4699-ba14-1f5fe6a1ae1b None None] MFA Rules not processed for user `891005fa5ff74041b75747d16cc66798`. Rule list: `[]` (Enabled: `True`). {{(pid=80378) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 02 10:17:09.363311 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.fernet_utils [None req-52ef870e-e9a0-4699-ba14-1f5fe6a1ae1b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80378) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 10:17:09.367868 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG dbcounter [-] [80378] Writing DB stats keystone:SELECT=100 {{(pid=80378) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 10:17:09.409908 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-748e7f61-baf7-4fa2-9da3-40fbf50f0f3d admin admin] Authenticating user token {{(pid=80377) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:17:09.411547 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.fernet_utils [None req-748e7f61-baf7-4fa2-9da3-40fbf50f0f3d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80377) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 10:17:09.439074 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 981/1965] 10.222.0.82 () {58 vars in 958 bytes} [Tue Aug 2 10:17:09 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 02 10:17:09.449290 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52ef870e-e9a0-4699-ba14-1f5fe6a1ae1b None None] Authenticating user token {{(pid=80378) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:17:09.450724 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.fernet_utils [None req-52ef870e-e9a0-4699-ba14-1f5fe6a1ae1b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80378) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 10:17:09.484105 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-748e7f61-baf7-4fa2-9da3-40fbf50f0f3d admin admin] Validating token access rules against request {{(pid=80377) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:17:09.488849 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1141971-dac8-4b7e-b805-e06933ba7933 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '891005fa5ff74041b75747d16cc66798', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7047912c5acd4adeba5d87c3bf895e39', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80377) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:17:09.489519 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1141971-dac8-4b7e-b805-e06933ba7933 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:17:09.489831 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1141971-dac8-4b7e-b805-e06933ba7933 admin admin] SCRIPT_NAME: `/identity` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:17:09.490119 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1141971-dac8-4b7e-b805-e06933ba7933 admin admin] PATH_INFO: `/v3/users/eb2783502f874dfaa315ea300113932c` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:17:09.491338 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1141971-dac8-4b7e-b805-e06933ba7933 admin admin] RBAC: Authorizing `identity:delete_user(user_id=eb2783502f874dfaa315ea300113932c)` {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:17:09.493323 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1141971-dac8-4b7e-b805-e06933ba7933 admin admin] RBAC: Authorization granted {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:17:09.516100 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52ef870e-e9a0-4699-ba14-1f5fe6a1ae1b None None] Validating token access rules against request {{(pid=80378) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:17:09.520706 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94befcd3-4e90-48eb-9b67-fe79e1d18041 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '33b87170f0634b67bce60750847b5cbe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f1b38ebb804c41c69772c998aae72c41', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80378) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:17:09.521625 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94befcd3-4e90-48eb-9b67-fe79e1d18041 service neutron] REQUEST_METHOD: `GET` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:17:09.522002 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94befcd3-4e90-48eb-9b67-fe79e1d18041 service neutron] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:17:09.522284 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94befcd3-4e90-48eb-9b67-fe79e1d18041 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:17:09.533214 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94befcd3-4e90-48eb-9b67-fe79e1d18041 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:17:09.534726 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94befcd3-4e90-48eb-9b67-fe79e1d18041 service neutron] RBAC: Authorization granted {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:17:09.542513 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.notifications [None req-b1141971-dac8-4b7e-b805-e06933ba7933 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'eb2783502f874dfaa315ea300113932c'} {{(pid=80377) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 10:17:09.546127 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 982/1966] 10.222.0.82 () {60 vars in 1365 bytes} [Tue Aug 2 10:17:09 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 97 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 02 10:17:09.551689 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.notifications [None req-b1141971-dac8-4b7e-b805-e06933ba7933 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'eb2783502f874dfaa315ea300113932c'} {{(pid=80377) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 10:17:09.560157 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.notifications [None req-b1141971-dac8-4b7e-b805-e06933ba7933 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'eb2783502f874dfaa315ea300113932c'} {{(pid=80377) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 10:17:09.561043 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.notifications [None req-b1141971-dac8-4b7e-b805-e06933ba7933 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'eb2783502f874dfaa315ea300113932c'} {{(pid=80377) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 10:17:09.574629 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.notifications [None req-b1141971-dac8-4b7e-b805-e06933ba7933 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'eb2783502f874dfaa315ea300113932c'} {{(pid=80377) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 10:17:09.581126 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: [pid: 80377|app: 0|req: 985/1967] 10.222.0.82 () {58 vars in 1224 bytes} [Tue Aug 2 10:17:09 2022] DELETE /identity/v3/users/eb2783502f874dfaa315ea300113932c => generated 0 bytes in 172 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 02 10:17:09.586087 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a4586d1-198f-45cd-a1b3-596cac57a85d None None] REQUEST_METHOD: `POST` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:17:09.586381 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a4586d1-198f-45cd-a1b3-596cac57a85d None None] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:17:09.586664 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a4586d1-198f-45cd-a1b3-596cac57a85d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:17:09.612552 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.auth.core [None req-8a4586d1-198f-45cd-a1b3-596cac57a85d None None] MFA Rules not processed for user `891005fa5ff74041b75747d16cc66798`. Rule list: `[]` (Enabled: `True`). {{(pid=80378) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 02 10:17:09.614114 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.fernet_utils [None req-8a4586d1-198f-45cd-a1b3-596cac57a85d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80378) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 10:17:09.681541 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 983/1968] 10.222.0.82 () {58 vars in 958 bytes} [Tue Aug 2 10:17:09 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 02 10:17:09.692195 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1141971-dac8-4b7e-b805-e06933ba7933 admin admin] Authenticating user token {{(pid=80377) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:17:09.693831 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.fernet_utils [None req-b1141971-dac8-4b7e-b805-e06933ba7933 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80377) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 10:17:09.743181 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG dbcounter [-] [80377] Writing DB stats keystone:SELECT=83,keystone:DELETE=15,keystone:INSERT=2 {{(pid=80377) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 10:17:09.767676 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1141971-dac8-4b7e-b805-e06933ba7933 admin admin] Validating token access rules against request {{(pid=80377) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:17:09.771399 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05ca02f7-ac86-4b54-98f3-d9b66e9d1926 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '33b87170f0634b67bce60750847b5cbe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f1b38ebb804c41c69772c998aae72c41', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80377) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:17:09.772040 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05ca02f7-ac86-4b54-98f3-d9b66e9d1926 service neutron] REQUEST_METHOD: `GET` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:17:09.772323 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05ca02f7-ac86-4b54-98f3-d9b66e9d1926 service neutron] SCRIPT_NAME: `/identity` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:17:09.772600 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05ca02f7-ac86-4b54-98f3-d9b66e9d1926 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:17:09.783696 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05ca02f7-ac86-4b54-98f3-d9b66e9d1926 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:17:09.785165 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05ca02f7-ac86-4b54-98f3-d9b66e9d1926 service neutron] RBAC: Authorization granted {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:17:09.795194 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: [pid: 80377|app: 0|req: 986/1969] 10.222.0.82 () {60 vars in 1365 bytes} [Tue Aug 2 10:17:09 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 104 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 02 10:17:09.886851 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a4586d1-198f-45cd-a1b3-596cac57a85d None None] Authenticating user token {{(pid=80378) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:17:09.888927 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.fernet_utils [None req-8a4586d1-198f-45cd-a1b3-596cac57a85d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80378) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 10:17:09.902409 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a4586d1-198f-45cd-a1b3-596cac57a85d None None] Validating token access rules against request {{(pid=80378) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:17:09.908159 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-028c88b7-72d3-4319-9bc3-8dadafac7a28 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '891005fa5ff74041b75747d16cc66798', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7047912c5acd4adeba5d87c3bf895e39', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80378) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:17:09.908883 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-028c88b7-72d3-4319-9bc3-8dadafac7a28 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:17:09.909168 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-028c88b7-72d3-4319-9bc3-8dadafac7a28 admin admin] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:17:09.909451 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-028c88b7-72d3-4319-9bc3-8dadafac7a28 admin admin] PATH_INFO: `/v3/projects/b88e65e0895748b89c4156f4d00b4461` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:17:09.910763 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-028c88b7-72d3-4319-9bc3-8dadafac7a28 admin admin] RBAC: Authorizing `identity:delete_project(project_id=b88e65e0895748b89c4156f4d00b4461)` {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:17:09.912755 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-028c88b7-72d3-4319-9bc3-8dadafac7a28 admin admin] RBAC: Authorization granted {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:17:09.975010 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG dbcounter [-] [80378] Writing DB stats keystone:SELECT=96,keystone:DELETE=4 {{(pid=80378) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 10:17:09.977259 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.notifications [None req-028c88b7-72d3-4319-9bc3-8dadafac7a28 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'b88e65e0895748b89c4156f4d00b4461', 'request_id': 'req-028c88b7-72d3-4319-9bc3-8dadafac7a28'} {{(pid=80378) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 10:17:09.984273 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.notifications [None req-028c88b7-72d3-4319-9bc3-8dadafac7a28 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'b88e65e0895748b89c4156f4d00b4461', 'request_id': 'req-028c88b7-72d3-4319-9bc3-8dadafac7a28'} {{(pid=80378) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 10:17:09.990249 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.notifications [None req-028c88b7-72d3-4319-9bc3-8dadafac7a28 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'b88e65e0895748b89c4156f4d00b4461', 'request_id': 'req-028c88b7-72d3-4319-9bc3-8dadafac7a28'} {{(pid=80378) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 10:17:10.003939 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.notifications [None req-028c88b7-72d3-4319-9bc3-8dadafac7a28 admin admin] The token cache is being invalidate because project b88e65e0895748b89c4156f4d00b4461 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80378) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 02 10:17:10.004241 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.notifications [None req-028c88b7-72d3-4319-9bc3-8dadafac7a28 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80378) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 10:17:10.004899 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.notifications [None req-028c88b7-72d3-4319-9bc3-8dadafac7a28 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80378) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 10:17:10.009305 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 984/1970] 10.222.0.82 () {58 vars in 1233 bytes} [Tue Aug 2 10:17:09 2022] DELETE /identity/v3/projects/b88e65e0895748b89c4156f4d00b4461 => generated 0 bytes in 120 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 02 10:17:10.010651 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05ca02f7-ac86-4b54-98f3-d9b66e9d1926 service neutron] Authenticating user token {{(pid=80377) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:17:10.021678 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05ca02f7-ac86-4b54-98f3-d9b66e9d1926 service neutron] Validating token access rules against request {{(pid=80377) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:17:10.114354 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f89bbb39-964c-44fc-b98f-48eb0b0eb138 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '891005fa5ff74041b75747d16cc66798', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7047912c5acd4adeba5d87c3bf895e39', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80377) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:17:10.115031 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f89bbb39-964c-44fc-b98f-48eb0b0eb138 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:17:10.115330 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f89bbb39-964c-44fc-b98f-48eb0b0eb138 admin admin] SCRIPT_NAME: `/identity` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:17:10.115611 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f89bbb39-964c-44fc-b98f-48eb0b0eb138 admin admin] PATH_INFO: `/v3/users/edc86e1645d742d48a50e74e9060cd6e` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:17:10.116909 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f89bbb39-964c-44fc-b98f-48eb0b0eb138 admin admin] RBAC: Authorizing `identity:delete_user(user_id=edc86e1645d742d48a50e74e9060cd6e)` {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:17:10.119112 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f89bbb39-964c-44fc-b98f-48eb0b0eb138 admin admin] RBAC: Authorization granted {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:17:10.189376 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.notifications [None req-f89bbb39-964c-44fc-b98f-48eb0b0eb138 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'edc86e1645d742d48a50e74e9060cd6e'} {{(pid=80377) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 10:17:10.213361 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.notifications [None req-f89bbb39-964c-44fc-b98f-48eb0b0eb138 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'edc86e1645d742d48a50e74e9060cd6e'} {{(pid=80377) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 10:17:10.227781 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-028c88b7-72d3-4319-9bc3-8dadafac7a28 admin admin] Authenticating user token {{(pid=80378) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:17:10.229364 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.fernet_utils [None req-028c88b7-72d3-4319-9bc3-8dadafac7a28 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80378) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 10:17:10.234894 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.notifications [None req-f89bbb39-964c-44fc-b98f-48eb0b0eb138 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'edc86e1645d742d48a50e74e9060cd6e'} {{(pid=80377) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 10:17:10.236844 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.notifications [None req-f89bbb39-964c-44fc-b98f-48eb0b0eb138 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'edc86e1645d742d48a50e74e9060cd6e'} {{(pid=80377) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 10:17:10.250780 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-028c88b7-72d3-4319-9bc3-8dadafac7a28 admin admin] Validating token access rules against request {{(pid=80378) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:17:10.282085 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-302ca6c5-bbff-4126-b4fd-a6e900694ecf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '891005fa5ff74041b75747d16cc66798', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7047912c5acd4adeba5d87c3bf895e39', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80378) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:17:10.282756 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-302ca6c5-bbff-4126-b4fd-a6e900694ecf admin admin] REQUEST_METHOD: `DELETE` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:17:10.283042 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-302ca6c5-bbff-4126-b4fd-a6e900694ecf admin admin] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:17:10.283798 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-302ca6c5-bbff-4126-b4fd-a6e900694ecf admin admin] PATH_INFO: `/v3/projects/aea8ccf0fd71475da7205bd1db84c609` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:17:10.291951 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-302ca6c5-bbff-4126-b4fd-a6e900694ecf admin admin] RBAC: Authorizing `identity:delete_project(project_id=aea8ccf0fd71475da7205bd1db84c609)` {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:17:10.293925 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-302ca6c5-bbff-4126-b4fd-a6e900694ecf admin admin] RBAC: Authorization granted {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:17:10.297838 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.notifications [None req-f89bbb39-964c-44fc-b98f-48eb0b0eb138 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'edc86e1645d742d48a50e74e9060cd6e'} {{(pid=80377) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 10:17:10.313596 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: [pid: 80377|app: 0|req: 987/1971] 10.222.0.82 () {58 vars in 1224 bytes} [Tue Aug 2 10:17:10 2022] DELETE /identity/v3/users/edc86e1645d742d48a50e74e9060cd6e => generated 0 bytes in 304 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 02 10:17:10.449810 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.notifications [None req-302ca6c5-bbff-4126-b4fd-a6e900694ecf admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'aea8ccf0fd71475da7205bd1db84c609', 'request_id': 'req-302ca6c5-bbff-4126-b4fd-a6e900694ecf'} {{(pid=80378) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 10:17:10.464566 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.notifications [None req-302ca6c5-bbff-4126-b4fd-a6e900694ecf admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'aea8ccf0fd71475da7205bd1db84c609', 'request_id': 'req-302ca6c5-bbff-4126-b4fd-a6e900694ecf'} {{(pid=80378) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 10:17:10.493986 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.notifications [None req-302ca6c5-bbff-4126-b4fd-a6e900694ecf admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'aea8ccf0fd71475da7205bd1db84c609', 'request_id': 'req-302ca6c5-bbff-4126-b4fd-a6e900694ecf'} {{(pid=80378) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 10:17:10.521773 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.notifications [None req-302ca6c5-bbff-4126-b4fd-a6e900694ecf admin admin] The token cache is being invalidate because project aea8ccf0fd71475da7205bd1db84c609 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80378) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 02 10:17:10.522566 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.notifications [None req-302ca6c5-bbff-4126-b4fd-a6e900694ecf admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80378) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 10:17:10.523718 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.notifications [None req-302ca6c5-bbff-4126-b4fd-a6e900694ecf admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80378) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 10:17:10.526830 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 985/1972] 10.222.0.82 () {58 vars in 1233 bytes} [Tue Aug 2 10:17:10 2022] DELETE /identity/v3/projects/aea8ccf0fd71475da7205bd1db84c609 => generated 0 bytes in 300 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 02 10:17:10.531400 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f89bbb39-964c-44fc-b98f-48eb0b0eb138 admin admin] Authenticating user token {{(pid=80377) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:17:10.536070 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.fernet_utils [None req-f89bbb39-964c-44fc-b98f-48eb0b0eb138 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80377) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 10:17:10.697508 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG dbcounter [-] [80377] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80377) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 10:17:10.710544 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f89bbb39-964c-44fc-b98f-48eb0b0eb138 admin admin] Validating token access rules against request {{(pid=80377) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:17:10.826984 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ddcf7d89-079b-4b89-af7d-d459a5179b44 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '891005fa5ff74041b75747d16cc66798', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7047912c5acd4adeba5d87c3bf895e39', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80377) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:17:10.827748 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddcf7d89-079b-4b89-af7d-d459a5179b44 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:17:10.828438 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddcf7d89-079b-4b89-af7d-d459a5179b44 admin admin] SCRIPT_NAME: `/identity` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:17:10.831103 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddcf7d89-079b-4b89-af7d-d459a5179b44 admin admin] PATH_INFO: `/v3/users/45ab63261e53456cb3af5553d167a93f` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:17:10.833656 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ddcf7d89-079b-4b89-af7d-d459a5179b44 admin admin] RBAC: Authorizing `identity:delete_user(user_id=45ab63261e53456cb3af5553d167a93f)` {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:17:10.840739 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ddcf7d89-079b-4b89-af7d-d459a5179b44 admin admin] RBAC: Authorization granted {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:17:10.942706 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.notifications [None req-ddcf7d89-079b-4b89-af7d-d459a5179b44 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '45ab63261e53456cb3af5553d167a93f'} {{(pid=80377) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 10:17:10.959736 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.notifications [None req-ddcf7d89-079b-4b89-af7d-d459a5179b44 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '45ab63261e53456cb3af5553d167a93f'} {{(pid=80377) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 10:17:10.973133 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.notifications [None req-ddcf7d89-079b-4b89-af7d-d459a5179b44 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '45ab63261e53456cb3af5553d167a93f'} {{(pid=80377) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 10:17:10.974158 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.notifications [None req-ddcf7d89-079b-4b89-af7d-d459a5179b44 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '45ab63261e53456cb3af5553d167a93f'} {{(pid=80377) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 10:17:10.992297 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.notifications [None req-ddcf7d89-079b-4b89-af7d-d459a5179b44 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '45ab63261e53456cb3af5553d167a93f'} {{(pid=80377) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 10:17:11.007252 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: [pid: 80377|app: 0|req: 988/1973] 10.222.0.82 () {58 vars in 1224 bytes} [Tue Aug 2 10:17:10 2022] DELETE /identity/v3/users/45ab63261e53456cb3af5553d167a93f => generated 0 bytes in 476 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 02 10:17:11.334202 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-302ca6c5-bbff-4126-b4fd-a6e900694ecf admin admin] Authenticating user token {{(pid=80378) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:17:11.335477 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.fernet_utils [None req-302ca6c5-bbff-4126-b4fd-a6e900694ecf admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80378) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 10:17:11.358777 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-302ca6c5-bbff-4126-b4fd-a6e900694ecf admin admin] Validating token access rules against request {{(pid=80378) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:17:11.458517 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97639b12-1fac-4e63-b155-6f848159ca6c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '891005fa5ff74041b75747d16cc66798', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7047912c5acd4adeba5d87c3bf895e39', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80378) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:17:11.459279 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97639b12-1fac-4e63-b155-6f848159ca6c admin admin] REQUEST_METHOD: `DELETE` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:17:11.459634 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97639b12-1fac-4e63-b155-6f848159ca6c admin admin] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:17:11.460091 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97639b12-1fac-4e63-b155-6f848159ca6c admin admin] PATH_INFO: `/v3/projects/5723260254ef4af6a96055d5cfabe5d5` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:17:11.461408 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97639b12-1fac-4e63-b155-6f848159ca6c admin admin] RBAC: Authorizing `identity:delete_project(project_id=5723260254ef4af6a96055d5cfabe5d5)` {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:17:11.463581 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97639b12-1fac-4e63-b155-6f848159ca6c admin admin] RBAC: Authorization granted {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:17:11.520777 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.notifications [None req-97639b12-1fac-4e63-b155-6f848159ca6c admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '5723260254ef4af6a96055d5cfabe5d5', 'request_id': 'req-97639b12-1fac-4e63-b155-6f848159ca6c'} {{(pid=80378) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 10:17:11.529837 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.notifications [None req-97639b12-1fac-4e63-b155-6f848159ca6c admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '5723260254ef4af6a96055d5cfabe5d5', 'request_id': 'req-97639b12-1fac-4e63-b155-6f848159ca6c'} {{(pid=80378) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 10:17:11.536404 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG dbcounter [-] [80378] Writing DB stats keystone:SELECT=82,keystone:DELETE=16,keystone:INSERT=2 {{(pid=80378) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 10:17:11.537553 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.notifications [None req-97639b12-1fac-4e63-b155-6f848159ca6c admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '5723260254ef4af6a96055d5cfabe5d5', 'request_id': 'req-97639b12-1fac-4e63-b155-6f848159ca6c'} {{(pid=80378) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 10:17:11.557995 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.notifications [None req-97639b12-1fac-4e63-b155-6f848159ca6c admin admin] The token cache is being invalidate because project 5723260254ef4af6a96055d5cfabe5d5 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80378) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 02 10:17:11.558295 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.notifications [None req-97639b12-1fac-4e63-b155-6f848159ca6c admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80378) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 10:17:11.558964 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.notifications [None req-97639b12-1fac-4e63-b155-6f848159ca6c admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80378) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 10:17:11.561152 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 986/1974] 10.222.0.82 () {58 vars in 1233 bytes} [Tue Aug 2 10:17:11 2022] DELETE /identity/v3/projects/5723260254ef4af6a96055d5cfabe5d5 => generated 0 bytes in 229 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 02 10:17:11.566013 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efe2402b-d025-40d5-98d3-00a128330238 None None] REQUEST_METHOD: `POST` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:17:11.566483 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efe2402b-d025-40d5-98d3-00a128330238 None None] SCRIPT_NAME: `/identity` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:17:11.567004 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efe2402b-d025-40d5-98d3-00a128330238 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:17:11.587734 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97639b12-1fac-4e63-b155-6f848159ca6c admin admin] Authenticating user token {{(pid=80378) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:17:11.588984 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.auth.core [None req-efe2402b-d025-40d5-98d3-00a128330238 None None] MFA Rules not processed for user `891005fa5ff74041b75747d16cc66798`. Rule list: `[]` (Enabled: `True`). {{(pid=80377) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 02 10:17:11.589652 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.fernet_utils [None req-97639b12-1fac-4e63-b155-6f848159ca6c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80378) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 10:17:11.590251 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.fernet_utils [None req-efe2402b-d025-40d5-98d3-00a128330238 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80377) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 10:17:11.672344 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: [pid: 80377|app: 0|req: 989/1975] 10.222.0.82 () {58 vars in 958 bytes} [Tue Aug 2 10:17:11 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 107 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 02 10:17:11.680843 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da64e7f7-6808-4e39-8240-5b4e68b96813 None None] REQUEST_METHOD: `POST` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:17:11.681177 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da64e7f7-6808-4e39-8240-5b4e68b96813 None None] SCRIPT_NAME: `/identity` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:17:11.681571 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da64e7f7-6808-4e39-8240-5b4e68b96813 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:17:11.684128 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97639b12-1fac-4e63-b155-6f848159ca6c admin admin] Validating token access rules against request {{(pid=80378) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:17:11.689058 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ce2ddda-6d70-426a-acf3-5bffc47d1700 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '891005fa5ff74041b75747d16cc66798', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7047912c5acd4adeba5d87c3bf895e39', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80378) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:17:11.689956 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ce2ddda-6d70-426a-acf3-5bffc47d1700 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:17:11.690242 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ce2ddda-6d70-426a-acf3-5bffc47d1700 admin admin] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:17:11.690530 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ce2ddda-6d70-426a-acf3-5bffc47d1700 admin admin] PATH_INFO: `/v3/projects/1aa245953e8e4a25943016810bb93904` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:17:11.692412 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ce2ddda-6d70-426a-acf3-5bffc47d1700 admin admin] RBAC: Authorizing `identity:delete_project(project_id=1aa245953e8e4a25943016810bb93904)` {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:17:11.695425 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ce2ddda-6d70-426a-acf3-5bffc47d1700 admin admin] RBAC: Authorization granted {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:17:11.704044 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.auth.core [None req-da64e7f7-6808-4e39-8240-5b4e68b96813 None None] MFA Rules not processed for user `891005fa5ff74041b75747d16cc66798`. Rule list: `[]` (Enabled: `True`). {{(pid=80377) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 02 10:17:11.705906 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.fernet_utils [None req-da64e7f7-6808-4e39-8240-5b4e68b96813 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80377) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 10:17:11.717040 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: [pid: 80377|app: 0|req: 990/1976] 10.222.0.82 () {58 vars in 958 bytes} [Tue Aug 2 10:17:11 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 37 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 02 10:17:11.721151 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da64e7f7-6808-4e39-8240-5b4e68b96813 None None] Authenticating user token {{(pid=80377) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:17:11.730666 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da64e7f7-6808-4e39-8240-5b4e68b96813 None None] Validating token access rules against request {{(pid=80377) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:17:11.736937 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c53b28b-5736-425d-8d23-24a25c89260a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '891005fa5ff74041b75747d16cc66798', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7047912c5acd4adeba5d87c3bf895e39', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80377) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:17:11.737662 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c53b28b-5736-425d-8d23-24a25c89260a admin admin] REQUEST_METHOD: `GET` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:17:11.738043 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c53b28b-5736-425d-8d23-24a25c89260a admin admin] SCRIPT_NAME: `/identity` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:17:11.738323 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c53b28b-5736-425d-8d23-24a25c89260a admin admin] PATH_INFO: `/v3/domains` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:17:11.739161 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c53b28b-5736-425d-8d23-24a25c89260a admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80377) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 02 10:17:11.739471 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c53b28b-5736-425d-8d23-24a25c89260a admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:17:11.741123 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c53b28b-5736-425d-8d23-24a25c89260a admin admin] RBAC: Authorization granted {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:17:11.758179 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: [pid: 80377|app: 0|req: 991/1977] 10.222.0.82 () {58 vars in 1153 bytes} [Tue Aug 2 10:17:11 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 02 10:17:11.760683 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c53b28b-5736-425d-8d23-24a25c89260a admin admin] Authenticating user token {{(pid=80377) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:17:11.762829 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.notifications [None req-5ce2ddda-6d70-426a-acf3-5bffc47d1700 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '1aa245953e8e4a25943016810bb93904', 'request_id': 'req-5ce2ddda-6d70-426a-acf3-5bffc47d1700'} {{(pid=80378) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 10:17:11.769768 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG dbcounter [-] [80377] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80377) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 10:17:11.770487 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.notifications [None req-5ce2ddda-6d70-426a-acf3-5bffc47d1700 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '1aa245953e8e4a25943016810bb93904', 'request_id': 'req-5ce2ddda-6d70-426a-acf3-5bffc47d1700'} {{(pid=80378) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 10:17:11.777426 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.notifications [None req-5ce2ddda-6d70-426a-acf3-5bffc47d1700 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '1aa245953e8e4a25943016810bb93904', 'request_id': 'req-5ce2ddda-6d70-426a-acf3-5bffc47d1700'} {{(pid=80378) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 10:17:11.778312 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c53b28b-5736-425d-8d23-24a25c89260a admin admin] Validating token access rules against request {{(pid=80377) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:17:11.794992 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.notifications [None req-5ce2ddda-6d70-426a-acf3-5bffc47d1700 admin admin] The token cache is being invalidate because project 1aa245953e8e4a25943016810bb93904 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80378) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 02 10:17:11.795303 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.notifications [None req-5ce2ddda-6d70-426a-acf3-5bffc47d1700 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80378) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 10:17:11.796134 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.notifications [None req-5ce2ddda-6d70-426a-acf3-5bffc47d1700 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80378) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 10:17:11.798300 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 987/1978] 10.222.0.82 () {58 vars in 1233 bytes} [Tue Aug 2 10:17:11 2022] DELETE /identity/v3/projects/1aa245953e8e4a25943016810bb93904 => generated 0 bytes in 211 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 02 10:17:11.806956 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce806a91-b49e-43e6-8039-345e8c1c8bfc None None] REQUEST_METHOD: `POST` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:17:11.807238 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce806a91-b49e-43e6-8039-345e8c1c8bfc None None] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:17:11.807515 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce806a91-b49e-43e6-8039-345e8c1c8bfc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:17:11.832491 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.auth.core [None req-ce806a91-b49e-43e6-8039-345e8c1c8bfc None None] MFA Rules not processed for user `891005fa5ff74041b75747d16cc66798`. Rule list: `[]` (Enabled: `True`). {{(pid=80378) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 02 10:17:11.834197 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.fernet_utils [None req-ce806a91-b49e-43e6-8039-345e8c1c8bfc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80378) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 10:17:11.889619 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07b20cef-fd40-47c2-9e17-babbacb42d63 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '891005fa5ff74041b75747d16cc66798', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7047912c5acd4adeba5d87c3bf895e39', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80377) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:17:11.890579 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07b20cef-fd40-47c2-9e17-babbacb42d63 admin admin] REQUEST_METHOD: `POST` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:17:11.891092 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07b20cef-fd40-47c2-9e17-babbacb42d63 admin admin] SCRIPT_NAME: `/identity` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:17:11.891447 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07b20cef-fd40-47c2-9e17-babbacb42d63 admin admin] PATH_INFO: `/v3/projects` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:17:11.892554 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07b20cef-fd40-47c2-9e17-babbacb42d63 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:17:11.893980 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07b20cef-fd40-47c2-9e17-babbacb42d63 admin admin] RBAC: Authorization granted {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:17:11.945808 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: [pid: 80377|app: 0|req: 992/1979] 10.222.0.82 () {60 vars in 1153 bytes} [Tue Aug 2 10:17:11 2022] POST /identity/v3/projects => generated 385 bytes in 186 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 02 10:17:11.951635 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07b20cef-fd40-47c2-9e17-babbacb42d63 admin admin] Authenticating user token {{(pid=80377) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:17:11.954958 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.fernet_utils [None req-07b20cef-fd40-47c2-9e17-babbacb42d63 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80377) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 10:17:11.992795 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 988/1980] 10.222.0.82 () {58 vars in 958 bytes} [Tue Aug 2 10:17:11 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 187 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 02 10:17:11.994323 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8f72bdb-8a85-47c6-bd51-65c2e37ebb30 None None] REQUEST_METHOD: `POST` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:17:11.994616 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8f72bdb-8a85-47c6-bd51-65c2e37ebb30 None None] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:17:11.994890 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8f72bdb-8a85-47c6-bd51-65c2e37ebb30 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:17:12.035469 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.auth.core [None req-d8f72bdb-8a85-47c6-bd51-65c2e37ebb30 None None] MFA Rules not processed for user `891005fa5ff74041b75747d16cc66798`. Rule list: `[]` (Enabled: `True`). {{(pid=80378) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 02 10:17:12.036896 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.fernet_utils [None req-d8f72bdb-8a85-47c6-bd51-65c2e37ebb30 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80378) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 10:17:12.073101 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07b20cef-fd40-47c2-9e17-babbacb42d63 admin admin] Validating token access rules against request {{(pid=80377) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:17:12.080248 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fceb4904-92fd-4a97-a803-1fd98b960912 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '891005fa5ff74041b75747d16cc66798', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7047912c5acd4adeba5d87c3bf895e39', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80377) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:17:12.081185 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fceb4904-92fd-4a97-a803-1fd98b960912 admin admin] REQUEST_METHOD: `POST` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:17:12.081478 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fceb4904-92fd-4a97-a803-1fd98b960912 admin admin] SCRIPT_NAME: `/identity` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:17:12.082009 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fceb4904-92fd-4a97-a803-1fd98b960912 admin admin] PATH_INFO: `/v3/users` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:17:12.082882 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fceb4904-92fd-4a97-a803-1fd98b960912 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:17:12.086407 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fceb4904-92fd-4a97-a803-1fd98b960912 admin admin] RBAC: Authorization granted {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:17:12.088351 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG dbcounter [-] [80378] Writing DB stats keystone:SELECT=90,keystone:INSERT=2,keystone:DELETE=8 {{(pid=80378) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 10:17:12.125404 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: [pid: 80377|app: 0|req: 993/1981] 10.222.0.82 () {60 vars in 1144 bytes} [Tue Aug 2 10:17:11 2022] POST /identity/v3/users => generated 293 bytes in 174 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 02 10:17:12.131194 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fceb4904-92fd-4a97-a803-1fd98b960912 admin admin] Authenticating user token {{(pid=80377) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:17:12.147282 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fceb4904-92fd-4a97-a803-1fd98b960912 admin admin] Validating token access rules against request {{(pid=80377) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:17:12.156286 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67e2e5c4-cdd1-480d-a356-f8b3060f850e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '891005fa5ff74041b75747d16cc66798', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7047912c5acd4adeba5d87c3bf895e39', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80377) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:17:12.157331 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67e2e5c4-cdd1-480d-a356-f8b3060f850e admin admin] REQUEST_METHOD: `GET` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:17:12.158381 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67e2e5c4-cdd1-480d-a356-f8b3060f850e admin admin] SCRIPT_NAME: `/identity` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:17:12.160205 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67e2e5c4-cdd1-480d-a356-f8b3060f850e admin admin] PATH_INFO: `/v3/roles` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:17:12.161139 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67e2e5c4-cdd1-480d-a356-f8b3060f850e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:17:12.164157 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67e2e5c4-cdd1-480d-a356-f8b3060f850e admin admin] RBAC: Authorization granted {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:17:12.191152 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: [pid: 80377|app: 0|req: 994/1982] 10.222.0.82 () {58 vars in 1122 bytes} [Tue Aug 2 10:17:12 2022] GET /identity/v3/roles => generated 1420 bytes in 62 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 02 10:17:12.197459 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67e2e5c4-cdd1-480d-a356-f8b3060f850e admin admin] Authenticating user token {{(pid=80377) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:17:12.226328 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 989/1983] 10.222.0.82 () {58 vars in 958 bytes} [Tue Aug 2 10:17:11 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 233 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 02 10:17:12.227569 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8f72bdb-8a85-47c6-bd51-65c2e37ebb30 None None] Authenticating user token {{(pid=80378) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:17:12.228544 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67e2e5c4-cdd1-480d-a356-f8b3060f850e admin admin] Validating token access rules against request {{(pid=80377) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:17:12.245914 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8f72bdb-8a85-47c6-bd51-65c2e37ebb30 None None] Validating token access rules against request {{(pid=80378) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:17:12.248476 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fd162a0-7134-4205-96ce-a8eb2fc6112c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '891005fa5ff74041b75747d16cc66798', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7047912c5acd4adeba5d87c3bf895e39', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80377) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:17:12.249123 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fd162a0-7134-4205-96ce-a8eb2fc6112c admin admin] REQUEST_METHOD: `GET` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:17:12.249407 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fd162a0-7134-4205-96ce-a8eb2fc6112c admin admin] SCRIPT_NAME: `/identity` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:17:12.249705 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fd162a0-7134-4205-96ce-a8eb2fc6112c admin admin] PATH_INFO: `/v3/roles` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:17:12.250516 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7fd162a0-7134-4205-96ce-a8eb2fc6112c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:17:12.252124 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7fd162a0-7134-4205-96ce-a8eb2fc6112c admin admin] RBAC: Authorization granted {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:17:12.253711 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e82928d-2012-4890-a2cc-077e2b64559c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '891005fa5ff74041b75747d16cc66798', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7047912c5acd4adeba5d87c3bf895e39', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80378) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:17:12.254546 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e82928d-2012-4890-a2cc-077e2b64559c admin admin] REQUEST_METHOD: `GET` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:17:12.255302 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e82928d-2012-4890-a2cc-077e2b64559c admin admin] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:17:12.255589 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e82928d-2012-4890-a2cc-077e2b64559c admin admin] PATH_INFO: `/v3/domains` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:17:12.256590 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e82928d-2012-4890-a2cc-077e2b64559c admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80378) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 02 10:17:12.256711 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e82928d-2012-4890-a2cc-077e2b64559c admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:17:12.259000 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e82928d-2012-4890-a2cc-077e2b64559c admin admin] RBAC: Authorization granted {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:17:12.269167 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: [pid: 80377|app: 0|req: 995/1984] 10.222.0.82 () {58 vars in 1122 bytes} [Tue Aug 2 10:17:12 2022] GET /identity/v3/roles => generated 1420 bytes in 71 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 02 10:17:12.270901 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fd162a0-7134-4205-96ce-a8eb2fc6112c admin admin] Authenticating user token {{(pid=80377) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:17:12.277019 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 990/1985] 10.222.0.82 () {58 vars in 1153 bytes} [Tue Aug 2 10:17:12 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 50 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 02 10:17:12.287702 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fd162a0-7134-4205-96ce-a8eb2fc6112c admin admin] Validating token access rules against request {{(pid=80377) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:17:12.288124 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e82928d-2012-4890-a2cc-077e2b64559c admin admin] Authenticating user token {{(pid=80378) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:17:12.292864 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2c49717-6e06-468b-9da2-7aa24ed844e6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '891005fa5ff74041b75747d16cc66798', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7047912c5acd4adeba5d87c3bf895e39', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80377) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:17:12.293560 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2c49717-6e06-468b-9da2-7aa24ed844e6 admin admin] REQUEST_METHOD: `PUT` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:17:12.293902 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2c49717-6e06-468b-9da2-7aa24ed844e6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:17:12.294772 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2c49717-6e06-468b-9da2-7aa24ed844e6 admin admin] PATH_INFO: `/v3/projects/4dcc9e0034a349beb0c7b1cb91d54651/users/1eb84c516a1c4a75a0a4587c95f43f5c/roles/e1c04b19fc4543b8b54f58bbcffbc598` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:17:12.311027 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e82928d-2012-4890-a2cc-077e2b64559c admin admin] Validating token access rules against request {{(pid=80378) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:17:12.319095 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12ca5ad5-fbad-425b-b3e6-92d66e40e6a3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '891005fa5ff74041b75747d16cc66798', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7047912c5acd4adeba5d87c3bf895e39', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80378) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:17:12.320638 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12ca5ad5-fbad-425b-b3e6-92d66e40e6a3 admin admin] REQUEST_METHOD: `POST` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:17:12.321377 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12ca5ad5-fbad-425b-b3e6-92d66e40e6a3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:17:12.321937 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12ca5ad5-fbad-425b-b3e6-92d66e40e6a3 admin admin] PATH_INFO: `/v3/projects` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:17:12.323138 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12ca5ad5-fbad-425b-b3e6-92d66e40e6a3 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:17:12.325701 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12ca5ad5-fbad-425b-b3e6-92d66e40e6a3 admin admin] RBAC: Authorization granted {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:17:12.347424 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2c49717-6e06-468b-9da2-7aa24ed844e6 admin admin] RBAC: Authorizing `identity:create_grant(project_id=4dcc9e0034a349beb0c7b1cb91d54651, user_id=1eb84c516a1c4a75a0a4587c95f43f5c, role_id=e1c04b19fc4543b8b54f58bbcffbc598)` {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:17:12.350750 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2c49717-6e06-468b-9da2-7aa24ed844e6 admin admin] RBAC: Authorization granted {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:17:12.366713 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 991/1986] 10.222.0.82 () {60 vars in 1153 bytes} [Tue Aug 2 10:17:12 2022] POST /identity/v3/projects => generated 371 bytes in 79 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 02 10:17:12.370774 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12ca5ad5-fbad-425b-b3e6-92d66e40e6a3 admin admin] Authenticating user token {{(pid=80378) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:17:12.392563 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: [pid: 80377|app: 0|req: 996/1987] 10.222.0.82 () {60 vars in 1483 bytes} [Tue Aug 2 10:17:12 2022] PUT /identity/v3/projects/4dcc9e0034a349beb0c7b1cb91d54651/users/1eb84c516a1c4a75a0a4587c95f43f5c/roles/e1c04b19fc4543b8b54f58bbcffbc598 => generated 0 bytes in 122 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 02 10:17:12.404784 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6576a164-dc01-474d-a7cc-665a1ff67b42 None None] REQUEST_METHOD: `POST` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:17:12.405516 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6576a164-dc01-474d-a7cc-665a1ff67b42 None None] SCRIPT_NAME: `/identity` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:17:12.405516 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6576a164-dc01-474d-a7cc-665a1ff67b42 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:17:12.435872 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.auth.core [None req-6576a164-dc01-474d-a7cc-665a1ff67b42 None None] MFA Rules not processed for user `1eb84c516a1c4a75a0a4587c95f43f5c`. Rule list: `[]` (Enabled: `True`). {{(pid=80377) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 02 10:17:12.437137 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.fernet_utils [None req-6576a164-dc01-474d-a7cc-665a1ff67b42 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80377) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 10:17:12.441976 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12ca5ad5-fbad-425b-b3e6-92d66e40e6a3 admin admin] Validating token access rules against request {{(pid=80378) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:17:12.462118 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG dbcounter [-] [80377] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80377) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 10:17:12.537925 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f1fce09-788d-4fc0-afec-5983f26c7b4e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '891005fa5ff74041b75747d16cc66798', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7047912c5acd4adeba5d87c3bf895e39', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80378) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:17:12.538622 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f1fce09-788d-4fc0-afec-5983f26c7b4e admin admin] REQUEST_METHOD: `POST` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:17:12.538906 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f1fce09-788d-4fc0-afec-5983f26c7b4e admin admin] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:17:12.539239 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f1fce09-788d-4fc0-afec-5983f26c7b4e admin admin] PATH_INFO: `/v3/users` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:17:12.540121 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f1fce09-788d-4fc0-afec-5983f26c7b4e admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:17:12.544188 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f1fce09-788d-4fc0-afec-5983f26c7b4e admin admin] RBAC: Authorization granted {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:17:12.553661 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: [pid: 80377|app: 0|req: 997/1988] 10.222.0.82 () {58 vars in 958 bytes} [Tue Aug 2 10:17:12 2022] POST /identity/v3/auth/tokens => generated 3262 bytes in 150 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 02 10:17:12.555200 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6576a164-dc01-474d-a7cc-665a1ff67b42 None None] Authenticating user token {{(pid=80377) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:17:12.558789 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.fernet_utils [None req-6576a164-dc01-474d-a7cc-665a1ff67b42 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80377) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 10:17:12.586831 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 992/1989] 10.222.0.82 () {60 vars in 1144 bytes} [Tue Aug 2 10:17:12 2022] POST /identity/v3/users => generated 286 bytes in 217 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 02 10:17:12.593387 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f1fce09-788d-4fc0-afec-5983f26c7b4e admin admin] Authenticating user token {{(pid=80378) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:17:12.612594 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f1fce09-788d-4fc0-afec-5983f26c7b4e admin admin] Validating token access rules against request {{(pid=80378) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:17:12.632899 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5fdb245-9503-4a38-85e2-fe0bdf40997b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '891005fa5ff74041b75747d16cc66798', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7047912c5acd4adeba5d87c3bf895e39', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80378) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:17:12.633683 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5fdb245-9503-4a38-85e2-fe0bdf40997b admin admin] REQUEST_METHOD: `GET` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:17:12.634120 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5fdb245-9503-4a38-85e2-fe0bdf40997b admin admin] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:17:12.634497 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5fdb245-9503-4a38-85e2-fe0bdf40997b admin admin] PATH_INFO: `/v3/roles` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:17:12.635364 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5fdb245-9503-4a38-85e2-fe0bdf40997b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:17:12.637454 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5fdb245-9503-4a38-85e2-fe0bdf40997b admin admin] RBAC: Authorization granted {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:17:12.641670 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6576a164-dc01-474d-a7cc-665a1ff67b42 None None] Validating token access rules against request {{(pid=80377) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:17:12.649650 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1208b4a-6812-484f-9b77-74c56dfd8cc2 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1023de281ae74e9d8a6d6c312c1c97a7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f1b38ebb804c41c69772c998aae72c41', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80377) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:17:12.649933 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 993/1990] 10.222.0.82 () {58 vars in 1122 bytes} [Tue Aug 2 10:17:12 2022] GET /identity/v3/roles => generated 1420 bytes in 58 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 02 10:17:12.653237 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1208b4a-6812-484f-9b77-74c56dfd8cc2 service cinder] REQUEST_METHOD: `GET` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:17:12.654098 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1208b4a-6812-484f-9b77-74c56dfd8cc2 service cinder] SCRIPT_NAME: `/identity` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:17:12.654392 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1208b4a-6812-484f-9b77-74c56dfd8cc2 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:17:12.654929 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5fdb245-9503-4a38-85e2-fe0bdf40997b admin admin] Authenticating user token {{(pid=80378) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:17:12.666313 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5fdb245-9503-4a38-85e2-fe0bdf40997b admin admin] Validating token access rules against request {{(pid=80378) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:17:12.674154 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1208b4a-6812-484f-9b77-74c56dfd8cc2 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:17:12.675057 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a54aee7f-a72a-4acd-ad8d-35b8d15c60bf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '891005fa5ff74041b75747d16cc66798', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7047912c5acd4adeba5d87c3bf895e39', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80378) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:17:12.675784 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1208b4a-6812-484f-9b77-74c56dfd8cc2 service cinder] RBAC: Authorization granted {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:17:12.676502 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a54aee7f-a72a-4acd-ad8d-35b8d15c60bf admin admin] REQUEST_METHOD: `GET` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:17:12.677146 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a54aee7f-a72a-4acd-ad8d-35b8d15c60bf admin admin] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:17:12.677569 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a54aee7f-a72a-4acd-ad8d-35b8d15c60bf admin admin] PATH_INFO: `/v3/roles` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:17:12.678525 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a54aee7f-a72a-4acd-ad8d-35b8d15c60bf admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:17:12.681240 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a54aee7f-a72a-4acd-ad8d-35b8d15c60bf admin admin] RBAC: Authorization granted {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:17:12.688509 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: [pid: 80377|app: 0|req: 998/1991] 10.222.0.82 () {60 vars in 1365 bytes} [Tue Aug 2 10:17:12 2022] GET /identity/v3/auth/tokens => generated 3262 bytes in 134 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 02 10:17:12.689114 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG dbcounter [-] [80378] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=80378) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 10:17:12.701750 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1208b4a-6812-484f-9b77-74c56dfd8cc2 service cinder] Authenticating user token {{(pid=80377) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:17:12.703937 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 994/1992] 10.222.0.82 () {58 vars in 1122 bytes} [Tue Aug 2 10:17:12 2022] GET /identity/v3/roles => generated 1420 bytes in 50 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 02 10:17:12.715930 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1208b4a-6812-484f-9b77-74c56dfd8cc2 service cinder] Validating token access rules against request {{(pid=80377) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:17:12.721429 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-186c6b61-6d74-4311-be63-c17bb48526cd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '891005fa5ff74041b75747d16cc66798', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7047912c5acd4adeba5d87c3bf895e39', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80377) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:17:12.722372 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-186c6b61-6d74-4311-be63-c17bb48526cd admin admin] REQUEST_METHOD: `PUT` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:17:12.722610 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-186c6b61-6d74-4311-be63-c17bb48526cd admin admin] SCRIPT_NAME: `/identity` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:17:12.722943 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-186c6b61-6d74-4311-be63-c17bb48526cd admin admin] PATH_INFO: `/v3/projects/32759e90fe6b4ad68a2ad30ddffad139/users/b864801636474f14b35a8cafb07b53b7/roles/e1c04b19fc4543b8b54f58bbcffbc598` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:17:12.747202 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-186c6b61-6d74-4311-be63-c17bb48526cd admin admin] RBAC: Authorizing `identity:create_grant(project_id=32759e90fe6b4ad68a2ad30ddffad139, user_id=b864801636474f14b35a8cafb07b53b7, role_id=e1c04b19fc4543b8b54f58bbcffbc598)` {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:17:12.755013 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-186c6b61-6d74-4311-be63-c17bb48526cd admin admin] RBAC: Authorization granted {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:17:12.778340 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: [pid: 80377|app: 0|req: 999/1993] 10.222.0.82 () {60 vars in 1483 bytes} [Tue Aug 2 10:17:12 2022] PUT /identity/v3/projects/32759e90fe6b4ad68a2ad30ddffad139/users/b864801636474f14b35a8cafb07b53b7/roles/e1c04b19fc4543b8b54f58bbcffbc598 => generated 0 bytes in 78 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 02 10:17:12.791395 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f43a91ee-af65-47b9-814a-acbe2d9eca8b None None] REQUEST_METHOD: `POST` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:17:12.791681 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f43a91ee-af65-47b9-814a-acbe2d9eca8b None None] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:17:12.791963 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f43a91ee-af65-47b9-814a-acbe2d9eca8b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:17:12.827987 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.auth.core [None req-f43a91ee-af65-47b9-814a-acbe2d9eca8b None None] MFA Rules not processed for user `b864801636474f14b35a8cafb07b53b7`. Rule list: `[]` (Enabled: `True`). {{(pid=80378) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 02 10:17:12.829501 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.fernet_utils [None req-f43a91ee-af65-47b9-814a-acbe2d9eca8b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80378) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 10:17:12.938486 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 995/1994] 10.222.0.82 () {58 vars in 958 bytes} [Tue Aug 2 10:17:12 2022] POST /identity/v3/auth/tokens => generated 3248 bytes in 149 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 02 10:17:12.951776 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-186c6b61-6d74-4311-be63-c17bb48526cd admin admin] Authenticating user token {{(pid=80377) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:17:12.953823 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.fernet_utils [None req-186c6b61-6d74-4311-be63-c17bb48526cd admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80377) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 10:17:13.026484 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-186c6b61-6d74-4311-be63-c17bb48526cd admin admin] Validating token access rules against request {{(pid=80377) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:17:13.030475 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b0f9240-ed5e-4a25-8cd1-f1a7bf985387 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1023de281ae74e9d8a6d6c312c1c97a7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f1b38ebb804c41c69772c998aae72c41', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80377) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:17:13.031139 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b0f9240-ed5e-4a25-8cd1-f1a7bf985387 service cinder] REQUEST_METHOD: `GET` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:17:13.031428 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b0f9240-ed5e-4a25-8cd1-f1a7bf985387 service cinder] SCRIPT_NAME: `/identity` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:17:13.031709 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b0f9240-ed5e-4a25-8cd1-f1a7bf985387 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:17:13.040875 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b0f9240-ed5e-4a25-8cd1-f1a7bf985387 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:17:13.042495 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b0f9240-ed5e-4a25-8cd1-f1a7bf985387 service cinder] RBAC: Authorization granted {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:17:13.055275 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: [pid: 80377|app: 0|req: 1000/1995] 10.222.0.82 () {60 vars in 1365 bytes} [Tue Aug 2 10:17:12 2022] GET /identity/v3/auth/tokens => generated 3248 bytes in 105 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 02 10:17:13.089119 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f43a91ee-af65-47b9-814a-acbe2d9eca8b None None] Authenticating user token {{(pid=80378) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:17:13.160167 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f43a91ee-af65-47b9-814a-acbe2d9eca8b None None] Validating token access rules against request {{(pid=80378) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:17:13.164815 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed2b935c-5b12-4964-a155-6a1f2dc10852 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '891005fa5ff74041b75747d16cc66798', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7047912c5acd4adeba5d87c3bf895e39', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80378) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:17:13.165472 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed2b935c-5b12-4964-a155-6a1f2dc10852 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:17:13.165853 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed2b935c-5b12-4964-a155-6a1f2dc10852 admin admin] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:17:13.166145 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed2b935c-5b12-4964-a155-6a1f2dc10852 admin admin] PATH_INFO: `/v3/users/b864801636474f14b35a8cafb07b53b7` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:17:13.168639 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed2b935c-5b12-4964-a155-6a1f2dc10852 admin admin] RBAC: Authorizing `identity:delete_user(user_id=b864801636474f14b35a8cafb07b53b7)` {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:17:13.171167 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed2b935c-5b12-4964-a155-6a1f2dc10852 admin admin] RBAC: Authorization granted {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:17:13.230170 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.notifications [None req-ed2b935c-5b12-4964-a155-6a1f2dc10852 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'b864801636474f14b35a8cafb07b53b7'} {{(pid=80378) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 10:17:13.239172 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.notifications [None req-ed2b935c-5b12-4964-a155-6a1f2dc10852 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'b864801636474f14b35a8cafb07b53b7'} {{(pid=80378) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 10:17:13.247150 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.notifications [None req-ed2b935c-5b12-4964-a155-6a1f2dc10852 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'b864801636474f14b35a8cafb07b53b7'} {{(pid=80378) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 10:17:13.247884 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.notifications [None req-ed2b935c-5b12-4964-a155-6a1f2dc10852 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'b864801636474f14b35a8cafb07b53b7'} {{(pid=80378) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 10:17:13.262329 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.notifications [None req-ed2b935c-5b12-4964-a155-6a1f2dc10852 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'b864801636474f14b35a8cafb07b53b7'} {{(pid=80378) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 10:17:13.270312 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 996/1996] 10.222.0.82 () {58 vars in 1224 bytes} [Tue Aug 2 10:17:13 2022] DELETE /identity/v3/users/b864801636474f14b35a8cafb07b53b7 => generated 0 bytes in 182 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 02 10:17:13.274551 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0c63f64-0052-4303-8a00-f59a08b7491c None None] REQUEST_METHOD: `POST` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:17:13.278148 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0c63f64-0052-4303-8a00-f59a08b7491c None None] SCRIPT_NAME: `/identity` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:17:13.278148 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0c63f64-0052-4303-8a00-f59a08b7491c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:17:13.279986 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed2b935c-5b12-4964-a155-6a1f2dc10852 admin admin] Authenticating user token {{(pid=80378) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:17:13.281438 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.fernet_utils [None req-ed2b935c-5b12-4964-a155-6a1f2dc10852 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80378) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 10:17:13.296159 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG dbcounter [-] [80378] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80378) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 10:17:13.297229 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.auth.core [None req-d0c63f64-0052-4303-8a00-f59a08b7491c None None] MFA Rules not processed for user `891005fa5ff74041b75747d16cc66798`. Rule list: `[]` (Enabled: `True`). {{(pid=80377) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 02 10:17:13.298697 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.fernet_utils [None req-d0c63f64-0052-4303-8a00-f59a08b7491c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80377) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 10:17:13.347925 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG dbcounter [-] [80377] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80377) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 10:17:13.372921 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: [pid: 80377|app: 0|req: 1001/1997] 10.222.0.82 () {58 vars in 958 bytes} [Tue Aug 2 10:17:13 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 02 10:17:13.380011 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed2b935c-5b12-4964-a155-6a1f2dc10852 admin admin] Validating token access rules against request {{(pid=80378) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:17:13.387436 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25cd079c-cfb4-49f5-82ae-df58e4ff7336 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '891005fa5ff74041b75747d16cc66798', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7047912c5acd4adeba5d87c3bf895e39', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80378) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:17:13.388135 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25cd079c-cfb4-49f5-82ae-df58e4ff7336 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:17:13.388354 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0c63f64-0052-4303-8a00-f59a08b7491c None None] Authenticating user token {{(pid=80377) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:17:13.388476 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25cd079c-cfb4-49f5-82ae-df58e4ff7336 admin admin] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:17:13.388681 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25cd079c-cfb4-49f5-82ae-df58e4ff7336 admin admin] PATH_INFO: `/v3/users/5f848c9acf0a425c935bf996ae77df93` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:17:13.389889 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.fernet_utils [None req-d0c63f64-0052-4303-8a00-f59a08b7491c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80377) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 10:17:13.391440 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25cd079c-cfb4-49f5-82ae-df58e4ff7336 admin admin] RBAC: Authorizing `identity:delete_user(user_id=5f848c9acf0a425c935bf996ae77df93)` {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:17:13.393433 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25cd079c-cfb4-49f5-82ae-df58e4ff7336 admin admin] RBAC: Authorization granted {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:17:13.467799 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.notifications [None req-25cd079c-cfb4-49f5-82ae-df58e4ff7336 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '5f848c9acf0a425c935bf996ae77df93'} {{(pid=80378) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 10:17:13.475899 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0c63f64-0052-4303-8a00-f59a08b7491c None None] Validating token access rules against request {{(pid=80377) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:17:13.480429 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.notifications [None req-25cd079c-cfb4-49f5-82ae-df58e4ff7336 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '5f848c9acf0a425c935bf996ae77df93'} {{(pid=80378) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 10:17:13.485059 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51f152e4-5754-478e-a230-850bc6d63a7d service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '33b87170f0634b67bce60750847b5cbe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f1b38ebb804c41c69772c998aae72c41', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80377) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:17:13.486271 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51f152e4-5754-478e-a230-850bc6d63a7d service neutron] REQUEST_METHOD: `GET` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:17:13.486883 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51f152e4-5754-478e-a230-850bc6d63a7d service neutron] SCRIPT_NAME: `/identity` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:17:13.488122 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51f152e4-5754-478e-a230-850bc6d63a7d service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:17:13.491348 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.notifications [None req-25cd079c-cfb4-49f5-82ae-df58e4ff7336 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '5f848c9acf0a425c935bf996ae77df93'} {{(pid=80378) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 10:17:13.492031 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.notifications [None req-25cd079c-cfb4-49f5-82ae-df58e4ff7336 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '5f848c9acf0a425c935bf996ae77df93'} {{(pid=80378) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 10:17:13.502001 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51f152e4-5754-478e-a230-850bc6d63a7d service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:17:13.503876 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51f152e4-5754-478e-a230-850bc6d63a7d service neutron] RBAC: Authorization granted {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:17:13.531496 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: [pid: 80377|app: 0|req: 1002/1998] 10.222.0.82 () {60 vars in 1365 bytes} [Tue Aug 2 10:17:13 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 145 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 02 10:17:13.532347 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.notifications [None req-25cd079c-cfb4-49f5-82ae-df58e4ff7336 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '5f848c9acf0a425c935bf996ae77df93'} {{(pid=80378) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 10:17:13.542068 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 997/1999] 10.222.0.82 () {58 vars in 1224 bytes} [Tue Aug 2 10:17:13 2022] DELETE /identity/v3/users/5f848c9acf0a425c935bf996ae77df93 => generated 0 bytes in 263 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 02 10:17:13.549035 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfc75cab-1f04-4ffe-8dd2-4b2c3b6d851d None None] REQUEST_METHOD: `POST` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:17:13.549035 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfc75cab-1f04-4ffe-8dd2-4b2c3b6d851d None None] SCRIPT_NAME: `/identity` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:17:13.549291 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfc75cab-1f04-4ffe-8dd2-4b2c3b6d851d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:17:13.571383 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.auth.core [None req-cfc75cab-1f04-4ffe-8dd2-4b2c3b6d851d None None] MFA Rules not processed for user `891005fa5ff74041b75747d16cc66798`. Rule list: `[]` (Enabled: `True`). {{(pid=80377) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 02 10:17:13.572675 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.fernet_utils [None req-cfc75cab-1f04-4ffe-8dd2-4b2c3b6d851d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80377) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 10:17:13.636581 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25cd079c-cfb4-49f5-82ae-df58e4ff7336 admin admin] Authenticating user token {{(pid=80378) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:17:13.638775 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.fernet_utils [None req-25cd079c-cfb4-49f5-82ae-df58e4ff7336 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80378) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 10:17:13.673344 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: [pid: 80377|app: 0|req: 1003/2000] 10.222.0.82 () {58 vars in 958 bytes} [Tue Aug 2 10:17:13 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 126 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 02 10:17:13.691169 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfc75cab-1f04-4ffe-8dd2-4b2c3b6d851d None None] Authenticating user token {{(pid=80377) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:17:13.692951 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.fernet_utils [None req-cfc75cab-1f04-4ffe-8dd2-4b2c3b6d851d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80377) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 10:17:13.723768 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25cd079c-cfb4-49f5-82ae-df58e4ff7336 admin admin] Validating token access rules against request {{(pid=80378) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:17:13.739147 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-217de54c-ce11-4f9e-a5f6-a94c37bf5290 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '891005fa5ff74041b75747d16cc66798', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7047912c5acd4adeba5d87c3bf895e39', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80378) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:17:13.739819 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-217de54c-ce11-4f9e-a5f6-a94c37bf5290 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:17:13.740101 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-217de54c-ce11-4f9e-a5f6-a94c37bf5290 admin admin] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:17:13.740378 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-217de54c-ce11-4f9e-a5f6-a94c37bf5290 admin admin] PATH_INFO: `/v3/users/8a976d24cbfd407eb9da97e21e37a733` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:17:13.743606 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-217de54c-ce11-4f9e-a5f6-a94c37bf5290 admin admin] RBAC: Authorizing `identity:delete_user(user_id=8a976d24cbfd407eb9da97e21e37a733)` {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:17:13.746566 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-217de54c-ce11-4f9e-a5f6-a94c37bf5290 admin admin] RBAC: Authorization granted {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:17:13.778760 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfc75cab-1f04-4ffe-8dd2-4b2c3b6d851d None None] Validating token access rules against request {{(pid=80377) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:17:13.802311 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb121626-0a0b-4e72-ae5b-4b7244c0fb77 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '33b87170f0634b67bce60750847b5cbe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f1b38ebb804c41c69772c998aae72c41', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80377) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:17:13.803628 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb121626-0a0b-4e72-ae5b-4b7244c0fb77 service neutron] REQUEST_METHOD: `GET` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:17:13.803908 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb121626-0a0b-4e72-ae5b-4b7244c0fb77 service neutron] SCRIPT_NAME: `/identity` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:17:13.804504 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb121626-0a0b-4e72-ae5b-4b7244c0fb77 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:17:13.813858 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG dbcounter [-] [80378] Writing DB stats keystone:SELECT=84,keystone:DELETE=15,keystone:INSERT=1 {{(pid=80378) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 10:17:13.823846 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb121626-0a0b-4e72-ae5b-4b7244c0fb77 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:17:13.827768 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb121626-0a0b-4e72-ae5b-4b7244c0fb77 service neutron] RBAC: Authorization granted {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:17:13.838815 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.notifications [None req-217de54c-ce11-4f9e-a5f6-a94c37bf5290 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '8a976d24cbfd407eb9da97e21e37a733'} {{(pid=80378) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 10:17:13.847558 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: [pid: 80377|app: 0|req: 1004/2001] 10.222.0.82 () {60 vars in 1365 bytes} [Tue Aug 2 10:17:13 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 157 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 02 10:17:13.852037 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.notifications [None req-217de54c-ce11-4f9e-a5f6-a94c37bf5290 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '8a976d24cbfd407eb9da97e21e37a733'} {{(pid=80378) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 10:17:13.873024 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.notifications [None req-217de54c-ce11-4f9e-a5f6-a94c37bf5290 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '8a976d24cbfd407eb9da97e21e37a733'} {{(pid=80378) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 10:17:13.878905 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.notifications [None req-217de54c-ce11-4f9e-a5f6-a94c37bf5290 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '8a976d24cbfd407eb9da97e21e37a733'} {{(pid=80378) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 10:17:13.896783 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.notifications [None req-217de54c-ce11-4f9e-a5f6-a94c37bf5290 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '8a976d24cbfd407eb9da97e21e37a733'} {{(pid=80378) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 10:17:13.904315 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 998/2002] 10.222.0.82 () {58 vars in 1224 bytes} [Tue Aug 2 10:17:13 2022] DELETE /identity/v3/users/8a976d24cbfd407eb9da97e21e37a733 => generated 0 bytes in 268 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 02 10:17:13.908692 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-659b5e42-0be1-4dc5-8d6a-b1d9f301f5d3 None None] REQUEST_METHOD: `POST` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:17:13.909099 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-659b5e42-0be1-4dc5-8d6a-b1d9f301f5d3 None None] SCRIPT_NAME: `/identity` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:17:13.909380 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-659b5e42-0be1-4dc5-8d6a-b1d9f301f5d3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:17:13.930307 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.auth.core [None req-659b5e42-0be1-4dc5-8d6a-b1d9f301f5d3 None None] MFA Rules not processed for user `891005fa5ff74041b75747d16cc66798`. Rule list: `[]` (Enabled: `True`). {{(pid=80377) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 02 10:17:13.931824 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.fernet_utils [None req-659b5e42-0be1-4dc5-8d6a-b1d9f301f5d3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80377) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 10:17:13.964077 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG dbcounter [-] [80377] Writing DB stats keystone:SELECT=100 {{(pid=80377) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 10:17:13.990720 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-217de54c-ce11-4f9e-a5f6-a94c37bf5290 admin admin] Authenticating user token {{(pid=80378) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:17:13.992175 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.fernet_utils [None req-217de54c-ce11-4f9e-a5f6-a94c37bf5290 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80378) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 10:17:14.007654 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: [pid: 80377|app: 0|req: 1005/2003] 10.222.0.82 () {58 vars in 958 bytes} [Tue Aug 2 10:17:13 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 02 10:17:14.030173 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-659b5e42-0be1-4dc5-8d6a-b1d9f301f5d3 None None] Authenticating user token {{(pid=80377) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:17:14.033326 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.fernet_utils [None req-659b5e42-0be1-4dc5-8d6a-b1d9f301f5d3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80377) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 10:17:14.097387 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-217de54c-ce11-4f9e-a5f6-a94c37bf5290 admin admin] Validating token access rules against request {{(pid=80378) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:17:14.104408 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05015b70-411b-4202-9890-c4a2429dcc7e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '891005fa5ff74041b75747d16cc66798', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7047912c5acd4adeba5d87c3bf895e39', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80378) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:17:14.106057 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05015b70-411b-4202-9890-c4a2429dcc7e admin admin] REQUEST_METHOD: `DELETE` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:17:14.106928 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05015b70-411b-4202-9890-c4a2429dcc7e admin admin] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:17:14.106928 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05015b70-411b-4202-9890-c4a2429dcc7e admin admin] PATH_INFO: `/v3/projects/32759e90fe6b4ad68a2ad30ddffad139` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:17:14.107891 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05015b70-411b-4202-9890-c4a2429dcc7e admin admin] RBAC: Authorizing `identity:delete_project(project_id=32759e90fe6b4ad68a2ad30ddffad139)` {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:17:14.110035 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05015b70-411b-4202-9890-c4a2429dcc7e admin admin] RBAC: Authorization granted {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:17:14.110882 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-659b5e42-0be1-4dc5-8d6a-b1d9f301f5d3 None None] Validating token access rules against request {{(pid=80377) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:17:14.114895 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fb3bc90-8810-4ce6-919d-845ea05f7082 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '33b87170f0634b67bce60750847b5cbe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f1b38ebb804c41c69772c998aae72c41', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80377) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:17:14.115604 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fb3bc90-8810-4ce6-919d-845ea05f7082 service neutron] REQUEST_METHOD: `GET` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:17:14.115966 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fb3bc90-8810-4ce6-919d-845ea05f7082 service neutron] SCRIPT_NAME: `/identity` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:17:14.116320 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fb3bc90-8810-4ce6-919d-845ea05f7082 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:17:14.130741 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7fb3bc90-8810-4ce6-919d-845ea05f7082 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:17:14.134008 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7fb3bc90-8810-4ce6-919d-845ea05f7082 service neutron] RBAC: Authorization granted {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:17:14.156395 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: [pid: 80377|app: 0|req: 1006/2004] 10.222.0.82 () {60 vars in 1365 bytes} [Tue Aug 2 10:17:14 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 130 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 02 10:17:14.183320 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.notifications [None req-05015b70-411b-4202-9890-c4a2429dcc7e admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '32759e90fe6b4ad68a2ad30ddffad139', 'request_id': 'req-05015b70-411b-4202-9890-c4a2429dcc7e'} {{(pid=80378) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 10:17:14.191921 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.notifications [None req-05015b70-411b-4202-9890-c4a2429dcc7e admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '32759e90fe6b4ad68a2ad30ddffad139', 'request_id': 'req-05015b70-411b-4202-9890-c4a2429dcc7e'} {{(pid=80378) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 10:17:14.202097 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.notifications [None req-05015b70-411b-4202-9890-c4a2429dcc7e admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '32759e90fe6b4ad68a2ad30ddffad139', 'request_id': 'req-05015b70-411b-4202-9890-c4a2429dcc7e'} {{(pid=80378) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 10:17:14.220025 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.notifications [None req-05015b70-411b-4202-9890-c4a2429dcc7e admin admin] The token cache is being invalidate because project 32759e90fe6b4ad68a2ad30ddffad139 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80378) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 02 10:17:14.220432 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.notifications [None req-05015b70-411b-4202-9890-c4a2429dcc7e admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80378) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 10:17:14.221721 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.notifications [None req-05015b70-411b-4202-9890-c4a2429dcc7e admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80378) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 10:17:14.228171 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 999/2005] 10.222.0.82 () {58 vars in 1233 bytes} [Tue Aug 2 10:17:13 2022] DELETE /identity/v3/projects/32759e90fe6b4ad68a2ad30ddffad139 => generated 0 bytes in 234 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 02 10:17:14.230407 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b53baf9-bc07-4487-9e4f-5bd3f1b6ea77 None None] REQUEST_METHOD: `POST` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:17:14.230842 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b53baf9-bc07-4487-9e4f-5bd3f1b6ea77 None None] SCRIPT_NAME: `/identity` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:17:14.231169 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b53baf9-bc07-4487-9e4f-5bd3f1b6ea77 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:17:14.258795 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.auth.core [None req-9b53baf9-bc07-4487-9e4f-5bd3f1b6ea77 None None] MFA Rules not processed for user `891005fa5ff74041b75747d16cc66798`. Rule list: `[]` (Enabled: `True`). {{(pid=80377) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 02 10:17:14.260198 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.fernet_utils [None req-9b53baf9-bc07-4487-9e4f-5bd3f1b6ea77 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80377) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 10:17:14.279616 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05015b70-411b-4202-9890-c4a2429dcc7e admin admin] Authenticating user token {{(pid=80378) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:17:14.281261 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.fernet_utils [None req-05015b70-411b-4202-9890-c4a2429dcc7e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80378) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 10:17:14.384488 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG dbcounter [-] [80378] Writing DB stats keystone:SELECT=86,keystone:DELETE=12,keystone:INSERT=2 {{(pid=80378) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 10:17:14.404114 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: [pid: 80377|app: 0|req: 1007/2006] 10.222.0.82 () {58 vars in 958 bytes} [Tue Aug 2 10:17:14 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 175 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 02 10:17:14.412388 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4b1f60b-650e-48c5-9773-23c2631fd4ff None None] REQUEST_METHOD: `POST` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:17:14.412785 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4b1f60b-650e-48c5-9773-23c2631fd4ff None None] SCRIPT_NAME: `/identity` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:17:14.413160 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4b1f60b-650e-48c5-9773-23c2631fd4ff None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:17:14.429179 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05015b70-411b-4202-9890-c4a2429dcc7e admin admin] Validating token access rules against request {{(pid=80378) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:17:14.436040 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17f6bf63-c28b-48c0-8dc9-a44bae4d1138 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '891005fa5ff74041b75747d16cc66798', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7047912c5acd4adeba5d87c3bf895e39', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80378) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:17:14.437092 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17f6bf63-c28b-48c0-8dc9-a44bae4d1138 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:17:14.437092 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17f6bf63-c28b-48c0-8dc9-a44bae4d1138 admin admin] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:17:14.437440 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17f6bf63-c28b-48c0-8dc9-a44bae4d1138 admin admin] PATH_INFO: `/v3/projects/08cc037744b14429898b57ad3fe90d0b` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:17:14.438662 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.auth.core [None req-e4b1f60b-650e-48c5-9773-23c2631fd4ff None None] MFA Rules not processed for user `891005fa5ff74041b75747d16cc66798`. Rule list: `[]` (Enabled: `True`). {{(pid=80377) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 02 10:17:14.438781 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17f6bf63-c28b-48c0-8dc9-a44bae4d1138 admin admin] RBAC: Authorizing `identity:delete_project(project_id=08cc037744b14429898b57ad3fe90d0b)` {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:17:14.440036 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.fernet_utils [None req-e4b1f60b-650e-48c5-9773-23c2631fd4ff None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80377) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 10:17:14.442367 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17f6bf63-c28b-48c0-8dc9-a44bae4d1138 admin admin] RBAC: Authorization granted {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:17:14.449565 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: [pid: 80377|app: 0|req: 1008/2007] 10.222.0.82 () {58 vars in 958 bytes} [Tue Aug 2 10:17:14 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 38 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 02 10:17:14.454508 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4b1f60b-650e-48c5-9773-23c2631fd4ff None None] Authenticating user token {{(pid=80377) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:17:14.476425 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4b1f60b-650e-48c5-9773-23c2631fd4ff None None] Validating token access rules against request {{(pid=80377) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:17:14.486445 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cabe6e54-5e3c-4ede-b7de-559aa80db8dc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '891005fa5ff74041b75747d16cc66798', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7047912c5acd4adeba5d87c3bf895e39', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80377) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:17:14.487814 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cabe6e54-5e3c-4ede-b7de-559aa80db8dc admin admin] REQUEST_METHOD: `GET` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:17:14.488178 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cabe6e54-5e3c-4ede-b7de-559aa80db8dc admin admin] SCRIPT_NAME: `/identity` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:17:14.488526 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cabe6e54-5e3c-4ede-b7de-559aa80db8dc admin admin] PATH_INFO: `/v3/domains` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:17:14.489749 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cabe6e54-5e3c-4ede-b7de-559aa80db8dc admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80377) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 02 10:17:14.490183 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cabe6e54-5e3c-4ede-b7de-559aa80db8dc admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:17:14.491959 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cabe6e54-5e3c-4ede-b7de-559aa80db8dc admin admin] RBAC: Authorization granted {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:17:14.511159 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.notifications [None req-17f6bf63-c28b-48c0-8dc9-a44bae4d1138 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '08cc037744b14429898b57ad3fe90d0b', 'request_id': 'req-17f6bf63-c28b-48c0-8dc9-a44bae4d1138'} {{(pid=80378) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 10:17:14.513768 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: [pid: 80377|app: 0|req: 1009/2008] 10.222.0.82 () {58 vars in 1153 bytes} [Tue Aug 2 10:17:14 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 60 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 02 10:17:14.517057 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cabe6e54-5e3c-4ede-b7de-559aa80db8dc admin admin] Authenticating user token {{(pid=80377) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:17:14.518830 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.notifications [None req-17f6bf63-c28b-48c0-8dc9-a44bae4d1138 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '08cc037744b14429898b57ad3fe90d0b', 'request_id': 'req-17f6bf63-c28b-48c0-8dc9-a44bae4d1138'} {{(pid=80378) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 10:17:14.533414 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cabe6e54-5e3c-4ede-b7de-559aa80db8dc admin admin] Validating token access rules against request {{(pid=80377) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:17:14.551560 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.notifications [None req-17f6bf63-c28b-48c0-8dc9-a44bae4d1138 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '08cc037744b14429898b57ad3fe90d0b', 'request_id': 'req-17f6bf63-c28b-48c0-8dc9-a44bae4d1138'} {{(pid=80378) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 10:17:14.574286 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.notifications [None req-17f6bf63-c28b-48c0-8dc9-a44bae4d1138 admin admin] The token cache is being invalidate because project 08cc037744b14429898b57ad3fe90d0b was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80378) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 02 10:17:14.574664 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.notifications [None req-17f6bf63-c28b-48c0-8dc9-a44bae4d1138 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80378) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 10:17:14.575863 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.notifications [None req-17f6bf63-c28b-48c0-8dc9-a44bae4d1138 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80378) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 10:17:14.583341 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 1000/2009] 10.222.0.82 () {58 vars in 1233 bytes} [Tue Aug 2 10:17:14 2022] DELETE /identity/v3/projects/08cc037744b14429898b57ad3fe90d0b => generated 0 bytes in 301 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 02 10:17:14.584706 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17f6bf63-c28b-48c0-8dc9-a44bae4d1138 admin admin] Authenticating user token {{(pid=80378) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:17:14.588738 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.fernet_utils [None req-17f6bf63-c28b-48c0-8dc9-a44bae4d1138 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80378) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 10:17:14.622188 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG dbcounter [-] [80377] Writing DB stats keystone:SELECT=100 {{(pid=80377) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 10:17:14.635122 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4740c98f-9eb6-4229-a459-df9c6a6cdfb5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '891005fa5ff74041b75747d16cc66798', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7047912c5acd4adeba5d87c3bf895e39', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80377) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:17:14.635776 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4740c98f-9eb6-4229-a459-df9c6a6cdfb5 admin admin] REQUEST_METHOD: `POST` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:17:14.636060 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4740c98f-9eb6-4229-a459-df9c6a6cdfb5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:17:14.636338 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4740c98f-9eb6-4229-a459-df9c6a6cdfb5 admin admin] PATH_INFO: `/v3/projects` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:17:14.637197 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4740c98f-9eb6-4229-a459-df9c6a6cdfb5 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:17:14.639830 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4740c98f-9eb6-4229-a459-df9c6a6cdfb5 admin admin] RBAC: Authorization granted {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:17:14.682894 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: [pid: 80377|app: 0|req: 1010/2010] 10.222.0.82 () {60 vars in 1153 bytes} [Tue Aug 2 10:17:14 2022] POST /identity/v3/projects => generated 385 bytes in 163 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 02 10:17:14.682894 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4740c98f-9eb6-4229-a459-df9c6a6cdfb5 admin admin] Authenticating user token {{(pid=80377) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:17:14.688024 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.fernet_utils [None req-4740c98f-9eb6-4229-a459-df9c6a6cdfb5 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80377) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 10:17:14.714700 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17f6bf63-c28b-48c0-8dc9-a44bae4d1138 admin admin] Validating token access rules against request {{(pid=80378) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:17:14.835554 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e644bef4-300f-450c-b65d-c72553154458 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '891005fa5ff74041b75747d16cc66798', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7047912c5acd4adeba5d87c3bf895e39', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80378) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:17:14.836026 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4740c98f-9eb6-4229-a459-df9c6a6cdfb5 admin admin] Validating token access rules against request {{(pid=80377) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:17:14.836209 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e644bef4-300f-450c-b65d-c72553154458 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:17:14.836499 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e644bef4-300f-450c-b65d-c72553154458 admin admin] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:17:14.836777 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e644bef4-300f-450c-b65d-c72553154458 admin admin] PATH_INFO: `/v3/users/92b55078f99e4e34b00305540703fa85` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:17:14.838115 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e644bef4-300f-450c-b65d-c72553154458 admin admin] RBAC: Authorizing `identity:delete_user(user_id=92b55078f99e4e34b00305540703fa85)` {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:17:14.842302 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e644bef4-300f-450c-b65d-c72553154458 admin admin] RBAC: Authorization granted {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:17:14.843736 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52258594-fd69-4e6a-b913-e07c72232bcf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '891005fa5ff74041b75747d16cc66798', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7047912c5acd4adeba5d87c3bf895e39', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80377) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:17:14.844606 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52258594-fd69-4e6a-b913-e07c72232bcf admin admin] REQUEST_METHOD: `DELETE` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:17:14.844898 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52258594-fd69-4e6a-b913-e07c72232bcf admin admin] SCRIPT_NAME: `/identity` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:17:14.849919 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52258594-fd69-4e6a-b913-e07c72232bcf admin admin] PATH_INFO: `/v3/projects/6eb831630f7c4594bbfa41ef86ae84df` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:17:14.852224 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52258594-fd69-4e6a-b913-e07c72232bcf admin admin] RBAC: Authorizing `identity:delete_project(project_id=6eb831630f7c4594bbfa41ef86ae84df)` {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:17:14.854330 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52258594-fd69-4e6a-b913-e07c72232bcf admin admin] RBAC: Authorization granted {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:17:14.889530 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG dbcounter [-] [80378] Writing DB stats keystone:SELECT=91,keystone:DELETE=8,keystone:INSERT=1 {{(pid=80378) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 10:17:14.960973 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.notifications [None req-52258594-fd69-4e6a-b913-e07c72232bcf admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '6eb831630f7c4594bbfa41ef86ae84df', 'request_id': 'req-52258594-fd69-4e6a-b913-e07c72232bcf'} {{(pid=80377) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 10:17:14.967340 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.notifications [None req-e644bef4-300f-450c-b65d-c72553154458 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '92b55078f99e4e34b00305540703fa85'} {{(pid=80378) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 10:17:14.970358 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.notifications [None req-52258594-fd69-4e6a-b913-e07c72232bcf admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '6eb831630f7c4594bbfa41ef86ae84df', 'request_id': 'req-52258594-fd69-4e6a-b913-e07c72232bcf'} {{(pid=80377) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 10:17:14.982232 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.notifications [None req-e644bef4-300f-450c-b65d-c72553154458 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '92b55078f99e4e34b00305540703fa85'} {{(pid=80378) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 10:17:14.985135 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.notifications [None req-52258594-fd69-4e6a-b913-e07c72232bcf admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '6eb831630f7c4594bbfa41ef86ae84df', 'request_id': 'req-52258594-fd69-4e6a-b913-e07c72232bcf'} {{(pid=80377) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 10:17:14.995283 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.notifications [None req-e644bef4-300f-450c-b65d-c72553154458 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '92b55078f99e4e34b00305540703fa85'} {{(pid=80378) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 10:17:14.996138 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.notifications [None req-e644bef4-300f-450c-b65d-c72553154458 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '92b55078f99e4e34b00305540703fa85'} {{(pid=80378) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 10:17:15.013873 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.notifications [None req-52258594-fd69-4e6a-b913-e07c72232bcf admin admin] The token cache is being invalidate because project 6eb831630f7c4594bbfa41ef86ae84df was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80377) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 02 10:17:15.014109 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.notifications [None req-52258594-fd69-4e6a-b913-e07c72232bcf admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80377) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 10:17:15.014803 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.notifications [None req-52258594-fd69-4e6a-b913-e07c72232bcf admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80377) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 10:17:15.015607 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.notifications [None req-e644bef4-300f-450c-b65d-c72553154458 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '92b55078f99e4e34b00305540703fa85'} {{(pid=80378) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 10:17:15.016484 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: [pid: 80377|app: 0|req: 1011/2011] 10.222.0.82 () {58 vars in 1233 bytes} [Tue Aug 2 10:17:14 2022] DELETE /identity/v3/projects/6eb831630f7c4594bbfa41ef86ae84df => generated 0 bytes in 336 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 02 10:17:15.017733 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52258594-fd69-4e6a-b913-e07c72232bcf admin admin] Authenticating user token {{(pid=80377) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:17:15.019945 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.fernet_utils [None req-52258594-fd69-4e6a-b913-e07c72232bcf admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80377) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 10:17:15.024300 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 1001/2012] 10.222.0.82 () {58 vars in 1224 bytes} [Tue Aug 2 10:17:14 2022] DELETE /identity/v3/users/92b55078f99e4e34b00305540703fa85 => generated 0 bytes in 440 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 02 10:17:15.025585 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e644bef4-300f-450c-b65d-c72553154458 admin admin] Authenticating user token {{(pid=80378) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:17:15.029138 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.fernet_utils [None req-e644bef4-300f-450c-b65d-c72553154458 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80378) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 10:17:15.118742 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e644bef4-300f-450c-b65d-c72553154458 admin admin] Validating token access rules against request {{(pid=80378) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:17:15.146118 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52258594-fd69-4e6a-b913-e07c72232bcf admin admin] Validating token access rules against request {{(pid=80377) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:17:15.146499 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1309c6f8-6d57-4c38-8033-355c3cd424e5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '891005fa5ff74041b75747d16cc66798', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7047912c5acd4adeba5d87c3bf895e39', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80378) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:17:15.146499 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1309c6f8-6d57-4c38-8033-355c3cd424e5 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:17:15.146671 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43d224d3-44d7-42ae-8c75-dea1dc66d41b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '891005fa5ff74041b75747d16cc66798', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7047912c5acd4adeba5d87c3bf895e39', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80377) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:17:15.146763 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1309c6f8-6d57-4c38-8033-355c3cd424e5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:17:15.146763 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1309c6f8-6d57-4c38-8033-355c3cd424e5 admin admin] PATH_INFO: `/v3/users/4fa9d12062364181a53ac83f9f700a33` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:17:15.146904 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43d224d3-44d7-42ae-8c75-dea1dc66d41b admin admin] REQUEST_METHOD: `POST` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:17:15.146904 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43d224d3-44d7-42ae-8c75-dea1dc66d41b admin admin] SCRIPT_NAME: `/identity` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:17:15.146904 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43d224d3-44d7-42ae-8c75-dea1dc66d41b admin admin] PATH_INFO: `/v3/users` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:17:15.146904 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43d224d3-44d7-42ae-8c75-dea1dc66d41b admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:17:15.147181 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1309c6f8-6d57-4c38-8033-355c3cd424e5 admin admin] RBAC: Authorizing `identity:delete_user(user_id=4fa9d12062364181a53ac83f9f700a33)` {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:17:15.147266 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43d224d3-44d7-42ae-8c75-dea1dc66d41b admin admin] RBAC: Authorization granted {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:17:15.147344 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1309c6f8-6d57-4c38-8033-355c3cd424e5 admin admin] RBAC: Authorization granted {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:17:15.186519 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG dbcounter [-] [80377] Writing DB stats keystone:SELECT=88,keystone:INSERT=5,keystone:DELETE=7 {{(pid=80377) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 10:17:15.195048 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: [pid: 80377|app: 0|req: 1012/2013] 10.222.0.82 () {60 vars in 1144 bytes} [Tue Aug 2 10:17:15 2022] POST /identity/v3/users => generated 293 bytes in 178 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 02 10:17:15.197290 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43d224d3-44d7-42ae-8c75-dea1dc66d41b admin admin] Authenticating user token {{(pid=80377) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:17:15.211507 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43d224d3-44d7-42ae-8c75-dea1dc66d41b admin admin] Validating token access rules against request {{(pid=80377) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:17:15.220693 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85b914bb-fbe9-451a-897f-a17fe6bc2771 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '891005fa5ff74041b75747d16cc66798', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7047912c5acd4adeba5d87c3bf895e39', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80377) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:17:15.222209 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85b914bb-fbe9-451a-897f-a17fe6bc2771 admin admin] REQUEST_METHOD: `GET` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:17:15.222564 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85b914bb-fbe9-451a-897f-a17fe6bc2771 admin admin] SCRIPT_NAME: `/identity` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:17:15.222911 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85b914bb-fbe9-451a-897f-a17fe6bc2771 admin admin] PATH_INFO: `/v3/roles` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:17:15.223805 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85b914bb-fbe9-451a-897f-a17fe6bc2771 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:17:15.225593 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85b914bb-fbe9-451a-897f-a17fe6bc2771 admin admin] RBAC: Authorization granted {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:17:15.241511 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: [pid: 80377|app: 0|req: 1013/2014] 10.222.0.82 () {58 vars in 1122 bytes} [Tue Aug 2 10:17:15 2022] GET /identity/v3/roles => generated 1420 bytes in 45 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 02 10:17:15.245848 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85b914bb-fbe9-451a-897f-a17fe6bc2771 admin admin] Authenticating user token {{(pid=80377) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:17:15.246894 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.notifications [None req-1309c6f8-6d57-4c38-8033-355c3cd424e5 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '4fa9d12062364181a53ac83f9f700a33'} {{(pid=80378) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 10:17:15.257221 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.notifications [None req-1309c6f8-6d57-4c38-8033-355c3cd424e5 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '4fa9d12062364181a53ac83f9f700a33'} {{(pid=80378) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 10:17:15.260650 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85b914bb-fbe9-451a-897f-a17fe6bc2771 admin admin] Validating token access rules against request {{(pid=80377) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:17:15.267017 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-670a739e-6ab7-47cd-a679-6ddfc0f322bc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '891005fa5ff74041b75747d16cc66798', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7047912c5acd4adeba5d87c3bf895e39', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80377) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:17:15.267568 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-670a739e-6ab7-47cd-a679-6ddfc0f322bc admin admin] REQUEST_METHOD: `GET` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:17:15.267856 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-670a739e-6ab7-47cd-a679-6ddfc0f322bc admin admin] SCRIPT_NAME: `/identity` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:17:15.267984 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.notifications [None req-1309c6f8-6d57-4c38-8033-355c3cd424e5 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '4fa9d12062364181a53ac83f9f700a33'} {{(pid=80378) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 10:17:15.268117 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-670a739e-6ab7-47cd-a679-6ddfc0f322bc admin admin] PATH_INFO: `/v3/roles` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:17:15.268939 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-670a739e-6ab7-47cd-a679-6ddfc0f322bc admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:17:15.269141 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.notifications [None req-1309c6f8-6d57-4c38-8033-355c3cd424e5 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '4fa9d12062364181a53ac83f9f700a33'} {{(pid=80378) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 10:17:15.270885 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-670a739e-6ab7-47cd-a679-6ddfc0f322bc admin admin] RBAC: Authorization granted {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:17:15.291001 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: [pid: 80377|app: 0|req: 1014/2015] 10.222.0.82 () {58 vars in 1122 bytes} [Tue Aug 2 10:17:15 2022] GET /identity/v3/roles => generated 1420 bytes in 43 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 02 10:17:15.291231 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.notifications [None req-1309c6f8-6d57-4c38-8033-355c3cd424e5 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '4fa9d12062364181a53ac83f9f700a33'} {{(pid=80378) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 10:17:15.293925 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-670a739e-6ab7-47cd-a679-6ddfc0f322bc admin admin] Authenticating user token {{(pid=80377) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:17:15.296049 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.fernet_utils [None req-670a739e-6ab7-47cd-a679-6ddfc0f322bc admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80377) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 10:17:15.302893 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 1002/2016] 10.222.0.82 () {58 vars in 1224 bytes} [Tue Aug 2 10:17:15 2022] DELETE /identity/v3/users/4fa9d12062364181a53ac83f9f700a33 => generated 0 bytes in 278 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 02 10:17:15.319825 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-670a739e-6ab7-47cd-a679-6ddfc0f322bc admin admin] Validating token access rules against request {{(pid=80377) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:17:15.402827 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-156dcba8-6ffe-4d38-9122-8bb70b6dff79 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '891005fa5ff74041b75747d16cc66798', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7047912c5acd4adeba5d87c3bf895e39', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80377) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:17:15.404296 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-156dcba8-6ffe-4d38-9122-8bb70b6dff79 admin admin] REQUEST_METHOD: `PUT` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:17:15.404441 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-156dcba8-6ffe-4d38-9122-8bb70b6dff79 admin admin] SCRIPT_NAME: `/identity` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:17:15.405049 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-156dcba8-6ffe-4d38-9122-8bb70b6dff79 admin admin] PATH_INFO: `/v3/projects/bdbf0111b2d84f12b205134fde982ae4/users/54bb78d9af854abf845b47f2bb925b74/roles/e1c04b19fc4543b8b54f58bbcffbc598` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:17:15.432031 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-156dcba8-6ffe-4d38-9122-8bb70b6dff79 admin admin] RBAC: Authorizing `identity:create_grant(project_id=bdbf0111b2d84f12b205134fde982ae4, user_id=54bb78d9af854abf845b47f2bb925b74, role_id=e1c04b19fc4543b8b54f58bbcffbc598)` {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:17:15.435506 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-156dcba8-6ffe-4d38-9122-8bb70b6dff79 admin admin] RBAC: Authorization granted {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:17:15.449764 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: [pid: 80377|app: 0|req: 1015/2017] 10.222.0.82 () {60 vars in 1483 bytes} [Tue Aug 2 10:17:15 2022] PUT /identity/v3/projects/bdbf0111b2d84f12b205134fde982ae4/users/54bb78d9af854abf845b47f2bb925b74/roles/e1c04b19fc4543b8b54f58bbcffbc598 => generated 0 bytes in 158 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 02 10:17:15.456263 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-481b711b-9318-4c5b-9dbe-3c60563766e7 None None] REQUEST_METHOD: `POST` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:17:15.456263 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-481b711b-9318-4c5b-9dbe-3c60563766e7 None None] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:17:15.456263 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-481b711b-9318-4c5b-9dbe-3c60563766e7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:17:15.468963 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-156dcba8-6ffe-4d38-9122-8bb70b6dff79 admin admin] Authenticating user token {{(pid=80377) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:17:15.471694 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.fernet_utils [None req-156dcba8-6ffe-4d38-9122-8bb70b6dff79 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80377) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 10:17:15.477223 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.auth.core [None req-481b711b-9318-4c5b-9dbe-3c60563766e7 None None] MFA Rules not processed for user `891005fa5ff74041b75747d16cc66798`. Rule list: `[]` (Enabled: `True`). {{(pid=80378) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 02 10:17:15.478821 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.fernet_utils [None req-481b711b-9318-4c5b-9dbe-3c60563766e7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80378) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 10:17:15.487734 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG dbcounter [-] [80378] Writing DB stats keystone:DELETE=19,keystone:SELECT=79,keystone:INSERT=2 {{(pid=80378) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 10:17:15.564263 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-156dcba8-6ffe-4d38-9122-8bb70b6dff79 admin admin] Validating token access rules against request {{(pid=80377) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:17:15.568129 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 1003/2018] 10.222.0.82 () {58 vars in 958 bytes} [Tue Aug 2 10:17:15 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 111 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 02 10:17:15.570364 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d65d51e2-a1cc-435f-aa54-e91634d795a7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '891005fa5ff74041b75747d16cc66798', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7047912c5acd4adeba5d87c3bf895e39', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80377) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:17:15.571153 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d65d51e2-a1cc-435f-aa54-e91634d795a7 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:17:15.571533 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d65d51e2-a1cc-435f-aa54-e91634d795a7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:17:15.571909 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d65d51e2-a1cc-435f-aa54-e91634d795a7 admin admin] PATH_INFO: `/v3/projects/f7f58addc65446bdb71511d824001610` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:17:15.573204 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d65d51e2-a1cc-435f-aa54-e91634d795a7 admin admin] RBAC: Authorizing `identity:delete_project(project_id=f7f58addc65446bdb71511d824001610)` {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:17:15.575271 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d65d51e2-a1cc-435f-aa54-e91634d795a7 admin admin] RBAC: Authorization granted {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:17:15.578050 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-481b711b-9318-4c5b-9dbe-3c60563766e7 None None] Authenticating user token {{(pid=80378) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:17:15.579466 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.fernet_utils [None req-481b711b-9318-4c5b-9dbe-3c60563766e7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80378) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 10:17:15.627209 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.notifications [None req-d65d51e2-a1cc-435f-aa54-e91634d795a7 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'f7f58addc65446bdb71511d824001610', 'request_id': 'req-d65d51e2-a1cc-435f-aa54-e91634d795a7'} {{(pid=80377) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 10:17:15.636971 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.notifications [None req-d65d51e2-a1cc-435f-aa54-e91634d795a7 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'f7f58addc65446bdb71511d824001610', 'request_id': 'req-d65d51e2-a1cc-435f-aa54-e91634d795a7'} {{(pid=80377) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 10:17:15.642738 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG dbcounter [-] [80377] Writing DB stats keystone:SELECT=93,keystone:INSERT=1,keystone:DELETE=6 {{(pid=80377) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 10:17:15.647890 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.notifications [None req-d65d51e2-a1cc-435f-aa54-e91634d795a7 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'f7f58addc65446bdb71511d824001610', 'request_id': 'req-d65d51e2-a1cc-435f-aa54-e91634d795a7'} {{(pid=80377) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 10:17:15.648079 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-481b711b-9318-4c5b-9dbe-3c60563766e7 None None] Validating token access rules against request {{(pid=80378) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:17:15.664682 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.notifications [None req-d65d51e2-a1cc-435f-aa54-e91634d795a7 admin admin] The token cache is being invalidate because project f7f58addc65446bdb71511d824001610 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80377) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 02 10:17:15.664982 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.notifications [None req-d65d51e2-a1cc-435f-aa54-e91634d795a7 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80377) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 10:17:15.665701 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.notifications [None req-d65d51e2-a1cc-435f-aa54-e91634d795a7 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80377) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 10:17:15.667505 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: [pid: 80377|app: 0|req: 1016/2019] 10.222.0.82 () {58 vars in 1233 bytes} [Tue Aug 2 10:17:15 2022] DELETE /identity/v3/projects/f7f58addc65446bdb71511d824001610 => generated 0 bytes in 199 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 02 10:17:15.668281 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d65d51e2-a1cc-435f-aa54-e91634d795a7 admin admin] Authenticating user token {{(pid=80377) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:17:15.669685 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.fernet_utils [None req-d65d51e2-a1cc-435f-aa54-e91634d795a7 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80377) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 10:17:15.711716 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-533820b3-d59a-4202-a539-e65adf40e703 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '33b87170f0634b67bce60750847b5cbe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f1b38ebb804c41c69772c998aae72c41', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80378) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:17:15.712452 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-533820b3-d59a-4202-a539-e65adf40e703 service neutron] REQUEST_METHOD: `GET` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:17:15.712763 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-533820b3-d59a-4202-a539-e65adf40e703 service neutron] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:17:15.713041 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-533820b3-d59a-4202-a539-e65adf40e703 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:17:15.714957 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.fernet_utils [None req-533820b3-d59a-4202-a539-e65adf40e703 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80378) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 10:17:15.750476 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d65d51e2-a1cc-435f-aa54-e91634d795a7 admin admin] Validating token access rules against request {{(pid=80377) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:17:15.756322 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e651eef9-d164-4422-8e25-14515eea3215 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '891005fa5ff74041b75747d16cc66798', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7047912c5acd4adeba5d87c3bf895e39', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80377) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:17:15.756996 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e651eef9-d164-4422-8e25-14515eea3215 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:17:15.757708 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e651eef9-d164-4422-8e25-14515eea3215 admin admin] SCRIPT_NAME: `/identity` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:17:15.758028 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e651eef9-d164-4422-8e25-14515eea3215 admin admin] PATH_INFO: `/v3/projects/84e78936aca74b67b7fd11f28a67c947` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:17:15.760198 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e651eef9-d164-4422-8e25-14515eea3215 admin admin] RBAC: Authorizing `identity:delete_project(project_id=84e78936aca74b67b7fd11f28a67c947)` {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:17:15.761625 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e651eef9-d164-4422-8e25-14515eea3215 admin admin] RBAC: Authorization granted {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:17:15.787850 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-533820b3-d59a-4202-a539-e65adf40e703 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:17:15.789295 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-533820b3-d59a-4202-a539-e65adf40e703 service neutron] RBAC: Authorization granted {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:17:15.802886 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 1004/2020] 10.222.0.82 () {60 vars in 1365 bytes} [Tue Aug 2 10:17:15 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 226 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 02 10:17:15.804520 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-160258dd-8db4-4c06-a61c-5118499ec369 None None] REQUEST_METHOD: `POST` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:17:15.804867 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-160258dd-8db4-4c06-a61c-5118499ec369 None None] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:17:15.805242 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-160258dd-8db4-4c06-a61c-5118499ec369 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:17:15.817742 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.notifications [None req-e651eef9-d164-4422-8e25-14515eea3215 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '84e78936aca74b67b7fd11f28a67c947', 'request_id': 'req-e651eef9-d164-4422-8e25-14515eea3215'} {{(pid=80377) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 10:17:15.828685 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.notifications [None req-e651eef9-d164-4422-8e25-14515eea3215 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '84e78936aca74b67b7fd11f28a67c947', 'request_id': 'req-e651eef9-d164-4422-8e25-14515eea3215'} {{(pid=80377) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 10:17:15.828859 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.auth.core [None req-160258dd-8db4-4c06-a61c-5118499ec369 None None] MFA Rules not processed for user `891005fa5ff74041b75747d16cc66798`. Rule list: `[]` (Enabled: `True`). {{(pid=80378) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 02 10:17:15.830502 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.fernet_utils [None req-160258dd-8db4-4c06-a61c-5118499ec369 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80378) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 10:17:15.836222 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.notifications [None req-e651eef9-d164-4422-8e25-14515eea3215 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '84e78936aca74b67b7fd11f28a67c947', 'request_id': 'req-e651eef9-d164-4422-8e25-14515eea3215'} {{(pid=80377) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 10:17:15.851033 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG dbcounter [-] [80378] Writing DB stats keystone:SELECT=100 {{(pid=80378) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 10:17:15.861132 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.notifications [None req-e651eef9-d164-4422-8e25-14515eea3215 admin admin] The token cache is being invalidate because project 84e78936aca74b67b7fd11f28a67c947 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80377) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 02 10:17:15.861601 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.notifications [None req-e651eef9-d164-4422-8e25-14515eea3215 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80377) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 10:17:15.863116 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.notifications [None req-e651eef9-d164-4422-8e25-14515eea3215 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80377) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 10:17:15.865505 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: [pid: 80377|app: 0|req: 1017/2021] 10.222.0.82 () {58 vars in 1233 bytes} [Tue Aug 2 10:17:15 2022] DELETE /identity/v3/projects/84e78936aca74b67b7fd11f28a67c947 => generated 0 bytes in 197 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 02 10:17:15.870779 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e978d8e-84c9-4223-b849-0aa2e74c1ff4 None None] REQUEST_METHOD: `POST` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:17:15.871151 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e978d8e-84c9-4223-b849-0aa2e74c1ff4 None None] SCRIPT_NAME: `/identity` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:17:15.871329 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e978d8e-84c9-4223-b849-0aa2e74c1ff4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:17:15.898677 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.auth.core [None req-9e978d8e-84c9-4223-b849-0aa2e74c1ff4 None None] MFA Rules not processed for user `891005fa5ff74041b75747d16cc66798`. Rule list: `[]` (Enabled: `True`). {{(pid=80377) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 02 10:17:15.899959 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.fernet_utils [None req-9e978d8e-84c9-4223-b849-0aa2e74c1ff4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80377) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 10:17:15.916560 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 1005/2022] 10.222.0.82 () {58 vars in 958 bytes} [Tue Aug 2 10:17:15 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 113 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 02 10:17:15.924867 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-963a271f-9601-4ff6-aa12-4519d99a366b None None] REQUEST_METHOD: `POST` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:17:15.925183 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-963a271f-9601-4ff6-aa12-4519d99a366b None None] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:17:15.925461 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-963a271f-9601-4ff6-aa12-4519d99a366b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:17:15.952479 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.auth.core [None req-963a271f-9601-4ff6-aa12-4519d99a366b None None] MFA Rules not processed for user `891005fa5ff74041b75747d16cc66798`. Rule list: `[]` (Enabled: `True`). {{(pid=80378) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 02 10:17:15.953871 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.fernet_utils [None req-963a271f-9601-4ff6-aa12-4519d99a366b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80378) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 10:17:15.965059 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 1006/2023] 10.222.0.82 () {58 vars in 958 bytes} [Tue Aug 2 10:17:15 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 41 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 02 10:17:15.971420 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-963a271f-9601-4ff6-aa12-4519d99a366b None None] Authenticating user token {{(pid=80378) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:17:15.987598 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: [pid: 80377|app: 0|req: 1018/2024] 10.222.0.82 () {58 vars in 958 bytes} [Tue Aug 2 10:17:15 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 118 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 02 10:17:15.991932 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-963a271f-9601-4ff6-aa12-4519d99a366b None None] Validating token access rules against request {{(pid=80378) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:17:15.996033 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d03d457-9f21-44d4-a065-afdfbc5d5286 None None] REQUEST_METHOD: `POST` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:17:15.996325 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d03d457-9f21-44d4-a065-afdfbc5d5286 None None] SCRIPT_NAME: `/identity` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:17:15.996641 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d03d457-9f21-44d4-a065-afdfbc5d5286 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:17:16.002060 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da8549db-45ac-4360-bd2f-ccd4c185fdae admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '891005fa5ff74041b75747d16cc66798', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7047912c5acd4adeba5d87c3bf895e39', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80378) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:17:16.003100 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da8549db-45ac-4360-bd2f-ccd4c185fdae admin admin] REQUEST_METHOD: `GET` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:17:16.003743 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da8549db-45ac-4360-bd2f-ccd4c185fdae admin admin] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:17:16.004225 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da8549db-45ac-4360-bd2f-ccd4c185fdae admin admin] PATH_INFO: `/v3/domains` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:17:16.006141 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da8549db-45ac-4360-bd2f-ccd4c185fdae admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80378) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 02 10:17:16.006733 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da8549db-45ac-4360-bd2f-ccd4c185fdae admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:17:16.008544 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da8549db-45ac-4360-bd2f-ccd4c185fdae admin admin] RBAC: Authorization granted {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:17:16.019067 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.auth.core [None req-2d03d457-9f21-44d4-a065-afdfbc5d5286 None None] MFA Rules not processed for user `891005fa5ff74041b75747d16cc66798`. Rule list: `[]` (Enabled: `True`). {{(pid=80377) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 02 10:17:16.020307 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.fernet_utils [None req-2d03d457-9f21-44d4-a065-afdfbc5d5286 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80377) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 10:17:16.022881 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 1007/2025] 10.222.0.82 () {58 vars in 1153 bytes} [Tue Aug 2 10:17:15 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 52 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 02 10:17:16.027271 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da8549db-45ac-4360-bd2f-ccd4c185fdae admin admin] Authenticating user token {{(pid=80378) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:17:16.029302 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: [pid: 80377|app: 0|req: 1019/2026] 10.222.0.82 () {58 vars in 958 bytes} [Tue Aug 2 10:17:15 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 34 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 02 10:17:16.033135 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d03d457-9f21-44d4-a065-afdfbc5d5286 None None] Authenticating user token {{(pid=80377) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:17:16.039964 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da8549db-45ac-4360-bd2f-ccd4c185fdae admin admin] Validating token access rules against request {{(pid=80378) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:17:16.042596 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d03d457-9f21-44d4-a065-afdfbc5d5286 None None] Validating token access rules against request {{(pid=80377) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:17:16.045927 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e1007da-52ef-4536-b52a-36d3ac1c20c4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '891005fa5ff74041b75747d16cc66798', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7047912c5acd4adeba5d87c3bf895e39', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80378) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:17:16.046556 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e1007da-52ef-4536-b52a-36d3ac1c20c4 admin admin] REQUEST_METHOD: `POST` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:17:16.046840 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e1007da-52ef-4536-b52a-36d3ac1c20c4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:17:16.047116 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e1007da-52ef-4536-b52a-36d3ac1c20c4 admin admin] PATH_INFO: `/v3/projects` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:17:16.047798 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e1007da-52ef-4536-b52a-36d3ac1c20c4 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:17:16.048291 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d5a79b2-0a58-4fd7-a0f0-746b9c600a25 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '891005fa5ff74041b75747d16cc66798', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7047912c5acd4adeba5d87c3bf895e39', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80377) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:17:16.048976 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d5a79b2-0a58-4fd7-a0f0-746b9c600a25 admin admin] REQUEST_METHOD: `GET` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:17:16.049340 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d5a79b2-0a58-4fd7-a0f0-746b9c600a25 admin admin] SCRIPT_NAME: `/identity` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:17:16.049680 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d5a79b2-0a58-4fd7-a0f0-746b9c600a25 admin admin] PATH_INFO: `/v3/domains` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:17:16.049802 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e1007da-52ef-4536-b52a-36d3ac1c20c4 admin admin] RBAC: Authorization granted {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:17:16.050509 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d5a79b2-0a58-4fd7-a0f0-746b9c600a25 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80377) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 02 10:17:16.050872 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d5a79b2-0a58-4fd7-a0f0-746b9c600a25 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:17:16.052510 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d5a79b2-0a58-4fd7-a0f0-746b9c600a25 admin admin] RBAC: Authorization granted {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:17:16.063866 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG dbcounter [-] [80377] Writing DB stats keystone:SELECT=90,keystone:INSERT=2,keystone:DELETE=8 {{(pid=80377) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 10:17:16.068174 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: [pid: 80377|app: 0|req: 1020/2027] 10.222.0.82 () {58 vars in 1153 bytes} [Tue Aug 2 10:17:16 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 02 10:17:16.072362 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d5a79b2-0a58-4fd7-a0f0-746b9c600a25 admin admin] Authenticating user token {{(pid=80377) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:17:16.084344 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d5a79b2-0a58-4fd7-a0f0-746b9c600a25 admin admin] Validating token access rules against request {{(pid=80377) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:17:16.091530 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 1008/2028] 10.222.0.82 () {60 vars in 1153 bytes} [Tue Aug 2 10:17:16 2022] POST /identity/v3/projects => generated 379 bytes in 62 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 02 10:17:16.092817 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e1007da-52ef-4536-b52a-36d3ac1c20c4 admin admin] Authenticating user token {{(pid=80378) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:17:16.094395 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e29b6a5f-267f-49ce-88f9-d53d3677643f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '891005fa5ff74041b75747d16cc66798', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7047912c5acd4adeba5d87c3bf895e39', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80377) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:17:16.095102 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e29b6a5f-267f-49ce-88f9-d53d3677643f admin admin] REQUEST_METHOD: `POST` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:17:16.095458 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e29b6a5f-267f-49ce-88f9-d53d3677643f admin admin] SCRIPT_NAME: `/identity` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:17:16.095803 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e29b6a5f-267f-49ce-88f9-d53d3677643f admin admin] PATH_INFO: `/v3/projects` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:17:16.096558 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e29b6a5f-267f-49ce-88f9-d53d3677643f admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:17:16.098647 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e29b6a5f-267f-49ce-88f9-d53d3677643f admin admin] RBAC: Authorization granted {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:17:16.132135 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: [pid: 80377|app: 0|req: 1021/2029] 10.222.0.82 () {60 vars in 1153 bytes} [Tue Aug 2 10:17:16 2022] POST /identity/v3/projects => generated 383 bytes in 60 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 02 10:17:16.136628 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e29b6a5f-267f-49ce-88f9-d53d3677643f admin admin] Authenticating user token {{(pid=80377) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:17:16.169697 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e1007da-52ef-4536-b52a-36d3ac1c20c4 admin admin] Validating token access rules against request {{(pid=80378) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:17:16.232413 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e29b6a5f-267f-49ce-88f9-d53d3677643f admin admin] Validating token access rules against request {{(pid=80377) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:17:16.242366 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d44c0a3-c1dc-4d64-bed9-12c9f9057987 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '891005fa5ff74041b75747d16cc66798', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7047912c5acd4adeba5d87c3bf895e39', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80377) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:17:16.242366 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d44c0a3-c1dc-4d64-bed9-12c9f9057987 admin admin] REQUEST_METHOD: `POST` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:17:16.242720 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d44c0a3-c1dc-4d64-bed9-12c9f9057987 admin admin] SCRIPT_NAME: `/identity` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:17:16.242956 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d44c0a3-c1dc-4d64-bed9-12c9f9057987 admin admin] PATH_INFO: `/v3/users` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:17:16.243666 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d44c0a3-c1dc-4d64-bed9-12c9f9057987 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:17:16.245103 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-128cfadd-b959-4ba9-818b-6e73ff2c441f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '891005fa5ff74041b75747d16cc66798', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7047912c5acd4adeba5d87c3bf895e39', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80378) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:17:16.246036 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d44c0a3-c1dc-4d64-bed9-12c9f9057987 admin admin] RBAC: Authorization granted {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:17:16.246250 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-128cfadd-b959-4ba9-818b-6e73ff2c441f admin admin] REQUEST_METHOD: `POST` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:17:16.246546 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-128cfadd-b959-4ba9-818b-6e73ff2c441f admin admin] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:17:16.246919 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-128cfadd-b959-4ba9-818b-6e73ff2c441f admin admin] PATH_INFO: `/v3/users` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:17:16.248323 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-128cfadd-b959-4ba9-818b-6e73ff2c441f admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:17:16.250819 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-128cfadd-b959-4ba9-818b-6e73ff2c441f admin admin] RBAC: Authorization granted {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:17:16.290544 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: [pid: 80377|app: 0|req: 1022/2030] 10.222.0.82 () {60 vars in 1144 bytes} [Tue Aug 2 10:17:16 2022] POST /identity/v3/users => generated 292 bytes in 154 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 02 10:17:16.294278 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d44c0a3-c1dc-4d64-bed9-12c9f9057987 admin admin] Authenticating user token {{(pid=80377) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:17:16.309571 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 1009/2031] 10.222.0.82 () {60 vars in 1144 bytes} [Tue Aug 2 10:17:16 2022] POST /identity/v3/users => generated 290 bytes in 217 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 02 10:17:16.311135 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-128cfadd-b959-4ba9-818b-6e73ff2c441f admin admin] Authenticating user token {{(pid=80378) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:17:16.311574 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d44c0a3-c1dc-4d64-bed9-12c9f9057987 admin admin] Validating token access rules against request {{(pid=80377) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:17:16.319229 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7874b3f-e5e0-49b7-b35d-a8e293cfd757 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '891005fa5ff74041b75747d16cc66798', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7047912c5acd4adeba5d87c3bf895e39', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80377) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:17:16.320915 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7874b3f-e5e0-49b7-b35d-a8e293cfd757 admin admin] REQUEST_METHOD: `GET` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:17:16.320915 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7874b3f-e5e0-49b7-b35d-a8e293cfd757 admin admin] SCRIPT_NAME: `/identity` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:17:16.321332 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7874b3f-e5e0-49b7-b35d-a8e293cfd757 admin admin] PATH_INFO: `/v3/roles` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:17:16.323897 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7874b3f-e5e0-49b7-b35d-a8e293cfd757 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:17:16.325213 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-128cfadd-b959-4ba9-818b-6e73ff2c441f admin admin] Validating token access rules against request {{(pid=80378) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:17:16.326781 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7874b3f-e5e0-49b7-b35d-a8e293cfd757 admin admin] RBAC: Authorization granted {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:17:16.330925 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2598552-020c-4b73-8330-d2074c5894ce admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '891005fa5ff74041b75747d16cc66798', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7047912c5acd4adeba5d87c3bf895e39', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80378) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:17:16.332173 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2598552-020c-4b73-8330-d2074c5894ce admin admin] REQUEST_METHOD: `GET` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:17:16.332173 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2598552-020c-4b73-8330-d2074c5894ce admin admin] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:17:16.332173 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2598552-020c-4b73-8330-d2074c5894ce admin admin] PATH_INFO: `/v3/roles` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:17:16.332913 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2598552-020c-4b73-8330-d2074c5894ce admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:17:16.335272 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2598552-020c-4b73-8330-d2074c5894ce admin admin] RBAC: Authorization granted {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:17:16.345982 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: [pid: 80377|app: 0|req: 1023/2032] 10.222.0.82 () {58 vars in 1122 bytes} [Tue Aug 2 10:17:16 2022] GET /identity/v3/roles => generated 1420 bytes in 52 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 02 10:17:16.349482 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7874b3f-e5e0-49b7-b35d-a8e293cfd757 admin admin] Authenticating user token {{(pid=80377) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:17:16.352365 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 1010/2033] 10.222.0.82 () {58 vars in 1122 bytes} [Tue Aug 2 10:17:16 2022] GET /identity/v3/roles => generated 1420 bytes in 41 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 02 10:17:16.355311 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2598552-020c-4b73-8330-d2074c5894ce admin admin] Authenticating user token {{(pid=80378) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:17:16.362768 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG dbcounter [-] [80378] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=80378) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 10:17:16.363866 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7874b3f-e5e0-49b7-b35d-a8e293cfd757 admin admin] Validating token access rules against request {{(pid=80377) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:17:16.367619 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2598552-020c-4b73-8330-d2074c5894ce admin admin] Validating token access rules against request {{(pid=80378) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:17:16.370916 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45df9e04-ded7-4797-84e9-7029aa8538bc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '891005fa5ff74041b75747d16cc66798', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7047912c5acd4adeba5d87c3bf895e39', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80377) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:17:16.371737 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45df9e04-ded7-4797-84e9-7029aa8538bc admin admin] REQUEST_METHOD: `GET` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:17:16.372122 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45df9e04-ded7-4797-84e9-7029aa8538bc admin admin] SCRIPT_NAME: `/identity` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:17:16.372519 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45df9e04-ded7-4797-84e9-7029aa8538bc admin admin] PATH_INFO: `/v3/roles` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:17:16.373323 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45df9e04-ded7-4797-84e9-7029aa8538bc admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:17:16.374048 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6409db4-8544-4f13-9108-bf360b2c012d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '891005fa5ff74041b75747d16cc66798', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7047912c5acd4adeba5d87c3bf895e39', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80378) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:17:16.374837 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6409db4-8544-4f13-9108-bf360b2c012d admin admin] REQUEST_METHOD: `GET` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:17:16.375301 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6409db4-8544-4f13-9108-bf360b2c012d admin admin] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:17:16.375992 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6409db4-8544-4f13-9108-bf360b2c012d admin admin] PATH_INFO: `/v3/roles` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:17:16.376422 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45df9e04-ded7-4797-84e9-7029aa8538bc admin admin] RBAC: Authorization granted {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:17:16.377028 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6409db4-8544-4f13-9108-bf360b2c012d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:17:16.379077 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6409db4-8544-4f13-9108-bf360b2c012d admin admin] RBAC: Authorization granted {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:17:16.397093 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 1011/2034] 10.222.0.82 () {58 vars in 1122 bytes} [Tue Aug 2 10:17:16 2022] GET /identity/v3/roles => generated 1420 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 02 10:17:16.397294 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: [pid: 80377|app: 0|req: 1024/2035] 10.222.0.82 () {58 vars in 1122 bytes} [Tue Aug 2 10:17:16 2022] GET /identity/v3/roles => generated 1420 bytes in 45 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 02 10:17:16.397294 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45df9e04-ded7-4797-84e9-7029aa8538bc admin admin] Authenticating user token {{(pid=80377) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:17:16.398619 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6409db4-8544-4f13-9108-bf360b2c012d admin admin] Authenticating user token {{(pid=80378) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:17:16.409489 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45df9e04-ded7-4797-84e9-7029aa8538bc admin admin] Validating token access rules against request {{(pid=80377) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:17:16.410691 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6409db4-8544-4f13-9108-bf360b2c012d admin admin] Validating token access rules against request {{(pid=80378) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:17:16.414842 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6545b348-4687-4083-92d3-7f285e0b552b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '891005fa5ff74041b75747d16cc66798', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7047912c5acd4adeba5d87c3bf895e39', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80377) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:17:16.415903 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6545b348-4687-4083-92d3-7f285e0b552b admin admin] REQUEST_METHOD: `PUT` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:17:16.416092 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92abff67-bd03-4603-9350-36fabaf31eef admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '891005fa5ff74041b75747d16cc66798', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7047912c5acd4adeba5d87c3bf895e39', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80378) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:17:16.416778 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92abff67-bd03-4603-9350-36fabaf31eef admin admin] REQUEST_METHOD: `PUT` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:17:16.417151 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92abff67-bd03-4603-9350-36fabaf31eef admin admin] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:17:16.417255 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6545b348-4687-4083-92d3-7f285e0b552b admin admin] SCRIPT_NAME: `/identity` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:17:16.417340 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92abff67-bd03-4603-9350-36fabaf31eef admin admin] PATH_INFO: `/v3/projects/c245a79031d74d889e50a87f958a7585/users/4b30c102907740d8bad46dace08d4ef2/roles/e1c04b19fc4543b8b54f58bbcffbc598` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:17:16.417507 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6545b348-4687-4083-92d3-7f285e0b552b admin admin] PATH_INFO: `/v3/projects/abddcee3849944c8a1d10ed888a4720e/users/805781d7bb864aed9d311bc861e9bd4f/roles/e1c04b19fc4543b8b54f58bbcffbc598` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:17:16.440917 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6545b348-4687-4083-92d3-7f285e0b552b admin admin] RBAC: Authorizing `identity:create_grant(project_id=abddcee3849944c8a1d10ed888a4720e, user_id=805781d7bb864aed9d311bc861e9bd4f, role_id=e1c04b19fc4543b8b54f58bbcffbc598)` {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:17:16.442543 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92abff67-bd03-4603-9350-36fabaf31eef admin admin] RBAC: Authorizing `identity:create_grant(project_id=c245a79031d74d889e50a87f958a7585, user_id=4b30c102907740d8bad46dace08d4ef2, role_id=e1c04b19fc4543b8b54f58bbcffbc598)` {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:17:16.444373 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6545b348-4687-4083-92d3-7f285e0b552b admin admin] RBAC: Authorization granted {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:17:16.447414 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92abff67-bd03-4603-9350-36fabaf31eef admin admin] RBAC: Authorization granted {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:17:16.480490 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 1012/2037] 10.222.0.82 () {60 vars in 1483 bytes} [Tue Aug 2 10:17:16 2022] PUT /identity/v3/projects/c245a79031d74d889e50a87f958a7585/users/4b30c102907740d8bad46dace08d4ef2/roles/e1c04b19fc4543b8b54f58bbcffbc598 => generated 0 bytes in 72 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 02 10:17:16.482075 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: [pid: 80377|app: 0|req: 1025/2037] 10.222.0.82 () {60 vars in 1483 bytes} [Tue Aug 2 10:17:16 2022] PUT /identity/v3/projects/abddcee3849944c8a1d10ed888a4720e/users/805781d7bb864aed9d311bc861e9bd4f/roles/e1c04b19fc4543b8b54f58bbcffbc598 => generated 0 bytes in 76 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 02 10:17:16.483196 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a47395f-4ba7-4599-980f-023f673d42a6 None None] REQUEST_METHOD: `POST` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:17:16.483564 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a47395f-4ba7-4599-980f-023f673d42a6 None None] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:17:16.483968 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a47395f-4ba7-4599-980f-023f673d42a6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:17:16.484205 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68eb0d54-744b-4e21-b63b-c6b02f0b6ce1 None None] REQUEST_METHOD: `POST` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:17:16.484502 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68eb0d54-744b-4e21-b63b-c6b02f0b6ce1 None None] SCRIPT_NAME: `/identity` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:17:16.484860 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68eb0d54-744b-4e21-b63b-c6b02f0b6ce1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:17:16.515071 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.auth.core [None req-4a47395f-4ba7-4599-980f-023f673d42a6 None None] MFA Rules not processed for user `805781d7bb864aed9d311bc861e9bd4f`. Rule list: `[]` (Enabled: `True`). {{(pid=80378) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 02 10:17:16.516413 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.fernet_utils [None req-4a47395f-4ba7-4599-980f-023f673d42a6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80378) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 10:17:16.516770 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.auth.core [None req-68eb0d54-744b-4e21-b63b-c6b02f0b6ce1 None None] MFA Rules not processed for user `4b30c102907740d8bad46dace08d4ef2`. Rule list: `[]` (Enabled: `True`). {{(pid=80377) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 02 10:17:16.518567 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.fernet_utils [None req-68eb0d54-744b-4e21-b63b-c6b02f0b6ce1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80377) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 10:17:16.606478 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 1013/2038] 10.222.0.82 () {58 vars in 958 bytes} [Tue Aug 2 10:17:16 2022] POST /identity/v3/auth/tokens => generated 3256 bytes in 125 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 02 10:17:16.613991 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a47395f-4ba7-4599-980f-023f673d42a6 None None] Authenticating user token {{(pid=80378) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:17:16.625068 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: [pid: 80377|app: 0|req: 1026/2039] 10.222.0.82 () {58 vars in 958 bytes} [Tue Aug 2 10:17:16 2022] POST /identity/v3/auth/tokens => generated 3260 bytes in 138 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 02 10:17:16.627681 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68eb0d54-744b-4e21-b63b-c6b02f0b6ce1 None None] Authenticating user token {{(pid=80377) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:17:16.649261 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG dbcounter [-] [80377] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80377) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 10:17:16.725951 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a47395f-4ba7-4599-980f-023f673d42a6 None None] Validating token access rules against request {{(pid=80378) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:17:16.728074 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68eb0d54-744b-4e21-b63b-c6b02f0b6ce1 None None] Validating token access rules against request {{(pid=80377) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:17:16.731403 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ca9974f-3268-4dc1-af26-9b29e15e09b1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '891005fa5ff74041b75747d16cc66798', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7047912c5acd4adeba5d87c3bf895e39', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80378) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:17:16.732073 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ca9974f-3268-4dc1-af26-9b29e15e09b1 admin admin] REQUEST_METHOD: `POST` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:17:16.732357 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ca9974f-3268-4dc1-af26-9b29e15e09b1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:17:16.732634 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ca9974f-3268-4dc1-af26-9b29e15e09b1 admin admin] PATH_INFO: `/v3/projects` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:17:16.733187 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4fb55bd-3557-4107-94f2-a70157777382 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '891005fa5ff74041b75747d16cc66798', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7047912c5acd4adeba5d87c3bf895e39', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80377) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:17:16.733336 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ca9974f-3268-4dc1-af26-9b29e15e09b1 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:17:16.733921 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4fb55bd-3557-4107-94f2-a70157777382 admin admin] REQUEST_METHOD: `POST` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:17:16.734275 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4fb55bd-3557-4107-94f2-a70157777382 admin admin] SCRIPT_NAME: `/identity` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:17:16.734660 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4fb55bd-3557-4107-94f2-a70157777382 admin admin] PATH_INFO: `/v3/projects` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:17:16.735299 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ca9974f-3268-4dc1-af26-9b29e15e09b1 admin admin] RBAC: Authorization granted {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:17:16.735402 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4fb55bd-3557-4107-94f2-a70157777382 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:17:16.737355 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4fb55bd-3557-4107-94f2-a70157777382 admin admin] RBAC: Authorization granted {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:17:16.768007 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: [pid: 80377|app: 0|req: 1027/2040] 10.222.0.82 () {60 vars in 1153 bytes} [Tue Aug 2 10:17:16 2022] POST /identity/v3/projects => generated 383 bytes in 142 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 02 10:17:16.770483 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 1014/2041] 10.222.0.82 () {60 vars in 1153 bytes} [Tue Aug 2 10:17:16 2022] POST /identity/v3/projects => generated 379 bytes in 157 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 02 10:17:16.774004 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ca9974f-3268-4dc1-af26-9b29e15e09b1 admin admin] Authenticating user token {{(pid=80378) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:17:16.774641 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4fb55bd-3557-4107-94f2-a70157777382 admin admin] Authenticating user token {{(pid=80377) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:17:16.835976 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG dbcounter [-] [80378] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80378) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 10:17:16.859342 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4fb55bd-3557-4107-94f2-a70157777382 admin admin] Validating token access rules against request {{(pid=80377) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:17:16.865068 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22fbb81a-a59c-4f52-89e8-e404a6d6027d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '891005fa5ff74041b75747d16cc66798', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7047912c5acd4adeba5d87c3bf895e39', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80377) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:17:16.865999 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22fbb81a-a59c-4f52-89e8-e404a6d6027d admin admin] REQUEST_METHOD: `POST` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:17:16.866359 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22fbb81a-a59c-4f52-89e8-e404a6d6027d admin admin] SCRIPT_NAME: `/identity` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:17:16.866742 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22fbb81a-a59c-4f52-89e8-e404a6d6027d admin admin] PATH_INFO: `/v3/users` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:17:16.867867 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22fbb81a-a59c-4f52-89e8-e404a6d6027d admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:17:16.869518 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22fbb81a-a59c-4f52-89e8-e404a6d6027d admin admin] RBAC: Authorization granted {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:17:16.871947 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ca9974f-3268-4dc1-af26-9b29e15e09b1 admin admin] Validating token access rules against request {{(pid=80378) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:17:16.886943 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07156ba3-e004-4f48-af86-2836253521af admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '891005fa5ff74041b75747d16cc66798', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7047912c5acd4adeba5d87c3bf895e39', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80378) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:17:16.887665 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07156ba3-e004-4f48-af86-2836253521af admin admin] REQUEST_METHOD: `POST` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:17:16.888718 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07156ba3-e004-4f48-af86-2836253521af admin admin] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:17:16.889043 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07156ba3-e004-4f48-af86-2836253521af admin admin] PATH_INFO: `/v3/users` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:17:16.890160 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07156ba3-e004-4f48-af86-2836253521af admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:17:16.893856 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07156ba3-e004-4f48-af86-2836253521af admin admin] RBAC: Authorization granted {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:17:16.901053 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: [pid: 80377|app: 0|req: 1028/2042] 10.222.0.82 () {60 vars in 1144 bytes} [Tue Aug 2 10:17:16 2022] POST /identity/v3/users => generated 298 bytes in 127 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 02 10:17:16.905564 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22fbb81a-a59c-4f52-89e8-e404a6d6027d admin admin] Authenticating user token {{(pid=80377) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:17:16.916592 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22fbb81a-a59c-4f52-89e8-e404a6d6027d admin admin] Validating token access rules against request {{(pid=80377) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:17:16.921864 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 1015/2043] 10.222.0.82 () {60 vars in 1144 bytes} [Tue Aug 2 10:17:16 2022] POST /identity/v3/users => generated 290 bytes in 145 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 02 10:17:16.921992 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b7932b6-b025-4b0c-9e4a-33e1502d9a2c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '891005fa5ff74041b75747d16cc66798', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7047912c5acd4adeba5d87c3bf895e39', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80377) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:17:16.922208 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b7932b6-b025-4b0c-9e4a-33e1502d9a2c admin admin] REQUEST_METHOD: `GET` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:17:16.922754 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b7932b6-b025-4b0c-9e4a-33e1502d9a2c admin admin] SCRIPT_NAME: `/identity` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:17:16.922754 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b7932b6-b025-4b0c-9e4a-33e1502d9a2c admin admin] PATH_INFO: `/v3/roles` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:17:16.922948 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07156ba3-e004-4f48-af86-2836253521af admin admin] Authenticating user token {{(pid=80378) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:17:16.923486 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b7932b6-b025-4b0c-9e4a-33e1502d9a2c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:17:16.925233 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b7932b6-b025-4b0c-9e4a-33e1502d9a2c admin admin] RBAC: Authorization granted {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:17:16.935393 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07156ba3-e004-4f48-af86-2836253521af admin admin] Validating token access rules against request {{(pid=80378) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:17:16.937872 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: [pid: 80377|app: 0|req: 1029/2044] 10.222.0.82 () {58 vars in 1122 bytes} [Tue Aug 2 10:17:16 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 02 10:17:16.942207 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b7932b6-b025-4b0c-9e4a-33e1502d9a2c admin admin] Authenticating user token {{(pid=80377) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:17:16.944442 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69c1346f-149f-4324-8a06-f50b94eac7ad admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '891005fa5ff74041b75747d16cc66798', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7047912c5acd4adeba5d87c3bf895e39', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80378) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:17:16.945091 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69c1346f-149f-4324-8a06-f50b94eac7ad admin admin] REQUEST_METHOD: `GET` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:17:16.945372 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69c1346f-149f-4324-8a06-f50b94eac7ad admin admin] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:17:16.945643 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69c1346f-149f-4324-8a06-f50b94eac7ad admin admin] PATH_INFO: `/v3/roles` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:17:16.946365 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69c1346f-149f-4324-8a06-f50b94eac7ad admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:17:16.948141 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69c1346f-149f-4324-8a06-f50b94eac7ad admin admin] RBAC: Authorization granted {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:17:16.958285 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b7932b6-b025-4b0c-9e4a-33e1502d9a2c admin admin] Validating token access rules against request {{(pid=80377) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:17:16.964154 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4b43197-4a82-4ce3-a074-5f1af05438ba admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '891005fa5ff74041b75747d16cc66798', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7047912c5acd4adeba5d87c3bf895e39', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80377) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:17:16.965072 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4b43197-4a82-4ce3-a074-5f1af05438ba admin admin] REQUEST_METHOD: `PUT` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:17:16.965466 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4b43197-4a82-4ce3-a074-5f1af05438ba admin admin] SCRIPT_NAME: `/identity` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:17:16.965898 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4b43197-4a82-4ce3-a074-5f1af05438ba admin admin] PATH_INFO: `/v3/projects/ca5ccb9165c74f3a8bd3101622cf79b0/users/51540feb0a154a82a116fc6d91cf5d46/roles/237966442f05411191105bf336bc14f3` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:17:16.973097 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 1016/2045] 10.222.0.82 () {58 vars in 1122 bytes} [Tue Aug 2 10:17:16 2022] GET /identity/v3/roles => generated 1420 bytes in 48 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 02 10:17:16.974493 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69c1346f-149f-4324-8a06-f50b94eac7ad admin admin] Authenticating user token {{(pid=80378) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:17:16.999770 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69c1346f-149f-4324-8a06-f50b94eac7ad admin admin] Validating token access rules against request {{(pid=80378) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:17:17.002806 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4b43197-4a82-4ce3-a074-5f1af05438ba admin admin] RBAC: Authorizing `identity:create_grant(project_id=ca5ccb9165c74f3a8bd3101622cf79b0, user_id=51540feb0a154a82a116fc6d91cf5d46, role_id=237966442f05411191105bf336bc14f3)` {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:17:17.005721 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4b43197-4a82-4ce3-a074-5f1af05438ba admin admin] RBAC: Authorization granted {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:17:17.016151 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eeff131d-33df-4b5a-a741-ad7185f4411d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '891005fa5ff74041b75747d16cc66798', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7047912c5acd4adeba5d87c3bf895e39', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80378) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:17:17.017002 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eeff131d-33df-4b5a-a741-ad7185f4411d admin admin] REQUEST_METHOD: `GET` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:17:17.017499 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eeff131d-33df-4b5a-a741-ad7185f4411d admin admin] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:17:17.017976 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eeff131d-33df-4b5a-a741-ad7185f4411d admin admin] PATH_INFO: `/v3/roles` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:17:17.019183 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eeff131d-33df-4b5a-a741-ad7185f4411d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:17:17.021107 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eeff131d-33df-4b5a-a741-ad7185f4411d admin admin] RBAC: Authorization granted {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:17:17.048261 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: [pid: 80377|app: 0|req: 1030/2046] 10.222.0.82 () {60 vars in 1483 bytes} [Tue Aug 2 10:17:16 2022] PUT /identity/v3/projects/ca5ccb9165c74f3a8bd3101622cf79b0/users/51540feb0a154a82a116fc6d91cf5d46/roles/237966442f05411191105bf336bc14f3 => generated 0 bytes in 106 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 02 10:17:17.052041 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4b43197-4a82-4ce3-a074-5f1af05438ba admin admin] Authenticating user token {{(pid=80377) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:17:17.057945 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 1017/2047] 10.222.0.82 () {58 vars in 1122 bytes} [Tue Aug 2 10:17:16 2022] GET /identity/v3/roles => generated 1420 bytes in 84 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 02 10:17:17.058826 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eeff131d-33df-4b5a-a741-ad7185f4411d admin admin] Authenticating user token {{(pid=80378) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:17:17.074619 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eeff131d-33df-4b5a-a741-ad7185f4411d admin admin] Validating token access rules against request {{(pid=80378) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:17:17.164544 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10a0a9ca-e067-4f37-801d-6b4aa976e078 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '891005fa5ff74041b75747d16cc66798', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7047912c5acd4adeba5d87c3bf895e39', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80378) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:17:17.165242 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10a0a9ca-e067-4f37-801d-6b4aa976e078 admin admin] REQUEST_METHOD: `PUT` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:17:17.165529 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10a0a9ca-e067-4f37-801d-6b4aa976e078 admin admin] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:17:17.165834 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10a0a9ca-e067-4f37-801d-6b4aa976e078 admin admin] PATH_INFO: `/v3/projects/6ea1d92c402e4adda4cfdce12956d682/users/c4e2fa69c37b47219e1c9b4d2590a12c/roles/e1c04b19fc4543b8b54f58bbcffbc598` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:17:17.200190 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10a0a9ca-e067-4f37-801d-6b4aa976e078 admin admin] RBAC: Authorizing `identity:create_grant(project_id=6ea1d92c402e4adda4cfdce12956d682, user_id=c4e2fa69c37b47219e1c9b4d2590a12c, role_id=e1c04b19fc4543b8b54f58bbcffbc598)` {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:17:17.203840 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10a0a9ca-e067-4f37-801d-6b4aa976e078 admin admin] RBAC: Authorization granted {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:17:17.225342 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG dbcounter [-] [80377] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80377) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 10:17:17.229741 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 1018/2048] 10.222.0.82 () {60 vars in 1483 bytes} [Tue Aug 2 10:17:17 2022] PUT /identity/v3/projects/6ea1d92c402e4adda4cfdce12956d682/users/c4e2fa69c37b47219e1c9b4d2590a12c/roles/e1c04b19fc4543b8b54f58bbcffbc598 => generated 0 bytes in 171 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 02 10:17:17.246865 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f163ab2c-32d2-4de3-bd29-bac125a315a7 None None] REQUEST_METHOD: `POST` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:17:17.247168 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f163ab2c-32d2-4de3-bd29-bac125a315a7 None None] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:17:17.248341 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f163ab2c-32d2-4de3-bd29-bac125a315a7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:17:17.267254 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4b43197-4a82-4ce3-a074-5f1af05438ba admin admin] Validating token access rules against request {{(pid=80377) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:17:17.306782 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.auth.core [None req-f163ab2c-32d2-4de3-bd29-bac125a315a7 None None] MFA Rules not processed for user `c4e2fa69c37b47219e1c9b4d2590a12c`. Rule list: `[]` (Enabled: `True`). {{(pid=80378) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 02 10:17:17.308095 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.fernet_utils [None req-f163ab2c-32d2-4de3-bd29-bac125a315a7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80378) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 10:17:17.464049 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 1019/2049] 10.222.0.82 () {58 vars in 958 bytes} [Tue Aug 2 10:17:17 2022] POST /identity/v3/auth/tokens => generated 3256 bytes in 220 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 02 10:17:17.471479 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f163ab2c-32d2-4de3-bd29-bac125a315a7 None None] Authenticating user token {{(pid=80378) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:17:17.484871 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eabfc6eb-f74c-4c12-8a92-592477b3b536 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '891005fa5ff74041b75747d16cc66798', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7047912c5acd4adeba5d87c3bf895e39', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80377) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:17:17.485749 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eabfc6eb-f74c-4c12-8a92-592477b3b536 admin admin] REQUEST_METHOD: `GET` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:17:17.486068 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eabfc6eb-f74c-4c12-8a92-592477b3b536 admin admin] SCRIPT_NAME: `/identity` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:17:17.486399 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eabfc6eb-f74c-4c12-8a92-592477b3b536 admin admin] PATH_INFO: `/v3/roles` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:17:17.487286 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eabfc6eb-f74c-4c12-8a92-592477b3b536 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:17:17.489359 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eabfc6eb-f74c-4c12-8a92-592477b3b536 admin admin] RBAC: Authorization granted {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:17:17.511047 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: [pid: 80377|app: 0|req: 1031/2050] 10.222.0.82 () {58 vars in 1122 bytes} [Tue Aug 2 10:17:17 2022] GET /identity/v3/roles => generated 1420 bytes in 459 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 02 10:17:17.518863 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eabfc6eb-f74c-4c12-8a92-592477b3b536 admin admin] Authenticating user token {{(pid=80377) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:17:17.539474 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG dbcounter [-] [80378] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=80378) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 10:17:17.542966 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eabfc6eb-f74c-4c12-8a92-592477b3b536 admin admin] Validating token access rules against request {{(pid=80377) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:17:17.554450 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba474f9c-1760-47af-ad85-3f902908359c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '891005fa5ff74041b75747d16cc66798', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7047912c5acd4adeba5d87c3bf895e39', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80377) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:17:17.555309 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba474f9c-1760-47af-ad85-3f902908359c admin admin] REQUEST_METHOD: `PUT` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:17:17.556551 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba474f9c-1760-47af-ad85-3f902908359c admin admin] SCRIPT_NAME: `/identity` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:17:17.556929 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba474f9c-1760-47af-ad85-3f902908359c admin admin] PATH_INFO: `/v3/projects/ca5ccb9165c74f3a8bd3101622cf79b0/users/51540feb0a154a82a116fc6d91cf5d46/roles/237966442f05411191105bf336bc14f3` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:17:17.560306 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba474f9c-1760-47af-ad85-3f902908359c admin admin] RBAC: Authorizing `identity:create_grant(project_id=ca5ccb9165c74f3a8bd3101622cf79b0, user_id=51540feb0a154a82a116fc6d91cf5d46, role_id=237966442f05411191105bf336bc14f3)` {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:17:17.564977 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba474f9c-1760-47af-ad85-3f902908359c admin admin] RBAC: Authorization granted {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:17:17.595362 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: [pid: 80377|app: 0|req: 1032/2051] 10.222.0.82 () {60 vars in 1483 bytes} [Tue Aug 2 10:17:17 2022] PUT /identity/v3/projects/ca5ccb9165c74f3a8bd3101622cf79b0/users/51540feb0a154a82a116fc6d91cf5d46/roles/237966442f05411191105bf336bc14f3 => generated 0 bytes in 63 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 02 10:17:17.597907 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2ad5074-649e-4fcd-8413-412575498bc6 None None] REQUEST_METHOD: `POST` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:17:17.598349 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2ad5074-649e-4fcd-8413-412575498bc6 None None] SCRIPT_NAME: `/identity` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:17:17.598725 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2ad5074-649e-4fcd-8413-412575498bc6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:17:17.648288 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.auth.core [None req-b2ad5074-649e-4fcd-8413-412575498bc6 None None] MFA Rules not processed for user `51540feb0a154a82a116fc6d91cf5d46`. Rule list: `[]` (Enabled: `True`). {{(pid=80377) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 02 10:17:17.649963 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.fernet_utils [None req-b2ad5074-649e-4fcd-8413-412575498bc6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80377) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 10:17:17.660728 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f163ab2c-32d2-4de3-bd29-bac125a315a7 None None] Validating token access rules against request {{(pid=80378) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:17:17.800704 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d3b7c03-b2bf-46d0-90cb-32c3fd7b45f9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '891005fa5ff74041b75747d16cc66798', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7047912c5acd4adeba5d87c3bf895e39', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80378) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:17:17.802744 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d3b7c03-b2bf-46d0-90cb-32c3fd7b45f9 admin admin] REQUEST_METHOD: `POST` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:17:17.803552 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d3b7c03-b2bf-46d0-90cb-32c3fd7b45f9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:17:17.804022 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d3b7c03-b2bf-46d0-90cb-32c3fd7b45f9 admin admin] PATH_INFO: `/v3/projects` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:17:17.805437 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d3b7c03-b2bf-46d0-90cb-32c3fd7b45f9 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:17:17.810728 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d3b7c03-b2bf-46d0-90cb-32c3fd7b45f9 admin admin] RBAC: Authorization granted {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:17:17.811345 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: [pid: 80377|app: 0|req: 1033/2052] 10.222.0.82 () {58 vars in 958 bytes} [Tue Aug 2 10:17:17 2022] POST /identity/v3/auth/tokens => generated 3327 bytes in 214 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 02 10:17:17.824823 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2ad5074-649e-4fcd-8413-412575498bc6 None None] Authenticating user token {{(pid=80377) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:17:17.827011 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.fernet_utils [None req-b2ad5074-649e-4fcd-8413-412575498bc6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80377) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 10:17:17.886877 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 1020/2053] 10.222.0.82 () {60 vars in 1153 bytes} [Tue Aug 2 10:17:17 2022] POST /identity/v3/projects => generated 377 bytes in 416 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 02 10:17:17.892278 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d3b7c03-b2bf-46d0-90cb-32c3fd7b45f9 admin admin] Authenticating user token {{(pid=80378) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:17:17.917431 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2ad5074-649e-4fcd-8413-412575498bc6 None None] Validating token access rules against request {{(pid=80377) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:17:17.946670 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG dbcounter [-] [80377] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80377) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 10:17:18.002236 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6aa8e8a8-ecc1-46bb-b083-2f57df4e620f service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1023de281ae74e9d8a6d6c312c1c97a7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f1b38ebb804c41c69772c998aae72c41', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80377) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:17:18.003203 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6aa8e8a8-ecc1-46bb-b083-2f57df4e620f service cinder] REQUEST_METHOD: `GET` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:17:18.003954 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6aa8e8a8-ecc1-46bb-b083-2f57df4e620f service cinder] SCRIPT_NAME: `/identity` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:17:18.005005 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6aa8e8a8-ecc1-46bb-b083-2f57df4e620f service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:17:18.047621 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d3b7c03-b2bf-46d0-90cb-32c3fd7b45f9 admin admin] Validating token access rules against request {{(pid=80378) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:17:18.066729 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f487a698-c2c2-4ac2-bd8d-870f0473f976 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '891005fa5ff74041b75747d16cc66798', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7047912c5acd4adeba5d87c3bf895e39', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80378) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:17:18.067863 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f487a698-c2c2-4ac2-bd8d-870f0473f976 admin admin] REQUEST_METHOD: `POST` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:17:18.068357 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f487a698-c2c2-4ac2-bd8d-870f0473f976 admin admin] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:17:18.069027 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f487a698-c2c2-4ac2-bd8d-870f0473f976 admin admin] PATH_INFO: `/v3/users` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:17:18.070497 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f487a698-c2c2-4ac2-bd8d-870f0473f976 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:17:18.082221 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f487a698-c2c2-4ac2-bd8d-870f0473f976 admin admin] RBAC: Authorization granted {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:17:18.148370 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 1021/2054] 10.222.0.82 () {60 vars in 1144 bytes} [Tue Aug 2 10:17:17 2022] POST /identity/v3/users => generated 295 bytes in 256 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 02 10:17:18.152523 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f487a698-c2c2-4ac2-bd8d-870f0473f976 admin admin] Authenticating user token {{(pid=80378) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:17:18.162704 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6aa8e8a8-ecc1-46bb-b083-2f57df4e620f service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:17:18.164360 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6aa8e8a8-ecc1-46bb-b083-2f57df4e620f service cinder] RBAC: Authorization granted {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:17:18.165057 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f487a698-c2c2-4ac2-bd8d-870f0473f976 admin admin] Validating token access rules against request {{(pid=80378) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:17:18.171954 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd79c1e6-8048-45b8-be11-c350385d7b0a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '891005fa5ff74041b75747d16cc66798', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7047912c5acd4adeba5d87c3bf895e39', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80378) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:17:18.172767 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd79c1e6-8048-45b8-be11-c350385d7b0a admin admin] REQUEST_METHOD: `GET` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:17:18.173168 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd79c1e6-8048-45b8-be11-c350385d7b0a admin admin] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:17:18.173543 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd79c1e6-8048-45b8-be11-c350385d7b0a admin admin] PATH_INFO: `/v3/roles` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:17:18.174438 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd79c1e6-8048-45b8-be11-c350385d7b0a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:17:18.176355 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd79c1e6-8048-45b8-be11-c350385d7b0a admin admin] RBAC: Authorization granted {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:17:18.180170 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: [pid: 80377|app: 0|req: 1034/2055] 10.222.0.82 () {60 vars in 1365 bytes} [Tue Aug 2 10:17:17 2022] GET /identity/v3/auth/tokens => generated 3327 bytes in 356 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 02 10:17:18.190669 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 1022/2056] 10.222.0.82 () {58 vars in 1122 bytes} [Tue Aug 2 10:17:18 2022] GET /identity/v3/roles => generated 1420 bytes in 39 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 02 10:17:18.194840 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6aa8e8a8-ecc1-46bb-b083-2f57df4e620f service cinder] Authenticating user token {{(pid=80377) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:17:18.206905 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6aa8e8a8-ecc1-46bb-b083-2f57df4e620f service cinder] Validating token access rules against request {{(pid=80377) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:17:18.212645 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b5d6d18-4f7c-4645-87d5-a053b1b84877 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '891005fa5ff74041b75747d16cc66798', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7047912c5acd4adeba5d87c3bf895e39', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80377) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:17:18.213401 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b5d6d18-4f7c-4645-87d5-a053b1b84877 admin admin] REQUEST_METHOD: `PUT` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:17:18.213684 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b5d6d18-4f7c-4645-87d5-a053b1b84877 admin admin] SCRIPT_NAME: `/identity` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:17:18.214568 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b5d6d18-4f7c-4645-87d5-a053b1b84877 admin admin] PATH_INFO: `/v3/projects/49695c58be9542959c6852e9207b1043/users/4a6a1371a7f945da87ecc410eca99e37/roles/237966442f05411191105bf336bc14f3` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:17:18.237586 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b5d6d18-4f7c-4645-87d5-a053b1b84877 admin admin] RBAC: Authorizing `identity:create_grant(project_id=49695c58be9542959c6852e9207b1043, user_id=4a6a1371a7f945da87ecc410eca99e37, role_id=237966442f05411191105bf336bc14f3)` {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:17:18.240553 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b5d6d18-4f7c-4645-87d5-a053b1b84877 admin admin] RBAC: Authorization granted {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:17:18.269851 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: [pid: 80377|app: 0|req: 1035/2057] 10.222.0.82 () {60 vars in 1483 bytes} [Tue Aug 2 10:17:18 2022] PUT /identity/v3/projects/49695c58be9542959c6852e9207b1043/users/4a6a1371a7f945da87ecc410eca99e37/roles/237966442f05411191105bf336bc14f3 => generated 0 bytes in 76 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 02 10:17:18.273824 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd79c1e6-8048-45b8-be11-c350385d7b0a admin admin] Authenticating user token {{(pid=80378) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:17:18.287838 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd79c1e6-8048-45b8-be11-c350385d7b0a admin admin] Validating token access rules against request {{(pid=80378) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:17:18.311440 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b5d6d18-4f7c-4645-87d5-a053b1b84877 admin admin] Authenticating user token {{(pid=80377) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:17:18.336939 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG dbcounter [-] [80378] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=80378) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 10:17:18.377402 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-890cc6ac-365e-4c45-a803-f209b8565f7d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '891005fa5ff74041b75747d16cc66798', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7047912c5acd4adeba5d87c3bf895e39', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80378) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:17:18.378201 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-890cc6ac-365e-4c45-a803-f209b8565f7d admin admin] REQUEST_METHOD: `GET` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:17:18.378607 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-890cc6ac-365e-4c45-a803-f209b8565f7d admin admin] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:17:18.378985 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-890cc6ac-365e-4c45-a803-f209b8565f7d admin admin] PATH_INFO: `/v3/roles` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:17:18.379924 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-890cc6ac-365e-4c45-a803-f209b8565f7d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:17:18.381573 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-890cc6ac-365e-4c45-a803-f209b8565f7d admin admin] RBAC: Authorization granted {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:17:18.398681 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 1023/2058] 10.222.0.82 () {58 vars in 1122 bytes} [Tue Aug 2 10:17:18 2022] GET /identity/v3/roles => generated 1420 bytes in 125 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 02 10:17:18.402501 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-890cc6ac-365e-4c45-a803-f209b8565f7d admin admin] Authenticating user token {{(pid=80378) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:17:18.441182 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-890cc6ac-365e-4c45-a803-f209b8565f7d admin admin] Validating token access rules against request {{(pid=80378) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:17:18.442092 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b5d6d18-4f7c-4645-87d5-a053b1b84877 admin admin] Validating token access rules against request {{(pid=80377) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:17:18.453162 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b038374-8bcb-450e-8f0b-f4892ecce4f3 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1023de281ae74e9d8a6d6c312c1c97a7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f1b38ebb804c41c69772c998aae72c41', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80377) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:17:18.453828 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b038374-8bcb-450e-8f0b-f4892ecce4f3 service cinder] REQUEST_METHOD: `GET` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:17:18.454115 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b038374-8bcb-450e-8f0b-f4892ecce4f3 service cinder] SCRIPT_NAME: `/identity` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:17:18.454386 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b038374-8bcb-450e-8f0b-f4892ecce4f3 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:17:18.457449 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b24b8b3-7bd0-4b4b-99ca-9411779b04fc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '891005fa5ff74041b75747d16cc66798', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7047912c5acd4adeba5d87c3bf895e39', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80378) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:17:18.458179 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b24b8b3-7bd0-4b4b-99ca-9411779b04fc admin admin] REQUEST_METHOD: `PUT` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:17:18.458466 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b24b8b3-7bd0-4b4b-99ca-9411779b04fc admin admin] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:17:18.459513 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b24b8b3-7bd0-4b4b-99ca-9411779b04fc admin admin] PATH_INFO: `/v3/projects/49695c58be9542959c6852e9207b1043/users/4a6a1371a7f945da87ecc410eca99e37/roles/237966442f05411191105bf336bc14f3` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:17:18.471565 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b24b8b3-7bd0-4b4b-99ca-9411779b04fc admin admin] RBAC: Authorizing `identity:create_grant(project_id=49695c58be9542959c6852e9207b1043, user_id=4a6a1371a7f945da87ecc410eca99e37, role_id=237966442f05411191105bf336bc14f3)` {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:17:18.474560 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b24b8b3-7bd0-4b4b-99ca-9411779b04fc admin admin] RBAC: Authorization granted {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:17:18.491065 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 1024/2059] 10.222.0.82 () {60 vars in 1483 bytes} [Tue Aug 2 10:17:18 2022] PUT /identity/v3/projects/49695c58be9542959c6852e9207b1043/users/4a6a1371a7f945da87ecc410eca99e37/roles/237966442f05411191105bf336bc14f3 => generated 0 bytes in 89 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 02 10:17:18.502359 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9cd3595-4c71-459c-adb6-b678ed94292f None None] REQUEST_METHOD: `POST` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:17:18.502646 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9cd3595-4c71-459c-adb6-b678ed94292f None None] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:17:18.505351 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9cd3595-4c71-459c-adb6-b678ed94292f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:17:18.550155 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.auth.core [None req-e9cd3595-4c71-459c-adb6-b678ed94292f None None] MFA Rules not processed for user `4a6a1371a7f945da87ecc410eca99e37`. Rule list: `[]` (Enabled: `True`). {{(pid=80378) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 02 10:17:18.551417 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.fernet_utils [None req-e9cd3595-4c71-459c-adb6-b678ed94292f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80378) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 10:17:18.584838 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b038374-8bcb-450e-8f0b-f4892ecce4f3 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:17:18.586802 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b038374-8bcb-450e-8f0b-f4892ecce4f3 service cinder] RBAC: Authorization granted {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:17:18.600183 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: [pid: 80377|app: 0|req: 1036/2060] 10.222.0.82 () {60 vars in 1365 bytes} [Tue Aug 2 10:17:18 2022] GET /identity/v3/auth/tokens => generated 3260 bytes in 289 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 02 10:17:18.690981 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 1025/2061] 10.222.0.82 () {58 vars in 958 bytes} [Tue Aug 2 10:17:18 2022] POST /identity/v3/auth/tokens => generated 3321 bytes in 190 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 02 10:17:18.699108 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b038374-8bcb-450e-8f0b-f4892ecce4f3 service cinder] Authenticating user token {{(pid=80377) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:17:18.701513 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.fernet_utils [None req-7b038374-8bcb-450e-8f0b-f4892ecce4f3 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80377) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 10:17:18.710797 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG dbcounter [-] [80377] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80377) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 10:17:18.721092 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b038374-8bcb-450e-8f0b-f4892ecce4f3 service cinder] Validating token access rules against request {{(pid=80377) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:17:18.823337 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfec150e-9de8-46a3-8154-7c9d18ee6fcd service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1023de281ae74e9d8a6d6c312c1c97a7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f1b38ebb804c41c69772c998aae72c41', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80377) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:17:18.824033 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfec150e-9de8-46a3-8154-7c9d18ee6fcd service cinder] REQUEST_METHOD: `GET` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:17:18.824246 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfec150e-9de8-46a3-8154-7c9d18ee6fcd service cinder] SCRIPT_NAME: `/identity` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:17:18.824519 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfec150e-9de8-46a3-8154-7c9d18ee6fcd service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:17:18.836105 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dfec150e-9de8-46a3-8154-7c9d18ee6fcd service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:17:18.837593 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dfec150e-9de8-46a3-8154-7c9d18ee6fcd service cinder] RBAC: Authorization granted {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:17:18.872445 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: [pid: 80377|app: 0|req: 1037/2062] 10.222.0.82 () {60 vars in 1365 bytes} [Tue Aug 2 10:17:18 2022] GET /identity/v3/auth/tokens => generated 3321 bytes in 174 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 02 10:17:18.958540 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9cd3595-4c71-459c-adb6-b678ed94292f None None] Authenticating user token {{(pid=80378) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:17:18.981044 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9cd3595-4c71-459c-adb6-b678ed94292f None None] Validating token access rules against request {{(pid=80378) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:17:18.995653 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3d9c1b5-9ce9-4d6f-af26-2bfe0374964e service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1023de281ae74e9d8a6d6c312c1c97a7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f1b38ebb804c41c69772c998aae72c41', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80378) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:17:18.996456 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3d9c1b5-9ce9-4d6f-af26-2bfe0374964e service cinder] REQUEST_METHOD: `GET` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:17:18.996738 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3d9c1b5-9ce9-4d6f-af26-2bfe0374964e service cinder] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:17:18.997053 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3d9c1b5-9ce9-4d6f-af26-2bfe0374964e service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:17:19.123723 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3d9c1b5-9ce9-4d6f-af26-2bfe0374964e service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:17:19.126808 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3d9c1b5-9ce9-4d6f-af26-2bfe0374964e service cinder] RBAC: Authorization granted {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:17:19.150039 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 1026/2063] 10.222.0.82 () {60 vars in 1365 bytes} [Tue Aug 2 10:17:18 2022] GET /identity/v3/auth/tokens => generated 3256 bytes in 192 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 02 10:17:21.751738 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfec150e-9de8-46a3-8154-7c9d18ee6fcd service cinder] Authenticating user token {{(pid=80377) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:17:21.888083 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfec150e-9de8-46a3-8154-7c9d18ee6fcd service cinder] Validating token access rules against request {{(pid=80377) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:17:21.893362 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e2169c1-975f-494e-91ac-0ea4c8035cac admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '891005fa5ff74041b75747d16cc66798', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7047912c5acd4adeba5d87c3bf895e39', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80377) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:17:21.894134 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e2169c1-975f-494e-91ac-0ea4c8035cac admin admin] REQUEST_METHOD: `DELETE` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:17:21.894417 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e2169c1-975f-494e-91ac-0ea4c8035cac admin admin] SCRIPT_NAME: `/identity` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:17:21.894696 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e2169c1-975f-494e-91ac-0ea4c8035cac admin admin] PATH_INFO: `/v3/users/4b30c102907740d8bad46dace08d4ef2` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:17:21.895960 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e2169c1-975f-494e-91ac-0ea4c8035cac admin admin] RBAC: Authorizing `identity:delete_user(user_id=4b30c102907740d8bad46dace08d4ef2)` {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:17:21.897959 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e2169c1-975f-494e-91ac-0ea4c8035cac admin admin] RBAC: Authorization granted {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:17:21.964328 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.notifications [None req-3e2169c1-975f-494e-91ac-0ea4c8035cac admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '4b30c102907740d8bad46dace08d4ef2'} {{(pid=80377) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 10:17:21.976696 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.notifications [None req-3e2169c1-975f-494e-91ac-0ea4c8035cac admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '4b30c102907740d8bad46dace08d4ef2'} {{(pid=80377) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 10:17:21.996731 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.notifications [None req-3e2169c1-975f-494e-91ac-0ea4c8035cac admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '4b30c102907740d8bad46dace08d4ef2'} {{(pid=80377) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 10:17:21.997688 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.notifications [None req-3e2169c1-975f-494e-91ac-0ea4c8035cac admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '4b30c102907740d8bad46dace08d4ef2'} {{(pid=80377) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 10:17:22.035163 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.notifications [None req-3e2169c1-975f-494e-91ac-0ea4c8035cac admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '4b30c102907740d8bad46dace08d4ef2'} {{(pid=80377) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 10:17:22.052421 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: [pid: 80377|app: 0|req: 1038/2064] 10.222.0.82 () {58 vars in 1224 bytes} [Tue Aug 2 10:17:21 2022] DELETE /identity/v3/users/4b30c102907740d8bad46dace08d4ef2 => generated 0 bytes in 307 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 02 10:17:22.057501 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f9b1bf5-4407-4f47-93bb-2247071236ae None None] REQUEST_METHOD: `POST` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:17:22.058082 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f9b1bf5-4407-4f47-93bb-2247071236ae None None] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:17:22.059423 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f9b1bf5-4407-4f47-93bb-2247071236ae None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:17:22.129759 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.auth.core [None req-2f9b1bf5-4407-4f47-93bb-2247071236ae None None] MFA Rules not processed for user `891005fa5ff74041b75747d16cc66798`. Rule list: `[]` (Enabled: `True`). {{(pid=80378) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 02 10:17:22.131581 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.fernet_utils [None req-2f9b1bf5-4407-4f47-93bb-2247071236ae None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80378) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 10:17:22.273528 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG dbcounter [-] [80378] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80378) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 10:17:22.330436 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 1027/2065] 10.222.0.82 () {58 vars in 958 bytes} [Tue Aug 2 10:17:22 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 274 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 02 10:17:22.348667 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e2169c1-975f-494e-91ac-0ea4c8035cac admin admin] Authenticating user token {{(pid=80377) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:17:22.350464 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.fernet_utils [None req-3e2169c1-975f-494e-91ac-0ea4c8035cac admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80377) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 10:17:22.447920 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG dbcounter [-] [80377] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80377) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 10:17:22.549697 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e2169c1-975f-494e-91ac-0ea4c8035cac admin admin] Validating token access rules against request {{(pid=80377) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:17:22.563116 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c98838db-ec8b-4860-af85-8991d8e0b628 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '33b87170f0634b67bce60750847b5cbe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f1b38ebb804c41c69772c998aae72c41', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80377) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:17:22.563943 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c98838db-ec8b-4860-af85-8991d8e0b628 service neutron] REQUEST_METHOD: `GET` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:17:22.564305 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c98838db-ec8b-4860-af85-8991d8e0b628 service neutron] SCRIPT_NAME: `/identity` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:17:22.564607 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c98838db-ec8b-4860-af85-8991d8e0b628 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:17:22.593974 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c98838db-ec8b-4860-af85-8991d8e0b628 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:17:22.597363 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c98838db-ec8b-4860-af85-8991d8e0b628 service neutron] RBAC: Authorization granted {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:17:22.634396 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: [pid: 80377|app: 0|req: 1039/2066] 10.222.0.82 () {60 vars in 1365 bytes} [Tue Aug 2 10:17:22 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 286 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 02 10:17:22.737159 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f9b1bf5-4407-4f47-93bb-2247071236ae None None] Authenticating user token {{(pid=80378) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:17:22.738813 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.fernet_utils [None req-2f9b1bf5-4407-4f47-93bb-2247071236ae None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80378) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 10:17:22.776395 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f9b1bf5-4407-4f47-93bb-2247071236ae None None] Validating token access rules against request {{(pid=80378) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:17:22.786902 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf195ad3-1935-444c-8733-fee4e14309cb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '891005fa5ff74041b75747d16cc66798', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7047912c5acd4adeba5d87c3bf895e39', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80378) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:17:22.787570 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf195ad3-1935-444c-8733-fee4e14309cb admin admin] REQUEST_METHOD: `DELETE` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:17:22.787849 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf195ad3-1935-444c-8733-fee4e14309cb admin admin] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:17:22.788351 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf195ad3-1935-444c-8733-fee4e14309cb admin admin] PATH_INFO: `/v3/users/805781d7bb864aed9d311bc861e9bd4f` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:17:22.790606 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf195ad3-1935-444c-8733-fee4e14309cb admin admin] RBAC: Authorizing `identity:delete_user(user_id=805781d7bb864aed9d311bc861e9bd4f)` {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:17:22.793142 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf195ad3-1935-444c-8733-fee4e14309cb admin admin] RBAC: Authorization granted {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:17:22.919700 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.notifications [None req-cf195ad3-1935-444c-8733-fee4e14309cb admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '805781d7bb864aed9d311bc861e9bd4f'} {{(pid=80378) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 10:17:22.937392 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.notifications [None req-cf195ad3-1935-444c-8733-fee4e14309cb admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '805781d7bb864aed9d311bc861e9bd4f'} {{(pid=80378) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 10:17:22.956098 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.notifications [None req-cf195ad3-1935-444c-8733-fee4e14309cb admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '805781d7bb864aed9d311bc861e9bd4f'} {{(pid=80378) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 10:17:22.957030 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.notifications [None req-cf195ad3-1935-444c-8733-fee4e14309cb admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '805781d7bb864aed9d311bc861e9bd4f'} {{(pid=80378) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 10:17:22.984574 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.notifications [None req-cf195ad3-1935-444c-8733-fee4e14309cb admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '805781d7bb864aed9d311bc861e9bd4f'} {{(pid=80378) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 10:17:22.993892 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 1028/2067] 10.222.0.82 () {58 vars in 1224 bytes} [Tue Aug 2 10:17:22 2022] DELETE /identity/v3/users/805781d7bb864aed9d311bc861e9bd4f => generated 0 bytes in 258 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 02 10:17:22.998901 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8d6f746-0a5f-4ded-9d5f-e05323e59ea9 None None] REQUEST_METHOD: `POST` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:17:22.999404 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8d6f746-0a5f-4ded-9d5f-e05323e59ea9 None None] SCRIPT_NAME: `/identity` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:17:22.999480 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8d6f746-0a5f-4ded-9d5f-e05323e59ea9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:17:23.043284 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.auth.core [None req-c8d6f746-0a5f-4ded-9d5f-e05323e59ea9 None None] MFA Rules not processed for user `891005fa5ff74041b75747d16cc66798`. Rule list: `[]` (Enabled: `True`). {{(pid=80377) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 02 10:17:23.046520 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.fernet_utils [None req-c8d6f746-0a5f-4ded-9d5f-e05323e59ea9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80377) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 10:17:23.196551 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: [pid: 80377|app: 0|req: 1040/2068] 10.222.0.82 () {58 vars in 958 bytes} [Tue Aug 2 10:17:22 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 199 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 02 10:17:23.219929 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf195ad3-1935-444c-8733-fee4e14309cb admin admin] Authenticating user token {{(pid=80378) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:17:23.221659 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.fernet_utils [None req-cf195ad3-1935-444c-8733-fee4e14309cb admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80378) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 10:17:23.264305 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8d6f746-0a5f-4ded-9d5f-e05323e59ea9 None None] Authenticating user token {{(pid=80377) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:17:23.274007 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.fernet_utils [None req-c8d6f746-0a5f-4ded-9d5f-e05323e59ea9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80377) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 10:17:23.302606 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8d6f746-0a5f-4ded-9d5f-e05323e59ea9 None None] Validating token access rules against request {{(pid=80377) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:17:23.315815 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd3129c4-53c6-4eaa-97e6-0ae4e2e805ce admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '891005fa5ff74041b75747d16cc66798', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7047912c5acd4adeba5d87c3bf895e39', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80377) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:17:23.317029 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd3129c4-53c6-4eaa-97e6-0ae4e2e805ce admin admin] REQUEST_METHOD: `DELETE` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:17:23.317588 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd3129c4-53c6-4eaa-97e6-0ae4e2e805ce admin admin] SCRIPT_NAME: `/identity` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:17:23.318256 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd3129c4-53c6-4eaa-97e6-0ae4e2e805ce admin admin] PATH_INFO: `/v3/users/1eb84c516a1c4a75a0a4587c95f43f5c` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:17:23.320468 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd3129c4-53c6-4eaa-97e6-0ae4e2e805ce admin admin] RBAC: Authorizing `identity:delete_user(user_id=1eb84c516a1c4a75a0a4587c95f43f5c)` {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:17:23.323247 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf195ad3-1935-444c-8733-fee4e14309cb admin admin] Validating token access rules against request {{(pid=80378) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:17:23.324515 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd3129c4-53c6-4eaa-97e6-0ae4e2e805ce admin admin] RBAC: Authorization granted {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:17:23.340323 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00f6a9de-b86f-4ef6-907d-21c5afe89ede service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '33b87170f0634b67bce60750847b5cbe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f1b38ebb804c41c69772c998aae72c41', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80378) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:17:23.341614 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00f6a9de-b86f-4ef6-907d-21c5afe89ede service neutron] REQUEST_METHOD: `GET` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:17:23.341945 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00f6a9de-b86f-4ef6-907d-21c5afe89ede service neutron] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:17:23.342220 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00f6a9de-b86f-4ef6-907d-21c5afe89ede service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:17:23.356527 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00f6a9de-b86f-4ef6-907d-21c5afe89ede service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:17:23.358638 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00f6a9de-b86f-4ef6-907d-21c5afe89ede service neutron] RBAC: Authorization granted {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:17:23.373083 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 1029/2069] 10.222.0.82 () {60 vars in 1365 bytes} [Tue Aug 2 10:17:23 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 153 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 02 10:17:23.375651 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00f6a9de-b86f-4ef6-907d-21c5afe89ede service neutron] Authenticating user token {{(pid=80378) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:17:23.377952 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.fernet_utils [None req-00f6a9de-b86f-4ef6-907d-21c5afe89ede service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80378) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 10:17:23.397499 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00f6a9de-b86f-4ef6-907d-21c5afe89ede service neutron] Validating token access rules against request {{(pid=80378) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:17:23.407455 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-274d87eb-2ce1-4a2d-8c25-7620627de97d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '891005fa5ff74041b75747d16cc66798', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7047912c5acd4adeba5d87c3bf895e39', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80378) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:17:23.408389 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-274d87eb-2ce1-4a2d-8c25-7620627de97d admin admin] REQUEST_METHOD: `DELETE` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:17:23.408909 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-274d87eb-2ce1-4a2d-8c25-7620627de97d admin admin] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:17:23.409361 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-274d87eb-2ce1-4a2d-8c25-7620627de97d admin admin] PATH_INFO: `/v3/projects/c245a79031d74d889e50a87f958a7585` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:17:23.411442 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-274d87eb-2ce1-4a2d-8c25-7620627de97d admin admin] RBAC: Authorizing `identity:delete_project(project_id=c245a79031d74d889e50a87f958a7585)` {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:17:23.414871 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-274d87eb-2ce1-4a2d-8c25-7620627de97d admin admin] RBAC: Authorization granted {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:17:23.430340 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.notifications [None req-dd3129c4-53c6-4eaa-97e6-0ae4e2e805ce admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '1eb84c516a1c4a75a0a4587c95f43f5c'} {{(pid=80377) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 10:17:23.448090 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.notifications [None req-dd3129c4-53c6-4eaa-97e6-0ae4e2e805ce admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '1eb84c516a1c4a75a0a4587c95f43f5c'} {{(pid=80377) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 10:17:23.462702 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.notifications [None req-dd3129c4-53c6-4eaa-97e6-0ae4e2e805ce admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '1eb84c516a1c4a75a0a4587c95f43f5c'} {{(pid=80377) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 10:17:23.463507 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.notifications [None req-dd3129c4-53c6-4eaa-97e6-0ae4e2e805ce admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '1eb84c516a1c4a75a0a4587c95f43f5c'} {{(pid=80377) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 10:17:23.509057 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.notifications [None req-dd3129c4-53c6-4eaa-97e6-0ae4e2e805ce admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '1eb84c516a1c4a75a0a4587c95f43f5c'} {{(pid=80377) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 10:17:23.516215 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.notifications [None req-274d87eb-2ce1-4a2d-8c25-7620627de97d admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'c245a79031d74d889e50a87f958a7585', 'request_id': 'req-274d87eb-2ce1-4a2d-8c25-7620627de97d'} {{(pid=80378) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 10:17:23.519263 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG dbcounter [-] [80378] Writing DB stats keystone:SELECT=85,keystone:DELETE=14,keystone:INSERT=1 {{(pid=80378) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 10:17:23.530140 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.notifications [None req-274d87eb-2ce1-4a2d-8c25-7620627de97d admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'c245a79031d74d889e50a87f958a7585', 'request_id': 'req-274d87eb-2ce1-4a2d-8c25-7620627de97d'} {{(pid=80378) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 10:17:23.539036 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: [pid: 80377|app: 0|req: 1041/2070] 10.222.0.82 () {58 vars in 1224 bytes} [Tue Aug 2 10:17:23 2022] DELETE /identity/v3/users/1eb84c516a1c4a75a0a4587c95f43f5c => generated 0 bytes in 273 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 02 10:17:23.540155 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.notifications [None req-274d87eb-2ce1-4a2d-8c25-7620627de97d admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'c245a79031d74d889e50a87f958a7585', 'request_id': 'req-274d87eb-2ce1-4a2d-8c25-7620627de97d'} {{(pid=80378) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 10:17:23.544860 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf7dbe4a-f618-4603-a4aa-f3e62fb229f0 None None] REQUEST_METHOD: `POST` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:17:23.548449 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf7dbe4a-f618-4603-a4aa-f3e62fb229f0 None None] SCRIPT_NAME: `/identity` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:17:23.549926 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf7dbe4a-f618-4603-a4aa-f3e62fb229f0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:17:23.576599 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.notifications [None req-274d87eb-2ce1-4a2d-8c25-7620627de97d admin admin] The token cache is being invalidate because project c245a79031d74d889e50a87f958a7585 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80378) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 02 10:17:23.576977 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.notifications [None req-274d87eb-2ce1-4a2d-8c25-7620627de97d admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80378) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 10:17:23.578093 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.notifications [None req-274d87eb-2ce1-4a2d-8c25-7620627de97d admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80378) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 10:17:23.582298 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 1030/2071] 10.222.0.82 () {58 vars in 1233 bytes} [Tue Aug 2 10:17:23 2022] DELETE /identity/v3/projects/c245a79031d74d889e50a87f958a7585 => generated 0 bytes in 208 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 02 10:17:23.586124 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-274d87eb-2ce1-4a2d-8c25-7620627de97d admin admin] Authenticating user token {{(pid=80378) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:17:23.594352 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.fernet_utils [None req-274d87eb-2ce1-4a2d-8c25-7620627de97d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80378) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 10:17:23.612035 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.auth.core [None req-cf7dbe4a-f618-4603-a4aa-f3e62fb229f0 None None] MFA Rules not processed for user `891005fa5ff74041b75747d16cc66798`. Rule list: `[]` (Enabled: `True`). {{(pid=80377) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 02 10:17:23.613353 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.fernet_utils [None req-cf7dbe4a-f618-4603-a4aa-f3e62fb229f0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80377) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 10:17:23.674601 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG dbcounter [-] [80377] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80377) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 10:17:23.709397 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: [pid: 80377|app: 0|req: 1042/2072] 10.222.0.82 () {58 vars in 958 bytes} [Tue Aug 2 10:17:23 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 170 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 02 10:17:23.732127 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-274d87eb-2ce1-4a2d-8c25-7620627de97d admin admin] Validating token access rules against request {{(pid=80378) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:17:23.734258 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf7dbe4a-f618-4603-a4aa-f3e62fb229f0 None None] Authenticating user token {{(pid=80377) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:17:23.739709 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88631024-96e4-48d0-a491-cb0be2af50cb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '891005fa5ff74041b75747d16cc66798', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7047912c5acd4adeba5d87c3bf895e39', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80378) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:17:23.740615 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88631024-96e4-48d0-a491-cb0be2af50cb admin admin] REQUEST_METHOD: `DELETE` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:17:23.741043 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88631024-96e4-48d0-a491-cb0be2af50cb admin admin] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:17:23.741464 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88631024-96e4-48d0-a491-cb0be2af50cb admin admin] PATH_INFO: `/v3/users/51540feb0a154a82a116fc6d91cf5d46` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:17:23.742947 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88631024-96e4-48d0-a491-cb0be2af50cb admin admin] RBAC: Authorizing `identity:delete_user(user_id=51540feb0a154a82a116fc6d91cf5d46)` {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:17:23.743214 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.fernet_utils [None req-cf7dbe4a-f618-4603-a4aa-f3e62fb229f0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80377) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 10:17:23.745481 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88631024-96e4-48d0-a491-cb0be2af50cb admin admin] RBAC: Authorization granted {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:17:23.836251 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf7dbe4a-f618-4603-a4aa-f3e62fb229f0 None None] Validating token access rules against request {{(pid=80377) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:17:23.848220 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fef6568-ced2-48e6-babb-61c73831886b service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '33b87170f0634b67bce60750847b5cbe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f1b38ebb804c41c69772c998aae72c41', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80377) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:17:23.848220 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fef6568-ced2-48e6-babb-61c73831886b service neutron] REQUEST_METHOD: `GET` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:17:23.848220 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fef6568-ced2-48e6-babb-61c73831886b service neutron] SCRIPT_NAME: `/identity` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:17:23.848220 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fef6568-ced2-48e6-babb-61c73831886b service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:17:23.877837 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5fef6568-ced2-48e6-babb-61c73831886b service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:17:23.880353 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5fef6568-ced2-48e6-babb-61c73831886b service neutron] RBAC: Authorization granted {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:17:23.889147 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.notifications [None req-88631024-96e4-48d0-a491-cb0be2af50cb admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '51540feb0a154a82a116fc6d91cf5d46'} {{(pid=80378) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 10:17:23.893078 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: [pid: 80377|app: 0|req: 1043/2073] 10.222.0.82 () {60 vars in 1365 bytes} [Tue Aug 2 10:17:23 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 169 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 02 10:17:23.911675 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.notifications [None req-88631024-96e4-48d0-a491-cb0be2af50cb admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '51540feb0a154a82a116fc6d91cf5d46'} {{(pid=80378) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 10:17:23.938902 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.notifications [None req-88631024-96e4-48d0-a491-cb0be2af50cb admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '51540feb0a154a82a116fc6d91cf5d46'} {{(pid=80378) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 10:17:23.941199 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.notifications [None req-88631024-96e4-48d0-a491-cb0be2af50cb admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '51540feb0a154a82a116fc6d91cf5d46'} {{(pid=80378) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 10:17:24.006019 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.notifications [None req-88631024-96e4-48d0-a491-cb0be2af50cb admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '51540feb0a154a82a116fc6d91cf5d46'} {{(pid=80378) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 10:17:24.026379 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 1031/2074] 10.222.0.82 () {58 vars in 1224 bytes} [Tue Aug 2 10:17:23 2022] DELETE /identity/v3/users/51540feb0a154a82a116fc6d91cf5d46 => generated 0 bytes in 440 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 02 10:17:24.434702 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fef6568-ced2-48e6-babb-61c73831886b service neutron] Authenticating user token {{(pid=80377) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:17:24.437814 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.fernet_utils [None req-5fef6568-ced2-48e6-babb-61c73831886b service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80377) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 10:17:24.453769 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fef6568-ced2-48e6-babb-61c73831886b service neutron] Validating token access rules against request {{(pid=80377) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:17:24.515079 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88631024-96e4-48d0-a491-cb0be2af50cb admin admin] Authenticating user token {{(pid=80378) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:17:24.517101 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.fernet_utils [None req-88631024-96e4-48d0-a491-cb0be2af50cb admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80378) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 10:17:24.557812 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22abf816-d3da-4a90-a7bb-de0fcf6bfcdd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '891005fa5ff74041b75747d16cc66798', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7047912c5acd4adeba5d87c3bf895e39', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80377) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:17:24.558621 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22abf816-d3da-4a90-a7bb-de0fcf6bfcdd admin admin] REQUEST_METHOD: `DELETE` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:17:24.558971 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22abf816-d3da-4a90-a7bb-de0fcf6bfcdd admin admin] SCRIPT_NAME: `/identity` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:17:24.559318 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22abf816-d3da-4a90-a7bb-de0fcf6bfcdd admin admin] PATH_INFO: `/v3/projects/abddcee3849944c8a1d10ed888a4720e` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:17:24.561250 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22abf816-d3da-4a90-a7bb-de0fcf6bfcdd admin admin] RBAC: Authorizing `identity:delete_project(project_id=abddcee3849944c8a1d10ed888a4720e)` {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:17:24.563820 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22abf816-d3da-4a90-a7bb-de0fcf6bfcdd admin admin] RBAC: Authorization granted {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:17:24.657881 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.notifications [None req-22abf816-d3da-4a90-a7bb-de0fcf6bfcdd admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'abddcee3849944c8a1d10ed888a4720e', 'request_id': 'req-22abf816-d3da-4a90-a7bb-de0fcf6bfcdd'} {{(pid=80377) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 10:17:24.683553 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.notifications [None req-22abf816-d3da-4a90-a7bb-de0fcf6bfcdd admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'abddcee3849944c8a1d10ed888a4720e', 'request_id': 'req-22abf816-d3da-4a90-a7bb-de0fcf6bfcdd'} {{(pid=80377) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 10:17:24.697827 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.notifications [None req-22abf816-d3da-4a90-a7bb-de0fcf6bfcdd admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'abddcee3849944c8a1d10ed888a4720e', 'request_id': 'req-22abf816-d3da-4a90-a7bb-de0fcf6bfcdd'} {{(pid=80377) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 10:17:24.723497 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.notifications [None req-22abf816-d3da-4a90-a7bb-de0fcf6bfcdd admin admin] The token cache is being invalidate because project abddcee3849944c8a1d10ed888a4720e was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80377) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 02 10:17:24.723944 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.notifications [None req-22abf816-d3da-4a90-a7bb-de0fcf6bfcdd admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80377) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 10:17:24.724753 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.notifications [None req-22abf816-d3da-4a90-a7bb-de0fcf6bfcdd admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80377) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 10:17:24.729745 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: [pid: 80377|app: 0|req: 1044/2075] 10.222.0.82 () {58 vars in 1233 bytes} [Tue Aug 2 10:17:24 2022] DELETE /identity/v3/projects/abddcee3849944c8a1d10ed888a4720e => generated 0 bytes in 293 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 02 10:17:24.729927 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88631024-96e4-48d0-a491-cb0be2af50cb admin admin] Validating token access rules against request {{(pid=80378) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:17:24.731158 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22abf816-d3da-4a90-a7bb-de0fcf6bfcdd admin admin] Authenticating user token {{(pid=80377) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:17:24.733017 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.fernet_utils [None req-22abf816-d3da-4a90-a7bb-de0fcf6bfcdd admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80377) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 10:17:24.742201 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG dbcounter [-] [80378] Writing DB stats keystone:SELECT=85,keystone:DELETE=13,keystone:INSERT=2 {{(pid=80378) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 10:17:24.789710 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG dbcounter [-] [80377] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80377) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 10:17:24.882838 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f2c9ab6-b7db-488c-9293-826797ae64ad admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '891005fa5ff74041b75747d16cc66798', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7047912c5acd4adeba5d87c3bf895e39', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80378) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:17:24.883517 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f2c9ab6-b7db-488c-9293-826797ae64ad admin admin] REQUEST_METHOD: `DELETE` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:17:24.883805 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f2c9ab6-b7db-488c-9293-826797ae64ad admin admin] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:17:24.884083 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f2c9ab6-b7db-488c-9293-826797ae64ad admin admin] PATH_INFO: `/v3/projects/4dcc9e0034a349beb0c7b1cb91d54651` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:17:24.885716 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f2c9ab6-b7db-488c-9293-826797ae64ad admin admin] RBAC: Authorizing `identity:delete_project(project_id=4dcc9e0034a349beb0c7b1cb91d54651)` {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:17:24.889180 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f2c9ab6-b7db-488c-9293-826797ae64ad admin admin] RBAC: Authorization granted {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:17:24.896122 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22abf816-d3da-4a90-a7bb-de0fcf6bfcdd admin admin] Validating token access rules against request {{(pid=80377) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:17:24.905169 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eef9fafa-b22f-4ea8-a596-2d4ced5f0737 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '891005fa5ff74041b75747d16cc66798', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7047912c5acd4adeba5d87c3bf895e39', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80377) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:17:24.906212 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eef9fafa-b22f-4ea8-a596-2d4ced5f0737 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:17:24.906598 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eef9fafa-b22f-4ea8-a596-2d4ced5f0737 admin admin] SCRIPT_NAME: `/identity` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:17:24.907599 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eef9fafa-b22f-4ea8-a596-2d4ced5f0737 admin admin] PATH_INFO: `/v3/users/c4e2fa69c37b47219e1c9b4d2590a12c` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:17:24.909974 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eef9fafa-b22f-4ea8-a596-2d4ced5f0737 admin admin] RBAC: Authorizing `identity:delete_user(user_id=c4e2fa69c37b47219e1c9b4d2590a12c)` {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:17:24.912771 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eef9fafa-b22f-4ea8-a596-2d4ced5f0737 admin admin] RBAC: Authorization granted {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:17:24.952506 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.notifications [None req-1f2c9ab6-b7db-488c-9293-826797ae64ad admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '4dcc9e0034a349beb0c7b1cb91d54651', 'request_id': 'req-1f2c9ab6-b7db-488c-9293-826797ae64ad'} {{(pid=80378) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 10:17:24.960046 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.notifications [None req-1f2c9ab6-b7db-488c-9293-826797ae64ad admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '4dcc9e0034a349beb0c7b1cb91d54651', 'request_id': 'req-1f2c9ab6-b7db-488c-9293-826797ae64ad'} {{(pid=80378) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 10:17:24.967366 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.notifications [None req-1f2c9ab6-b7db-488c-9293-826797ae64ad admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '4dcc9e0034a349beb0c7b1cb91d54651', 'request_id': 'req-1f2c9ab6-b7db-488c-9293-826797ae64ad'} {{(pid=80378) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 10:17:24.988108 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.notifications [None req-eef9fafa-b22f-4ea8-a596-2d4ced5f0737 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'c4e2fa69c37b47219e1c9b4d2590a12c'} {{(pid=80377) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 10:17:24.990920 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.notifications [None req-1f2c9ab6-b7db-488c-9293-826797ae64ad admin admin] The token cache is being invalidate because project 4dcc9e0034a349beb0c7b1cb91d54651 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80378) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 02 10:17:24.991516 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.notifications [None req-1f2c9ab6-b7db-488c-9293-826797ae64ad admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80378) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 10:17:24.992443 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.notifications [None req-1f2c9ab6-b7db-488c-9293-826797ae64ad admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80378) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 10:17:24.995465 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 1032/2076] 10.222.0.82 () {58 vars in 1233 bytes} [Tue Aug 2 10:17:24 2022] DELETE /identity/v3/projects/4dcc9e0034a349beb0c7b1cb91d54651 => generated 0 bytes in 482 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 02 10:17:25.000481 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81167130-228c-41f7-8914-5efe7fe62028 None None] REQUEST_METHOD: `POST` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:17:25.000726 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81167130-228c-41f7-8914-5efe7fe62028 None None] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:17:25.001017 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81167130-228c-41f7-8914-5efe7fe62028 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:17:25.002019 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.notifications [None req-eef9fafa-b22f-4ea8-a596-2d4ced5f0737 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'c4e2fa69c37b47219e1c9b4d2590a12c'} {{(pid=80377) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 10:17:25.012715 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.notifications [None req-eef9fafa-b22f-4ea8-a596-2d4ced5f0737 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'c4e2fa69c37b47219e1c9b4d2590a12c'} {{(pid=80377) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 10:17:25.014438 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.notifications [None req-eef9fafa-b22f-4ea8-a596-2d4ced5f0737 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'c4e2fa69c37b47219e1c9b4d2590a12c'} {{(pid=80377) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 10:17:25.023397 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.auth.core [None req-81167130-228c-41f7-8914-5efe7fe62028 None None] MFA Rules not processed for user `891005fa5ff74041b75747d16cc66798`. Rule list: `[]` (Enabled: `True`). {{(pid=80378) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 02 10:17:25.024827 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.fernet_utils [None req-81167130-228c-41f7-8914-5efe7fe62028 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80378) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 10:17:25.032741 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.notifications [None req-eef9fafa-b22f-4ea8-a596-2d4ced5f0737 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'c4e2fa69c37b47219e1c9b4d2590a12c'} {{(pid=80377) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 10:17:25.042885 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: [pid: 80377|app: 0|req: 1045/2077] 10.222.0.82 () {58 vars in 1224 bytes} [Tue Aug 2 10:17:24 2022] DELETE /identity/v3/users/c4e2fa69c37b47219e1c9b4d2590a12c => generated 0 bytes in 312 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 02 10:17:25.117024 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 1033/2078] 10.222.0.82 () {58 vars in 958 bytes} [Tue Aug 2 10:17:24 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 112 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 02 10:17:25.119919 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a6a9c1f-16e1-4b15-b8d7-6beaf34b758a None None] REQUEST_METHOD: `POST` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:17:25.120444 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a6a9c1f-16e1-4b15-b8d7-6beaf34b758a None None] SCRIPT_NAME: `/identity` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:17:25.120739 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a6a9c1f-16e1-4b15-b8d7-6beaf34b758a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:17:25.152454 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.auth.core [None req-5a6a9c1f-16e1-4b15-b8d7-6beaf34b758a None None] MFA Rules not processed for user `891005fa5ff74041b75747d16cc66798`. Rule list: `[]` (Enabled: `True`). {{(pid=80377) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 02 10:17:25.154043 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.fernet_utils [None req-5a6a9c1f-16e1-4b15-b8d7-6beaf34b758a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80377) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 10:17:25.240498 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: [pid: 80377|app: 0|req: 1046/2079] 10.222.0.82 () {58 vars in 958 bytes} [Tue Aug 2 10:17:25 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 122 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 02 10:17:25.244991 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81167130-228c-41f7-8914-5efe7fe62028 None None] Authenticating user token {{(pid=80378) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:17:25.258909 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81167130-228c-41f7-8914-5efe7fe62028 None None] Validating token access rules against request {{(pid=80378) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:17:25.269690 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9b56069-81e6-46ee-9cf7-cef288517894 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '891005fa5ff74041b75747d16cc66798', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7047912c5acd4adeba5d87c3bf895e39', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80378) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:17:25.270182 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9b56069-81e6-46ee-9cf7-cef288517894 admin admin] REQUEST_METHOD: `GET` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:17:25.270954 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9b56069-81e6-46ee-9cf7-cef288517894 admin admin] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:17:25.270954 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9b56069-81e6-46ee-9cf7-cef288517894 admin admin] PATH_INFO: `/v3/domains` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:17:25.271605 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9b56069-81e6-46ee-9cf7-cef288517894 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80378) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 02 10:17:25.272333 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9b56069-81e6-46ee-9cf7-cef288517894 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:17:25.288132 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9b56069-81e6-46ee-9cf7-cef288517894 admin admin] RBAC: Authorization granted {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:17:25.333482 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd3d9c6c-5c12-490f-ade4-addec67c806e None None] REQUEST_METHOD: `POST` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:17:25.333776 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd3d9c6c-5c12-490f-ade4-addec67c806e None None] SCRIPT_NAME: `/identity` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:17:25.334082 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd3d9c6c-5c12-490f-ade4-addec67c806e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:17:25.350374 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 1034/2080] 10.222.0.82 () {58 vars in 1153 bytes} [Tue Aug 2 10:17:25 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 101 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 02 10:17:25.352653 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9b56069-81e6-46ee-9cf7-cef288517894 admin admin] Authenticating user token {{(pid=80378) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:17:25.378407 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9b56069-81e6-46ee-9cf7-cef288517894 admin admin] Validating token access rules against request {{(pid=80378) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:17:25.392840 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b1a1148-9446-4365-84b3-540ef36cd83f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '891005fa5ff74041b75747d16cc66798', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7047912c5acd4adeba5d87c3bf895e39', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80378) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:17:25.393436 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b1a1148-9446-4365-84b3-540ef36cd83f admin admin] REQUEST_METHOD: `POST` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:17:25.393727 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b1a1148-9446-4365-84b3-540ef36cd83f admin admin] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:17:25.394052 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b1a1148-9446-4365-84b3-540ef36cd83f admin admin] PATH_INFO: `/v3/projects` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:17:25.394969 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b1a1148-9446-4365-84b3-540ef36cd83f admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:17:25.396806 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b1a1148-9446-4365-84b3-540ef36cd83f admin admin] RBAC: Authorization granted {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:17:25.400369 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.auth.core [None req-fd3d9c6c-5c12-490f-ade4-addec67c806e None None] MFA Rules not processed for user `54bb78d9af854abf845b47f2bb925b74`. Rule list: `[]` (Enabled: `True`). {{(pid=80377) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 02 10:17:25.401616 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.fernet_utils [None req-fd3d9c6c-5c12-490f-ade4-addec67c806e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80377) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 10:17:25.449514 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG dbcounter [-] [80377] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80377) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 10:17:25.467456 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 1035/2081] 10.222.0.82 () {60 vars in 1153 bytes} [Tue Aug 2 10:17:25 2022] POST /identity/v3/projects => generated 375 bytes in 116 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 02 10:17:25.471668 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b1a1148-9446-4365-84b3-540ef36cd83f admin admin] Authenticating user token {{(pid=80378) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:17:25.480599 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG dbcounter [-] [80378] Writing DB stats keystone:SELECT=91,keystone:DELETE=7,keystone:INSERT=2 {{(pid=80378) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 10:17:25.574155 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: [pid: 80377|app: 0|req: 1047/2082] 10.222.0.82 () {58 vars in 958 bytes} [Tue Aug 2 10:17:25 2022] POST /identity/v3/auth/tokens => generated 3262 bytes in 241 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 02 10:17:25.576210 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd3d9c6c-5c12-490f-ade4-addec67c806e None None] Authenticating user token {{(pid=80377) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:17:25.578581 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.fernet_utils [None req-fd3d9c6c-5c12-490f-ade4-addec67c806e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80377) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 10:17:25.581064 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b1a1148-9446-4365-84b3-540ef36cd83f admin admin] Validating token access rules against request {{(pid=80378) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:17:25.586985 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-437798c0-056e-448d-84c5-bf0854481f27 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '891005fa5ff74041b75747d16cc66798', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7047912c5acd4adeba5d87c3bf895e39', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80378) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:17:25.587789 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-437798c0-056e-448d-84c5-bf0854481f27 admin admin] REQUEST_METHOD: `POST` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:17:25.588077 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-437798c0-056e-448d-84c5-bf0854481f27 admin admin] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:17:25.588496 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-437798c0-056e-448d-84c5-bf0854481f27 admin admin] PATH_INFO: `/v3/users` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:17:25.592144 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-437798c0-056e-448d-84c5-bf0854481f27 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:17:25.592144 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-437798c0-056e-448d-84c5-bf0854481f27 admin admin] RBAC: Authorization granted {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:17:25.598247 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd3d9c6c-5c12-490f-ade4-addec67c806e None None] Validating token access rules against request {{(pid=80377) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:17:25.604069 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a73550d-34a3-4cae-941a-5b2882f16e18 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '891005fa5ff74041b75747d16cc66798', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7047912c5acd4adeba5d87c3bf895e39', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80377) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:17:25.604858 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a73550d-34a3-4cae-941a-5b2882f16e18 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:17:25.605196 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a73550d-34a3-4cae-941a-5b2882f16e18 admin admin] SCRIPT_NAME: `/identity` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:17:25.605538 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a73550d-34a3-4cae-941a-5b2882f16e18 admin admin] PATH_INFO: `/v3/projects/ca5ccb9165c74f3a8bd3101622cf79b0` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:17:25.608362 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a73550d-34a3-4cae-941a-5b2882f16e18 admin admin] RBAC: Authorizing `identity:delete_project(project_id=ca5ccb9165c74f3a8bd3101622cf79b0)` {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:17:25.610720 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a73550d-34a3-4cae-941a-5b2882f16e18 admin admin] RBAC: Authorization granted {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:17:25.619155 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 1036/2083] 10.222.0.82 () {60 vars in 1144 bytes} [Tue Aug 2 10:17:25 2022] POST /identity/v3/users => generated 288 bytes in 148 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 02 10:17:25.624757 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-437798c0-056e-448d-84c5-bf0854481f27 admin admin] Authenticating user token {{(pid=80378) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:17:25.627408 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.fernet_utils [None req-437798c0-056e-448d-84c5-bf0854481f27 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80378) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 10:17:25.656127 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-437798c0-056e-448d-84c5-bf0854481f27 admin admin] Validating token access rules against request {{(pid=80378) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:17:25.672304 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1eb37d6-e47b-4eb6-a5c0-3966cd002670 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '891005fa5ff74041b75747d16cc66798', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7047912c5acd4adeba5d87c3bf895e39', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80378) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:17:25.673560 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1eb37d6-e47b-4eb6-a5c0-3966cd002670 admin admin] REQUEST_METHOD: `POST` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:17:25.673560 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1eb37d6-e47b-4eb6-a5c0-3966cd002670 admin admin] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:17:25.673750 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1eb37d6-e47b-4eb6-a5c0-3966cd002670 admin admin] PATH_INFO: `/v3/projects` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:17:25.674435 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1eb37d6-e47b-4eb6-a5c0-3966cd002670 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:17:25.676613 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1eb37d6-e47b-4eb6-a5c0-3966cd002670 admin admin] RBAC: Authorization granted {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:17:25.694698 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.notifications [None req-5a73550d-34a3-4cae-941a-5b2882f16e18 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'ca5ccb9165c74f3a8bd3101622cf79b0', 'request_id': 'req-5a73550d-34a3-4cae-941a-5b2882f16e18'} {{(pid=80377) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 10:17:25.712366 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.notifications [None req-5a73550d-34a3-4cae-941a-5b2882f16e18 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'ca5ccb9165c74f3a8bd3101622cf79b0', 'request_id': 'req-5a73550d-34a3-4cae-941a-5b2882f16e18'} {{(pid=80377) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 10:17:25.727319 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 1037/2084] 10.222.0.82 () {60 vars in 1153 bytes} [Tue Aug 2 10:17:25 2022] POST /identity/v3/projects => generated 381 bytes in 103 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 02 10:17:25.728438 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1eb37d6-e47b-4eb6-a5c0-3966cd002670 admin admin] Authenticating user token {{(pid=80378) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:17:25.731456 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.notifications [None req-5a73550d-34a3-4cae-941a-5b2882f16e18 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'ca5ccb9165c74f3a8bd3101622cf79b0', 'request_id': 'req-5a73550d-34a3-4cae-941a-5b2882f16e18'} {{(pid=80377) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 10:17:25.767089 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.notifications [None req-5a73550d-34a3-4cae-941a-5b2882f16e18 admin admin] The token cache is being invalidate because project ca5ccb9165c74f3a8bd3101622cf79b0 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80377) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 02 10:17:25.769721 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.notifications [None req-5a73550d-34a3-4cae-941a-5b2882f16e18 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80377) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 10:17:25.774244 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.notifications [None req-5a73550d-34a3-4cae-941a-5b2882f16e18 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80377) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 10:17:25.778178 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: [pid: 80377|app: 0|req: 1048/2085] 10.222.0.82 () {58 vars in 1233 bytes} [Tue Aug 2 10:17:25 2022] DELETE /identity/v3/projects/ca5ccb9165c74f3a8bd3101622cf79b0 => generated 0 bytes in 203 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 02 10:17:25.780179 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a73550d-34a3-4cae-941a-5b2882f16e18 admin admin] Authenticating user token {{(pid=80377) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:17:25.783469 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.fernet_utils [None req-5a73550d-34a3-4cae-941a-5b2882f16e18 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80377) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 10:17:25.826800 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1eb37d6-e47b-4eb6-a5c0-3966cd002670 admin admin] Validating token access rules against request {{(pid=80378) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:17:25.833944 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7da76650-fbf7-40a4-b76b-11627d8984da admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '891005fa5ff74041b75747d16cc66798', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7047912c5acd4adeba5d87c3bf895e39', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80378) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:17:25.834586 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7da76650-fbf7-40a4-b76b-11627d8984da admin admin] REQUEST_METHOD: `GET` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:17:25.834868 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7da76650-fbf7-40a4-b76b-11627d8984da admin admin] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:17:25.835140 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7da76650-fbf7-40a4-b76b-11627d8984da admin admin] PATH_INFO: `/v3/roles` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:17:25.836167 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7da76650-fbf7-40a4-b76b-11627d8984da admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:17:25.838271 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7da76650-fbf7-40a4-b76b-11627d8984da admin admin] RBAC: Authorization granted {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:17:25.879453 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 1038/2086] 10.222.0.82 () {58 vars in 1122 bytes} [Tue Aug 2 10:17:25 2022] GET /identity/v3/roles => generated 1420 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 02 10:17:25.879453 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f27c3525-dec5-4437-bc69-812aa746f0c5 None None] REQUEST_METHOD: `POST` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:17:25.879453 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f27c3525-dec5-4437-bc69-812aa746f0c5 None None] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:17:25.879453 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f27c3525-dec5-4437-bc69-812aa746f0c5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:17:25.907508 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.auth.core [None req-f27c3525-dec5-4437-bc69-812aa746f0c5 None None] MFA Rules not processed for user `891005fa5ff74041b75747d16cc66798`. Rule list: `[]` (Enabled: `True`). {{(pid=80378) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 02 10:17:25.908994 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.fernet_utils [None req-f27c3525-dec5-4437-bc69-812aa746f0c5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80378) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 10:17:25.911923 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a73550d-34a3-4cae-941a-5b2882f16e18 admin admin] Validating token access rules against request {{(pid=80377) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:17:25.918606 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5cc3b02-4089-44e1-85a8-c4593212597a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '891005fa5ff74041b75747d16cc66798', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7047912c5acd4adeba5d87c3bf895e39', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80377) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:17:25.928270 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 1039/2087] 10.222.0.82 () {58 vars in 958 bytes} [Tue Aug 2 10:17:25 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 42 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 02 10:17:25.928796 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5cc3b02-4089-44e1-85a8-c4593212597a admin admin] REQUEST_METHOD: `POST` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:17:25.928796 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5cc3b02-4089-44e1-85a8-c4593212597a admin admin] SCRIPT_NAME: `/identity` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:17:25.928796 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5cc3b02-4089-44e1-85a8-c4593212597a admin admin] PATH_INFO: `/v3/users` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:17:25.929040 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f27c3525-dec5-4437-bc69-812aa746f0c5 None None] Authenticating user token {{(pid=80378) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:17:25.929126 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5cc3b02-4089-44e1-85a8-c4593212597a admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:17:25.929222 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.fernet_utils [None req-f27c3525-dec5-4437-bc69-812aa746f0c5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80378) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 10:17:25.929309 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5cc3b02-4089-44e1-85a8-c4593212597a admin admin] RBAC: Authorization granted {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:17:25.949333 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f27c3525-dec5-4437-bc69-812aa746f0c5 None None] Validating token access rules against request {{(pid=80378) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:17:25.957474 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-976e6f40-67e5-49c2-8291-bee8fad28765 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '891005fa5ff74041b75747d16cc66798', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7047912c5acd4adeba5d87c3bf895e39', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80378) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:17:25.958170 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-976e6f40-67e5-49c2-8291-bee8fad28765 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:17:25.958318 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: [pid: 80377|app: 0|req: 1049/2088] 10.222.0.82 () {60 vars in 1144 bytes} [Tue Aug 2 10:17:25 2022] POST /identity/v3/users => generated 297 bytes in 180 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 02 10:17:25.959158 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5cc3b02-4089-44e1-85a8-c4593212597a admin admin] Authenticating user token {{(pid=80377) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:17:25.960854 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.fernet_utils [None req-f5cc3b02-4089-44e1-85a8-c4593212597a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80377) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 10:17:25.961606 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-976e6f40-67e5-49c2-8291-bee8fad28765 admin admin] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:17:25.962284 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-976e6f40-67e5-49c2-8291-bee8fad28765 admin admin] PATH_INFO: `/v3/projects/6ea1d92c402e4adda4cfdce12956d682` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:17:25.963721 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-976e6f40-67e5-49c2-8291-bee8fad28765 admin admin] RBAC: Authorizing `identity:delete_project(project_id=6ea1d92c402e4adda4cfdce12956d682)` {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:17:25.966516 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-976e6f40-67e5-49c2-8291-bee8fad28765 admin admin] RBAC: Authorization granted {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:17:25.977298 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5cc3b02-4089-44e1-85a8-c4593212597a admin admin] Validating token access rules against request {{(pid=80377) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:17:25.988133 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b41879b-543a-4968-aa4e-8406041a9181 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '891005fa5ff74041b75747d16cc66798', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7047912c5acd4adeba5d87c3bf895e39', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80377) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:17:25.989001 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b41879b-543a-4968-aa4e-8406041a9181 admin admin] REQUEST_METHOD: `GET` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:17:25.989453 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b41879b-543a-4968-aa4e-8406041a9181 admin admin] SCRIPT_NAME: `/identity` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:17:25.989809 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b41879b-543a-4968-aa4e-8406041a9181 admin admin] PATH_INFO: `/v3/roles` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:17:25.990862 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b41879b-543a-4968-aa4e-8406041a9181 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:17:25.994761 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b41879b-543a-4968-aa4e-8406041a9181 admin admin] RBAC: Authorization granted {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:17:26.012258 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: [pid: 80377|app: 0|req: 1050/2089] 10.222.0.82 () {58 vars in 1122 bytes} [Tue Aug 2 10:17:25 2022] GET /identity/v3/roles => generated 1420 bytes in 53 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 02 10:17:26.014520 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6bdfe64-187f-4464-99bb-7ec50170fc04 None None] REQUEST_METHOD: `POST` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:17:26.014809 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6bdfe64-187f-4464-99bb-7ec50170fc04 None None] SCRIPT_NAME: `/identity` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:17:26.015309 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6bdfe64-187f-4464-99bb-7ec50170fc04 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:17:26.055025 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.auth.core [None req-a6bdfe64-187f-4464-99bb-7ec50170fc04 None None] MFA Rules not processed for user `891005fa5ff74041b75747d16cc66798`. Rule list: `[]` (Enabled: `True`). {{(pid=80377) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 02 10:17:26.056411 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.fernet_utils [None req-a6bdfe64-187f-4464-99bb-7ec50170fc04 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80377) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 10:17:26.070705 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.notifications [None req-976e6f40-67e5-49c2-8291-bee8fad28765 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '6ea1d92c402e4adda4cfdce12956d682', 'request_id': 'req-976e6f40-67e5-49c2-8291-bee8fad28765'} {{(pid=80378) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 10:17:26.081366 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.notifications [None req-976e6f40-67e5-49c2-8291-bee8fad28765 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '6ea1d92c402e4adda4cfdce12956d682', 'request_id': 'req-976e6f40-67e5-49c2-8291-bee8fad28765'} {{(pid=80378) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 10:17:26.096373 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG dbcounter [-] [80378] Writing DB stats keystone:SELECT=92,keystone:INSERT=4,keystone:DELETE=4 {{(pid=80378) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 10:17:26.105339 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.notifications [None req-976e6f40-67e5-49c2-8291-bee8fad28765 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '6ea1d92c402e4adda4cfdce12956d682', 'request_id': 'req-976e6f40-67e5-49c2-8291-bee8fad28765'} {{(pid=80378) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 10:17:26.116456 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG dbcounter [-] [80377] Writing DB stats keystone:SELECT=89,keystone:DELETE=7,keystone:INSERT=4 {{(pid=80377) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 10:17:26.132930 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.notifications [None req-976e6f40-67e5-49c2-8291-bee8fad28765 admin admin] The token cache is being invalidate because project 6ea1d92c402e4adda4cfdce12956d682 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80378) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 02 10:17:26.133238 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.notifications [None req-976e6f40-67e5-49c2-8291-bee8fad28765 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80378) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 10:17:26.134441 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.notifications [None req-976e6f40-67e5-49c2-8291-bee8fad28765 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80378) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 10:17:26.142906 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 1040/2090] 10.222.0.82 () {58 vars in 1233 bytes} [Tue Aug 2 10:17:25 2022] DELETE /identity/v3/projects/6ea1d92c402e4adda4cfdce12956d682 => generated 0 bytes in 220 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 02 10:17:26.142906 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-976e6f40-67e5-49c2-8291-bee8fad28765 admin admin] Authenticating user token {{(pid=80378) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:17:26.145599 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.fernet_utils [None req-976e6f40-67e5-49c2-8291-bee8fad28765 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80378) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 10:17:26.197942 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: [pid: 80377|app: 0|req: 1051/2091] 10.222.0.82 () {58 vars in 958 bytes} [Tue Aug 2 10:17:26 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 182 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 02 10:17:26.197942 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6bdfe64-187f-4464-99bb-7ec50170fc04 None None] Authenticating user token {{(pid=80377) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:17:26.202586 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.fernet_utils [None req-a6bdfe64-187f-4464-99bb-7ec50170fc04 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80377) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 10:17:26.225125 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6bdfe64-187f-4464-99bb-7ec50170fc04 None None] Validating token access rules against request {{(pid=80377) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:17:26.230446 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ca42fea-adf0-4acb-a189-9568a107fcf7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '891005fa5ff74041b75747d16cc66798', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7047912c5acd4adeba5d87c3bf895e39', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80377) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:17:26.231149 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ca42fea-adf0-4acb-a189-9568a107fcf7 admin admin] REQUEST_METHOD: `PUT` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:17:26.231437 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ca42fea-adf0-4acb-a189-9568a107fcf7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:17:26.232515 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ca42fea-adf0-4acb-a189-9568a107fcf7 admin admin] PATH_INFO: `/v3/projects/5877952b334640b6b9d704e3f1409e7b/users/4a3080d2fc214bfc99cadab7c7e4ea89/roles/e1c04b19fc4543b8b54f58bbcffbc598` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:17:26.260331 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ca42fea-adf0-4acb-a189-9568a107fcf7 admin admin] RBAC: Authorizing `identity:create_grant(project_id=5877952b334640b6b9d704e3f1409e7b, user_id=4a3080d2fc214bfc99cadab7c7e4ea89, role_id=e1c04b19fc4543b8b54f58bbcffbc598)` {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:17:26.267229 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ca42fea-adf0-4acb-a189-9568a107fcf7 admin admin] RBAC: Authorization granted {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:17:26.271016 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-976e6f40-67e5-49c2-8291-bee8fad28765 admin admin] Validating token access rules against request {{(pid=80378) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:17:26.279463 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-759f81a1-038a-468b-bc50-10867d83123e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '891005fa5ff74041b75747d16cc66798', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7047912c5acd4adeba5d87c3bf895e39', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80378) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:17:26.280107 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-759f81a1-038a-468b-bc50-10867d83123e admin admin] REQUEST_METHOD: `GET` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:17:26.280389 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-759f81a1-038a-468b-bc50-10867d83123e admin admin] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:17:26.280723 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-759f81a1-038a-468b-bc50-10867d83123e admin admin] PATH_INFO: `/v3/roles` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:17:26.281444 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-759f81a1-038a-468b-bc50-10867d83123e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:17:26.283922 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-759f81a1-038a-468b-bc50-10867d83123e admin admin] RBAC: Authorization granted {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:17:26.296864 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 1041/2092] 10.222.0.82 () {58 vars in 1122 bytes} [Tue Aug 2 10:17:26 2022] GET /identity/v3/roles => generated 1420 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 02 10:17:26.298914 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-759f81a1-038a-468b-bc50-10867d83123e admin admin] Authenticating user token {{(pid=80378) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:17:26.301914 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.fernet_utils [None req-759f81a1-038a-468b-bc50-10867d83123e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80378) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 10:17:26.326082 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: [pid: 80377|app: 0|req: 1052/2093] 10.222.0.82 () {60 vars in 1483 bytes} [Tue Aug 2 10:17:26 2022] PUT /identity/v3/projects/5877952b334640b6b9d704e3f1409e7b/users/4a3080d2fc214bfc99cadab7c7e4ea89/roles/e1c04b19fc4543b8b54f58bbcffbc598 => generated 0 bytes in 124 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 02 10:17:26.326082 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ca42fea-adf0-4acb-a189-9568a107fcf7 admin admin] Authenticating user token {{(pid=80377) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:17:26.333380 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-759f81a1-038a-468b-bc50-10867d83123e admin admin] Validating token access rules against request {{(pid=80378) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:17:26.409643 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f753599-fe47-4c42-9860-e667d21bbf99 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '891005fa5ff74041b75747d16cc66798', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7047912c5acd4adeba5d87c3bf895e39', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80378) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:17:26.410486 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f753599-fe47-4c42-9860-e667d21bbf99 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:17:26.410858 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f753599-fe47-4c42-9860-e667d21bbf99 admin admin] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:17:26.411225 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f753599-fe47-4c42-9860-e667d21bbf99 admin admin] PATH_INFO: `/v3/users/4a6a1371a7f945da87ecc410eca99e37` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:17:26.412596 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f753599-fe47-4c42-9860-e667d21bbf99 admin admin] RBAC: Authorizing `identity:delete_user(user_id=4a6a1371a7f945da87ecc410eca99e37)` {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:17:26.412745 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ca42fea-adf0-4acb-a189-9568a107fcf7 admin admin] Validating token access rules against request {{(pid=80377) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:17:26.414935 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f753599-fe47-4c42-9860-e667d21bbf99 admin admin] RBAC: Authorization granted {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:17:26.417907 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18256a7e-cd78-427c-b59a-5e2479a1ec50 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '891005fa5ff74041b75747d16cc66798', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7047912c5acd4adeba5d87c3bf895e39', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80377) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:17:26.418264 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18256a7e-cd78-427c-b59a-5e2479a1ec50 admin admin] REQUEST_METHOD: `GET` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:17:26.418610 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18256a7e-cd78-427c-b59a-5e2479a1ec50 admin admin] SCRIPT_NAME: `/identity` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:17:26.418986 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18256a7e-cd78-427c-b59a-5e2479a1ec50 admin admin] PATH_INFO: `/v3/domains` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:17:26.419791 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18256a7e-cd78-427c-b59a-5e2479a1ec50 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80377) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 02 10:17:26.420169 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18256a7e-cd78-427c-b59a-5e2479a1ec50 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:17:26.421882 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18256a7e-cd78-427c-b59a-5e2479a1ec50 admin admin] RBAC: Authorization granted {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:17:26.438545 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: [pid: 80377|app: 0|req: 1053/2094] 10.222.0.82 () {58 vars in 1153 bytes} [Tue Aug 2 10:17:26 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 118 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 02 10:17:26.439470 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18256a7e-cd78-427c-b59a-5e2479a1ec50 admin admin] Authenticating user token {{(pid=80377) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:17:26.456846 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18256a7e-cd78-427c-b59a-5e2479a1ec50 admin admin] Validating token access rules against request {{(pid=80377) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:17:26.462474 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bc32c78-5339-4919-89c6-0bb059beae29 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '891005fa5ff74041b75747d16cc66798', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7047912c5acd4adeba5d87c3bf895e39', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80377) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:17:26.463338 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bc32c78-5339-4919-89c6-0bb059beae29 admin admin] REQUEST_METHOD: `PUT` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:17:26.463692 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bc32c78-5339-4919-89c6-0bb059beae29 admin admin] SCRIPT_NAME: `/identity` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:17:26.464087 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bc32c78-5339-4919-89c6-0bb059beae29 admin admin] PATH_INFO: `/v3/projects/afe3b2ebd99a4affbccaf8effc92e5e3/users/8db28092597b454dbfb040c0d8f35a96/roles/237966442f05411191105bf336bc14f3` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:17:26.484330 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5bc32c78-5339-4919-89c6-0bb059beae29 admin admin] RBAC: Authorizing `identity:create_grant(project_id=afe3b2ebd99a4affbccaf8effc92e5e3, user_id=8db28092597b454dbfb040c0d8f35a96, role_id=237966442f05411191105bf336bc14f3)` {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:17:26.486414 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.notifications [None req-4f753599-fe47-4c42-9860-e667d21bbf99 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '4a6a1371a7f945da87ecc410eca99e37'} {{(pid=80378) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 10:17:26.487672 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5bc32c78-5339-4919-89c6-0bb059beae29 admin admin] RBAC: Authorization granted {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:17:26.497384 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.notifications [None req-4f753599-fe47-4c42-9860-e667d21bbf99 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '4a6a1371a7f945da87ecc410eca99e37'} {{(pid=80378) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 10:17:26.510483 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: [pid: 80377|app: 0|req: 1054/2095] 10.222.0.82 () {60 vars in 1483 bytes} [Tue Aug 2 10:17:26 2022] PUT /identity/v3/projects/afe3b2ebd99a4affbccaf8effc92e5e3/users/8db28092597b454dbfb040c0d8f35a96/roles/237966442f05411191105bf336bc14f3 => generated 0 bytes in 71 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 02 10:17:26.511747 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3506c363-6a88-4976-9ec7-e549a7196e00 None None] REQUEST_METHOD: `POST` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:17:26.513982 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3506c363-6a88-4976-9ec7-e549a7196e00 None None] SCRIPT_NAME: `/identity` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:17:26.514649 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3506c363-6a88-4976-9ec7-e549a7196e00 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:17:26.516270 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.notifications [None req-4f753599-fe47-4c42-9860-e667d21bbf99 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '4a6a1371a7f945da87ecc410eca99e37'} {{(pid=80378) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 10:17:26.517172 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.notifications [None req-4f753599-fe47-4c42-9860-e667d21bbf99 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '4a6a1371a7f945da87ecc410eca99e37'} {{(pid=80378) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 10:17:26.542596 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.notifications [None req-4f753599-fe47-4c42-9860-e667d21bbf99 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '4a6a1371a7f945da87ecc410eca99e37'} {{(pid=80378) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 10:17:26.545877 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG dbcounter [-] [80378] Writing DB stats keystone:DELETE=13,keystone:SELECT=85,keystone:INSERT=2 {{(pid=80378) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 10:17:26.552512 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 1042/2096] 10.222.0.82 () {58 vars in 1224 bytes} [Tue Aug 2 10:17:26 2022] DELETE /identity/v3/users/4a6a1371a7f945da87ecc410eca99e37 => generated 0 bytes in 254 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 02 10:17:26.559095 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f753599-fe47-4c42-9860-e667d21bbf99 admin admin] Authenticating user token {{(pid=80378) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:17:26.561766 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.fernet_utils [None req-4f753599-fe47-4c42-9860-e667d21bbf99 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80378) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 10:17:26.577898 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.auth.core [None req-3506c363-6a88-4976-9ec7-e549a7196e00 None None] MFA Rules not processed for user `4a3080d2fc214bfc99cadab7c7e4ea89`. Rule list: `[]` (Enabled: `True`). {{(pid=80377) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 02 10:17:26.579619 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.fernet_utils [None req-3506c363-6a88-4976-9ec7-e549a7196e00 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80377) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 10:17:26.695823 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f753599-fe47-4c42-9860-e667d21bbf99 admin admin] Validating token access rules against request {{(pid=80378) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:17:26.705360 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b5d5adf-1e80-48db-919d-b8b19cdf3117 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '891005fa5ff74041b75747d16cc66798', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7047912c5acd4adeba5d87c3bf895e39', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80378) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:17:26.706039 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b5d5adf-1e80-48db-919d-b8b19cdf3117 admin admin] REQUEST_METHOD: `POST` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:17:26.706305 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b5d5adf-1e80-48db-919d-b8b19cdf3117 admin admin] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:17:26.706581 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b5d5adf-1e80-48db-919d-b8b19cdf3117 admin admin] PATH_INFO: `/v3/projects` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:17:26.707269 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b5d5adf-1e80-48db-919d-b8b19cdf3117 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:17:26.710508 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b5d5adf-1e80-48db-919d-b8b19cdf3117 admin admin] RBAC: Authorization granted {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:17:26.731366 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG dbcounter [-] [80377] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80377) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 10:17:26.742240 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: [pid: 80377|app: 0|req: 1055/2097] 10.222.0.82 () {58 vars in 958 bytes} [Tue Aug 2 10:17:26 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 231 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 02 10:17:26.743946 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3506c363-6a88-4976-9ec7-e549a7196e00 None None] Authenticating user token {{(pid=80377) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:17:26.746809 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.fernet_utils [None req-3506c363-6a88-4976-9ec7-e549a7196e00 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80377) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 10:17:26.774786 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 1043/2098] 10.222.0.82 () {60 vars in 1153 bytes} [Tue Aug 2 10:17:26 2022] POST /identity/v3/projects => generated 381 bytes in 221 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 02 10:17:26.775707 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b5d5adf-1e80-48db-919d-b8b19cdf3117 admin admin] Authenticating user token {{(pid=80378) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:17:26.777416 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.fernet_utils [None req-0b5d5adf-1e80-48db-919d-b8b19cdf3117 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80378) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 10:17:26.791210 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3506c363-6a88-4976-9ec7-e549a7196e00 None None] Validating token access rules against request {{(pid=80377) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:17:26.894609 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b5d5adf-1e80-48db-919d-b8b19cdf3117 admin admin] Validating token access rules against request {{(pid=80378) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:17:26.900067 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3cf4d611-8f55-402f-8a92-14aff4df6e43 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '891005fa5ff74041b75747d16cc66798', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7047912c5acd4adeba5d87c3bf895e39', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80377) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:17:26.901279 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cf4d611-8f55-402f-8a92-14aff4df6e43 admin admin] REQUEST_METHOD: `GET` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:17:26.901762 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cf4d611-8f55-402f-8a92-14aff4df6e43 admin admin] SCRIPT_NAME: `/identity` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:17:26.902299 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cf4d611-8f55-402f-8a92-14aff4df6e43 admin admin] PATH_INFO: `/v3/roles` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:17:26.903325 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3cf4d611-8f55-402f-8a92-14aff4df6e43 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:17:26.905469 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87dea667-425c-4bc0-a6cf-4097f9434344 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '891005fa5ff74041b75747d16cc66798', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7047912c5acd4adeba5d87c3bf895e39', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80378) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:17:26.906196 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87dea667-425c-4bc0-a6cf-4097f9434344 admin admin] REQUEST_METHOD: `POST` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:17:26.906439 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3cf4d611-8f55-402f-8a92-14aff4df6e43 admin admin] RBAC: Authorization granted {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:17:26.907670 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87dea667-425c-4bc0-a6cf-4097f9434344 admin admin] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:17:26.908441 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87dea667-425c-4bc0-a6cf-4097f9434344 admin admin] PATH_INFO: `/v3/projects` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:17:26.910731 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87dea667-425c-4bc0-a6cf-4097f9434344 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:17:26.914210 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87dea667-425c-4bc0-a6cf-4097f9434344 admin admin] RBAC: Authorization granted {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:17:26.920350 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: [pid: 80377|app: 0|req: 1056/2099] 10.222.0.82 () {58 vars in 1122 bytes} [Tue Aug 2 10:17:26 2022] GET /identity/v3/roles => generated 1420 bytes in 177 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 02 10:17:26.921950 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3cf4d611-8f55-402f-8a92-14aff4df6e43 admin admin] Authenticating user token {{(pid=80377) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:17:26.936341 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3cf4d611-8f55-402f-8a92-14aff4df6e43 admin admin] Validating token access rules against request {{(pid=80377) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:17:26.948014 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20ce33a7-5ef0-4c75-8acb-4c375018b2e3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '891005fa5ff74041b75747d16cc66798', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7047912c5acd4adeba5d87c3bf895e39', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80377) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:17:26.948779 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20ce33a7-5ef0-4c75-8acb-4c375018b2e3 admin admin] REQUEST_METHOD: `POST` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:17:26.949067 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20ce33a7-5ef0-4c75-8acb-4c375018b2e3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:17:26.949489 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20ce33a7-5ef0-4c75-8acb-4c375018b2e3 admin admin] PATH_INFO: `/v3/users` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:17:26.950239 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20ce33a7-5ef0-4c75-8acb-4c375018b2e3 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:17:26.952681 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20ce33a7-5ef0-4c75-8acb-4c375018b2e3 admin admin] RBAC: Authorization granted {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:17:26.972093 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 1044/2100] 10.222.0.82 () {60 vars in 1153 bytes} [Tue Aug 2 10:17:26 2022] POST /identity/v3/projects => generated 375 bytes in 196 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 02 10:17:26.973852 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87dea667-425c-4bc0-a6cf-4097f9434344 admin admin] Authenticating user token {{(pid=80378) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:17:27.007947 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: [pid: 80377|app: 0|req: 1057/2101] 10.222.0.82 () {60 vars in 1144 bytes} [Tue Aug 2 10:17:26 2022] POST /identity/v3/users => generated 291 bytes in 87 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 02 10:17:27.009398 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20ce33a7-5ef0-4c75-8acb-4c375018b2e3 admin admin] Authenticating user token {{(pid=80377) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:17:27.027503 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20ce33a7-5ef0-4c75-8acb-4c375018b2e3 admin admin] Validating token access rules against request {{(pid=80377) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:17:27.084160 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87dea667-425c-4bc0-a6cf-4097f9434344 admin admin] Validating token access rules against request {{(pid=80378) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:17:27.090449 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82e42f65-8507-454f-aeff-3bd594c38686 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '891005fa5ff74041b75747d16cc66798', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7047912c5acd4adeba5d87c3bf895e39', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80378) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:17:27.091170 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82e42f65-8507-454f-aeff-3bd594c38686 admin admin] REQUEST_METHOD: `PUT` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:17:27.091460 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82e42f65-8507-454f-aeff-3bd594c38686 admin admin] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:17:27.091736 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82e42f65-8507-454f-aeff-3bd594c38686 admin admin] PATH_INFO: `/v3/projects/afe3b2ebd99a4affbccaf8effc92e5e3/users/8db28092597b454dbfb040c0d8f35a96/roles/237966442f05411191105bf336bc14f3` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:17:27.095307 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82e42f65-8507-454f-aeff-3bd594c38686 admin admin] RBAC: Authorizing `identity:create_grant(project_id=afe3b2ebd99a4affbccaf8effc92e5e3, user_id=8db28092597b454dbfb040c0d8f35a96, role_id=237966442f05411191105bf336bc14f3)` {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:17:27.099110 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82e42f65-8507-454f-aeff-3bd594c38686 admin admin] RBAC: Authorization granted {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:17:27.109726 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18230b31-2285-4b2b-b7d3-42abd64c08c2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '891005fa5ff74041b75747d16cc66798', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7047912c5acd4adeba5d87c3bf895e39', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80377) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:17:27.110389 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18230b31-2285-4b2b-b7d3-42abd64c08c2 admin admin] REQUEST_METHOD: `POST` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:17:27.110676 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18230b31-2285-4b2b-b7d3-42abd64c08c2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:17:27.110955 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18230b31-2285-4b2b-b7d3-42abd64c08c2 admin admin] PATH_INFO: `/v3/users` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:17:27.111673 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18230b31-2285-4b2b-b7d3-42abd64c08c2 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:17:27.113922 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18230b31-2285-4b2b-b7d3-42abd64c08c2 admin admin] RBAC: Authorization granted {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:17:27.133653 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 1045/2102] 10.222.0.82 () {60 vars in 1483 bytes} [Tue Aug 2 10:17:26 2022] PUT /identity/v3/projects/afe3b2ebd99a4affbccaf8effc92e5e3/users/8db28092597b454dbfb040c0d8f35a96/roles/237966442f05411191105bf336bc14f3 => generated 0 bytes in 160 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 02 10:17:27.135849 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82e42f65-8507-454f-aeff-3bd594c38686 admin admin] Authenticating user token {{(pid=80378) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:17:27.145152 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: [pid: 80377|app: 0|req: 1058/2103] 10.222.0.82 () {60 vars in 1144 bytes} [Tue Aug 2 10:17:27 2022] POST /identity/v3/users => generated 294 bytes in 137 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 02 10:17:27.146183 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18230b31-2285-4b2b-b7d3-42abd64c08c2 admin admin] Authenticating user token {{(pid=80377) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:17:27.149345 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG dbcounter [-] [80378] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80378) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 10:17:27.150397 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.fernet_utils [None req-18230b31-2285-4b2b-b7d3-42abd64c08c2 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80377) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 10:17:27.170155 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18230b31-2285-4b2b-b7d3-42abd64c08c2 admin admin] Validating token access rules against request {{(pid=80377) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:17:27.248500 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82e42f65-8507-454f-aeff-3bd594c38686 admin admin] Validating token access rules against request {{(pid=80378) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:17:27.253364 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd0bfa4e-e929-425b-93fc-6d1ea50ed00a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '891005fa5ff74041b75747d16cc66798', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7047912c5acd4adeba5d87c3bf895e39', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80377) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:17:27.254150 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd0bfa4e-e929-425b-93fc-6d1ea50ed00a admin admin] REQUEST_METHOD: `DELETE` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:17:27.254793 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd0bfa4e-e929-425b-93fc-6d1ea50ed00a admin admin] SCRIPT_NAME: `/identity` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:17:27.255133 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd0bfa4e-e929-425b-93fc-6d1ea50ed00a admin admin] PATH_INFO: `/v3/projects/49695c58be9542959c6852e9207b1043` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:17:27.256779 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-973c8b8a-417b-46e9-8eab-a7d3cc716743 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '891005fa5ff74041b75747d16cc66798', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7047912c5acd4adeba5d87c3bf895e39', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80378) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:17:27.257083 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd0bfa4e-e929-425b-93fc-6d1ea50ed00a admin admin] RBAC: Authorizing `identity:delete_project(project_id=49695c58be9542959c6852e9207b1043)` {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:17:27.258513 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-973c8b8a-417b-46e9-8eab-a7d3cc716743 admin admin] REQUEST_METHOD: `GET` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:17:27.259057 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-973c8b8a-417b-46e9-8eab-a7d3cc716743 admin admin] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:17:27.259623 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd0bfa4e-e929-425b-93fc-6d1ea50ed00a admin admin] RBAC: Authorization granted {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:17:27.259989 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-973c8b8a-417b-46e9-8eab-a7d3cc716743 admin admin] PATH_INFO: `/v3/roles` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:17:27.260875 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-973c8b8a-417b-46e9-8eab-a7d3cc716743 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:17:27.263575 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-973c8b8a-417b-46e9-8eab-a7d3cc716743 admin admin] RBAC: Authorization granted {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:17:27.282209 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG dbcounter [-] [80377] Writing DB stats keystone:SELECT=94,keystone:INSERT=6 {{(pid=80377) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 10:17:27.286832 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 1046/2104] 10.222.0.82 () {58 vars in 1122 bytes} [Tue Aug 2 10:17:27 2022] GET /identity/v3/roles => generated 1420 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 02 10:17:27.290898 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-973c8b8a-417b-46e9-8eab-a7d3cc716743 admin admin] Authenticating user token {{(pid=80378) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:17:27.310707 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-973c8b8a-417b-46e9-8eab-a7d3cc716743 admin admin] Validating token access rules against request {{(pid=80378) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:17:27.326820 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1f36445-6fd7-4c89-ac59-8c36b06447d7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '891005fa5ff74041b75747d16cc66798', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7047912c5acd4adeba5d87c3bf895e39', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80378) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:17:27.329251 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1f36445-6fd7-4c89-ac59-8c36b06447d7 admin admin] REQUEST_METHOD: `GET` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:17:27.329634 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1f36445-6fd7-4c89-ac59-8c36b06447d7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:17:27.330032 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1f36445-6fd7-4c89-ac59-8c36b06447d7 admin admin] PATH_INFO: `/v3/roles` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:17:27.331550 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1f36445-6fd7-4c89-ac59-8c36b06447d7 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:17:27.334978 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1f36445-6fd7-4c89-ac59-8c36b06447d7 admin admin] RBAC: Authorization granted {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:17:27.364468 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 1047/2105] 10.222.0.82 () {58 vars in 1122 bytes} [Tue Aug 2 10:17:27 2022] GET /identity/v3/roles => generated 1420 bytes in 77 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 02 10:17:27.369827 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1f36445-6fd7-4c89-ac59-8c36b06447d7 admin admin] Authenticating user token {{(pid=80378) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:17:27.374812 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.notifications [None req-cd0bfa4e-e929-425b-93fc-6d1ea50ed00a admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '49695c58be9542959c6852e9207b1043', 'request_id': 'req-cd0bfa4e-e929-425b-93fc-6d1ea50ed00a'} {{(pid=80377) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 10:17:27.383357 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.notifications [None req-cd0bfa4e-e929-425b-93fc-6d1ea50ed00a admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '49695c58be9542959c6852e9207b1043', 'request_id': 'req-cd0bfa4e-e929-425b-93fc-6d1ea50ed00a'} {{(pid=80377) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 10:17:27.391288 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1f36445-6fd7-4c89-ac59-8c36b06447d7 admin admin] Validating token access rules against request {{(pid=80378) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:17:27.392365 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.notifications [None req-cd0bfa4e-e929-425b-93fc-6d1ea50ed00a admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '49695c58be9542959c6852e9207b1043', 'request_id': 'req-cd0bfa4e-e929-425b-93fc-6d1ea50ed00a'} {{(pid=80377) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 10:17:27.412622 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.notifications [None req-cd0bfa4e-e929-425b-93fc-6d1ea50ed00a admin admin] The token cache is being invalidate because project 49695c58be9542959c6852e9207b1043 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80377) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 02 10:17:27.413085 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.notifications [None req-cd0bfa4e-e929-425b-93fc-6d1ea50ed00a admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80377) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 10:17:27.414101 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.notifications [None req-cd0bfa4e-e929-425b-93fc-6d1ea50ed00a admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80377) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 10:17:27.421507 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: [pid: 80377|app: 0|req: 1059/2106] 10.222.0.82 () {58 vars in 1233 bytes} [Tue Aug 2 10:17:27 2022] DELETE /identity/v3/projects/49695c58be9542959c6852e9207b1043 => generated 0 bytes in 270 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 02 10:17:27.421507 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd0bfa4e-e929-425b-93fc-6d1ea50ed00a admin admin] Authenticating user token {{(pid=80377) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:17:27.421507 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.fernet_utils [None req-cd0bfa4e-e929-425b-93fc-6d1ea50ed00a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80377) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 10:17:27.491806 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30d538c0-af6b-4e77-88c3-f047acaafe87 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '891005fa5ff74041b75747d16cc66798', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7047912c5acd4adeba5d87c3bf895e39', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80378) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:17:27.493887 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30d538c0-af6b-4e77-88c3-f047acaafe87 admin admin] REQUEST_METHOD: `GET` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:17:27.494017 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30d538c0-af6b-4e77-88c3-f047acaafe87 admin admin] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:17:27.494512 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30d538c0-af6b-4e77-88c3-f047acaafe87 admin admin] PATH_INFO: `/v3/roles` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:17:27.496150 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30d538c0-af6b-4e77-88c3-f047acaafe87 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:17:27.498795 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30d538c0-af6b-4e77-88c3-f047acaafe87 admin admin] RBAC: Authorization granted {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:17:27.513438 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 1048/2107] 10.222.0.82 () {58 vars in 1122 bytes} [Tue Aug 2 10:17:27 2022] GET /identity/v3/roles => generated 1420 bytes in 144 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 02 10:17:27.515537 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd0bfa4e-e929-425b-93fc-6d1ea50ed00a admin admin] Validating token access rules against request {{(pid=80377) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:17:27.518360 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e703ecb1-0f0d-41f1-9930-686190714ab1 None None] REQUEST_METHOD: `POST` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:17:27.518741 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e703ecb1-0f0d-41f1-9930-686190714ab1 None None] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:17:27.519133 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e703ecb1-0f0d-41f1-9930-686190714ab1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:17:27.553480 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-450dea65-1871-4239-8e97-16c41ba77960 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '891005fa5ff74041b75747d16cc66798', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7047912c5acd4adeba5d87c3bf895e39', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80377) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:17:27.556772 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-450dea65-1871-4239-8e97-16c41ba77960 admin admin] REQUEST_METHOD: `PUT` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:17:27.558979 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-450dea65-1871-4239-8e97-16c41ba77960 admin admin] SCRIPT_NAME: `/identity` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:17:27.559299 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-450dea65-1871-4239-8e97-16c41ba77960 admin admin] PATH_INFO: `/v3/projects/13848405aa654dfa96cc1d7e8b18d8ac/users/7e6fcf8a332c4e81b40ac1cc3eb747c6/roles/237966442f05411191105bf336bc14f3` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:17:27.567473 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.auth.core [None req-e703ecb1-0f0d-41f1-9930-686190714ab1 None None] MFA Rules not processed for user `891005fa5ff74041b75747d16cc66798`. Rule list: `[]` (Enabled: `True`). {{(pid=80378) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 02 10:17:27.569450 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.fernet_utils [None req-e703ecb1-0f0d-41f1-9930-686190714ab1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80378) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 10:17:27.621126 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-450dea65-1871-4239-8e97-16c41ba77960 admin admin] RBAC: Authorizing `identity:create_grant(project_id=13848405aa654dfa96cc1d7e8b18d8ac, user_id=7e6fcf8a332c4e81b40ac1cc3eb747c6, role_id=237966442f05411191105bf336bc14f3)` {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:17:27.625628 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-450dea65-1871-4239-8e97-16c41ba77960 admin admin] RBAC: Authorization granted {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:17:27.641994 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 1049/2108] 10.222.0.82 () {58 vars in 958 bytes} [Tue Aug 2 10:17:27 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 128 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 02 10:17:27.643006 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e703ecb1-0f0d-41f1-9930-686190714ab1 None None] Authenticating user token {{(pid=80378) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:17:27.644473 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.fernet_utils [None req-e703ecb1-0f0d-41f1-9930-686190714ab1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80378) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 10:17:27.656972 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: [pid: 80377|app: 0|req: 1060/2109] 10.222.0.82 () {60 vars in 1483 bytes} [Tue Aug 2 10:17:27 2022] PUT /identity/v3/projects/13848405aa654dfa96cc1d7e8b18d8ac/users/7e6fcf8a332c4e81b40ac1cc3eb747c6/roles/237966442f05411191105bf336bc14f3 => generated 0 bytes in 240 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 02 10:17:27.658867 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30dd2eba-edfc-407f-bcf1-4e8c0f9927bd None None] REQUEST_METHOD: `POST` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:17:27.658867 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30dd2eba-edfc-407f-bcf1-4e8c0f9927bd None None] SCRIPT_NAME: `/identity` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:17:27.658867 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30dd2eba-edfc-407f-bcf1-4e8c0f9927bd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:17:27.662989 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e703ecb1-0f0d-41f1-9930-686190714ab1 None None] Validating token access rules against request {{(pid=80378) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:17:27.690737 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.auth.core [None req-30dd2eba-edfc-407f-bcf1-4e8c0f9927bd None None] MFA Rules not processed for user `891005fa5ff74041b75747d16cc66798`. Rule list: `[]` (Enabled: `True`). {{(pid=80377) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 02 10:17:27.691995 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.fernet_utils [None req-30dd2eba-edfc-407f-bcf1-4e8c0f9927bd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80377) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 10:17:27.747094 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4346d0cb-b338-4046-9253-c8676cf2bb27 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '891005fa5ff74041b75747d16cc66798', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7047912c5acd4adeba5d87c3bf895e39', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80378) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:17:27.749844 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4346d0cb-b338-4046-9253-c8676cf2bb27 admin admin] REQUEST_METHOD: `PUT` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:17:27.749844 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4346d0cb-b338-4046-9253-c8676cf2bb27 admin admin] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:17:27.749844 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4346d0cb-b338-4046-9253-c8676cf2bb27 admin admin] PATH_INFO: `/v3/projects/cb42e40657a64cc29779eb3552b569d7/users/3d76ef766e9b4f608240cffcbf2b5906/roles/e1c04b19fc4543b8b54f58bbcffbc598` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:17:27.770007 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4346d0cb-b338-4046-9253-c8676cf2bb27 admin admin] RBAC: Authorizing `identity:create_grant(project_id=cb42e40657a64cc29779eb3552b569d7, user_id=3d76ef766e9b4f608240cffcbf2b5906, role_id=e1c04b19fc4543b8b54f58bbcffbc598)` {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:17:27.773226 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4346d0cb-b338-4046-9253-c8676cf2bb27 admin admin] RBAC: Authorization granted {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:17:27.790893 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG dbcounter [-] [80378] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80378) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 10:17:27.816556 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 1050/2110] 10.222.0.82 () {60 vars in 1483 bytes} [Tue Aug 2 10:17:27 2022] PUT /identity/v3/projects/cb42e40657a64cc29779eb3552b569d7/users/3d76ef766e9b4f608240cffcbf2b5906/roles/e1c04b19fc4543b8b54f58bbcffbc598 => generated 0 bytes in 161 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 02 10:17:27.817675 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4346d0cb-b338-4046-9253-c8676cf2bb27 admin admin] Authenticating user token {{(pid=80378) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:17:27.828135 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: [pid: 80377|app: 0|req: 1061/2111] 10.222.0.82 () {58 vars in 958 bytes} [Tue Aug 2 10:17:27 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 168 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 02 10:17:27.828135 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fa82b44-0a5e-4f86-a13e-d03cd93fa11c None None] REQUEST_METHOD: `POST` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:17:27.828135 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fa82b44-0a5e-4f86-a13e-d03cd93fa11c None None] SCRIPT_NAME: `/identity` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:17:27.844073 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fa82b44-0a5e-4f86-a13e-d03cd93fa11c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:17:27.941579 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4346d0cb-b338-4046-9253-c8676cf2bb27 admin admin] Validating token access rules against request {{(pid=80378) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:17:27.948309 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a0c7af1-eb65-4255-816f-b2c11e6b7191 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '891005fa5ff74041b75747d16cc66798', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7047912c5acd4adeba5d87c3bf895e39', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80378) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:17:27.949052 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a0c7af1-eb65-4255-816f-b2c11e6b7191 admin admin] REQUEST_METHOD: `GET` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:17:27.949334 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a0c7af1-eb65-4255-816f-b2c11e6b7191 admin admin] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:17:27.949608 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a0c7af1-eb65-4255-816f-b2c11e6b7191 admin admin] PATH_INFO: `/v3/roles` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:17:27.950403 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a0c7af1-eb65-4255-816f-b2c11e6b7191 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:17:27.952961 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a0c7af1-eb65-4255-816f-b2c11e6b7191 admin admin] RBAC: Authorization granted {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:17:27.971656 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.auth.core [None req-9fa82b44-0a5e-4f86-a13e-d03cd93fa11c None None] MFA Rules not processed for user `3d76ef766e9b4f608240cffcbf2b5906`. Rule list: `[]` (Enabled: `True`). {{(pid=80377) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 02 10:17:27.978509 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.fernet_utils [None req-9fa82b44-0a5e-4f86-a13e-d03cd93fa11c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80377) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 10:17:27.979509 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 1051/2112] 10.222.0.82 () {58 vars in 1122 bytes} [Tue Aug 2 10:17:27 2022] GET /identity/v3/roles => generated 1420 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 02 10:17:27.980511 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a0c7af1-eb65-4255-816f-b2c11e6b7191 admin admin] Authenticating user token {{(pid=80378) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:17:27.995655 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a0c7af1-eb65-4255-816f-b2c11e6b7191 admin admin] Validating token access rules against request {{(pid=80378) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:17:28.001823 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2c78b42-d971-4e82-a864-757bb3d847e0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '891005fa5ff74041b75747d16cc66798', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7047912c5acd4adeba5d87c3bf895e39', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80378) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:17:28.002637 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2c78b42-d971-4e82-a864-757bb3d847e0 admin admin] REQUEST_METHOD: `GET` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:17:28.003117 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2c78b42-d971-4e82-a864-757bb3d847e0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:17:28.003469 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2c78b42-d971-4e82-a864-757bb3d847e0 admin admin] PATH_INFO: `/v3/domains` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:17:28.004247 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2c78b42-d971-4e82-a864-757bb3d847e0 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80378) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 02 10:17:28.004721 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2c78b42-d971-4e82-a864-757bb3d847e0 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:17:28.007922 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG dbcounter [-] [80377] Writing DB stats keystone:SELECT=91,keystone:DELETE=7,keystone:INSERT=2 {{(pid=80377) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 10:17:28.008478 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2c78b42-d971-4e82-a864-757bb3d847e0 admin admin] RBAC: Authorization granted {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:17:28.032973 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 1052/2113] 10.222.0.82 () {58 vars in 1153 bytes} [Tue Aug 2 10:17:27 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 53 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 02 10:17:28.035865 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2c78b42-d971-4e82-a864-757bb3d847e0 admin admin] Authenticating user token {{(pid=80378) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:17:28.059616 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2c78b42-d971-4e82-a864-757bb3d847e0 admin admin] Validating token access rules against request {{(pid=80378) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:17:28.073078 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6bfe448d-c9df-4f6e-8139-f305ed9d9860 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '891005fa5ff74041b75747d16cc66798', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7047912c5acd4adeba5d87c3bf895e39', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80378) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:17:28.073815 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bfe448d-c9df-4f6e-8139-f305ed9d9860 admin admin] REQUEST_METHOD: `PUT` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:17:28.074212 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bfe448d-c9df-4f6e-8139-f305ed9d9860 admin admin] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:17:28.074562 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bfe448d-c9df-4f6e-8139-f305ed9d9860 admin admin] PATH_INFO: `/v3/projects/13848405aa654dfa96cc1d7e8b18d8ac/users/7e6fcf8a332c4e81b40ac1cc3eb747c6/roles/237966442f05411191105bf336bc14f3` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:17:28.081774 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6bfe448d-c9df-4f6e-8139-f305ed9d9860 admin admin] RBAC: Authorizing `identity:create_grant(project_id=13848405aa654dfa96cc1d7e8b18d8ac, user_id=7e6fcf8a332c4e81b40ac1cc3eb747c6, role_id=237966442f05411191105bf336bc14f3)` {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:17:28.085528 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6bfe448d-c9df-4f6e-8139-f305ed9d9860 admin admin] RBAC: Authorization granted {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:17:28.103809 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 1053/2114] 10.222.0.82 () {60 vars in 1483 bytes} [Tue Aug 2 10:17:28 2022] PUT /identity/v3/projects/13848405aa654dfa96cc1d7e8b18d8ac/users/7e6fcf8a332c4e81b40ac1cc3eb747c6/roles/237966442f05411191105bf336bc14f3 => generated 0 bytes in 70 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 02 10:17:28.104892 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6bfe448d-c9df-4f6e-8139-f305ed9d9860 admin admin] Authenticating user token {{(pid=80378) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:17:28.194934 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: [pid: 80377|app: 0|req: 1062/2115] 10.222.0.82 () {58 vars in 958 bytes} [Tue Aug 2 10:17:27 2022] POST /identity/v3/auth/tokens => generated 3258 bytes in 366 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 02 10:17:28.194934 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ced69c2f-a78d-474d-af6d-f6cd18b709a1 None None] REQUEST_METHOD: `POST` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:17:28.194934 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ced69c2f-a78d-474d-af6d-f6cd18b709a1 None None] SCRIPT_NAME: `/identity` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:17:28.194934 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ced69c2f-a78d-474d-af6d-f6cd18b709a1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:17:28.235663 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.auth.core [None req-ced69c2f-a78d-474d-af6d-f6cd18b709a1 None None] MFA Rules not processed for user `7e6fcf8a332c4e81b40ac1cc3eb747c6`. Rule list: `[]` (Enabled: `True`). {{(pid=80377) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 02 10:17:28.238886 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.fernet_utils [None req-ced69c2f-a78d-474d-af6d-f6cd18b709a1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80377) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 10:17:28.242910 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6bfe448d-c9df-4f6e-8139-f305ed9d9860 admin admin] Validating token access rules against request {{(pid=80378) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:17:28.254023 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb7d9a26-c41a-4a07-85ad-f91a312717e1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '891005fa5ff74041b75747d16cc66798', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7047912c5acd4adeba5d87c3bf895e39', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80378) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:17:28.254872 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb7d9a26-c41a-4a07-85ad-f91a312717e1 admin admin] REQUEST_METHOD: `POST` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:17:28.255223 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb7d9a26-c41a-4a07-85ad-f91a312717e1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:17:28.255866 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb7d9a26-c41a-4a07-85ad-f91a312717e1 admin admin] PATH_INFO: `/v3/projects` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:17:28.258617 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb7d9a26-c41a-4a07-85ad-f91a312717e1 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:17:28.274481 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb7d9a26-c41a-4a07-85ad-f91a312717e1 admin admin] RBAC: Authorization granted {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:17:28.345157 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 1054/2116] 10.222.0.82 () {60 vars in 1153 bytes} [Tue Aug 2 10:17:28 2022] POST /identity/v3/projects => generated 385 bytes in 238 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 02 10:17:28.345157 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb7d9a26-c41a-4a07-85ad-f91a312717e1 admin admin] Authenticating user token {{(pid=80378) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:17:28.433644 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: [pid: 80377|app: 0|req: 1063/2117] 10.222.0.82 () {58 vars in 958 bytes} [Tue Aug 2 10:17:28 2022] POST /identity/v3/auth/tokens => generated 3319 bytes in 241 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 02 10:17:28.434621 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ced69c2f-a78d-474d-af6d-f6cd18b709a1 None None] Authenticating user token {{(pid=80377) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:17:28.437757 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG dbcounter [-] [80378] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80378) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 10:17:28.449601 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb7d9a26-c41a-4a07-85ad-f91a312717e1 admin admin] Validating token access rules against request {{(pid=80378) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:17:28.456307 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99061ddc-4cbd-45c7-839c-3e30cac3a592 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '891005fa5ff74041b75747d16cc66798', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7047912c5acd4adeba5d87c3bf895e39', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80378) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:17:28.457199 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99061ddc-4cbd-45c7-839c-3e30cac3a592 admin admin] REQUEST_METHOD: `POST` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:17:28.458091 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99061ddc-4cbd-45c7-839c-3e30cac3a592 admin admin] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:17:28.458521 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99061ddc-4cbd-45c7-839c-3e30cac3a592 admin admin] PATH_INFO: `/v3/projects` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:17:28.459357 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99061ddc-4cbd-45c7-839c-3e30cac3a592 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:17:28.461508 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99061ddc-4cbd-45c7-839c-3e30cac3a592 admin admin] RBAC: Authorization granted {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:17:28.468031 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ced69c2f-a78d-474d-af6d-f6cd18b709a1 None None] Validating token access rules against request {{(pid=80377) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:17:28.487732 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92c3531c-b5d7-4b5c-84bf-ec423c88806c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '891005fa5ff74041b75747d16cc66798', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7047912c5acd4adeba5d87c3bf895e39', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80377) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:17:28.489422 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92c3531c-b5d7-4b5c-84bf-ec423c88806c admin admin] REQUEST_METHOD: `POST` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:17:28.490045 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92c3531c-b5d7-4b5c-84bf-ec423c88806c admin admin] SCRIPT_NAME: `/identity` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:17:28.490943 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92c3531c-b5d7-4b5c-84bf-ec423c88806c admin admin] PATH_INFO: `/v3/users` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:17:28.495961 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92c3531c-b5d7-4b5c-84bf-ec423c88806c admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:17:28.502184 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92c3531c-b5d7-4b5c-84bf-ec423c88806c admin admin] RBAC: Authorization granted {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:17:28.516825 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 1055/2118] 10.222.0.82 () {60 vars in 1153 bytes} [Tue Aug 2 10:17:28 2022] POST /identity/v3/projects => generated 383 bytes in 174 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 02 10:17:28.518004 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99061ddc-4cbd-45c7-839c-3e30cac3a592 admin admin] Authenticating user token {{(pid=80378) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:17:28.521655 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.fernet_utils [None req-99061ddc-4cbd-45c7-839c-3e30cac3a592 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80378) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 10:17:28.577674 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: [pid: 80377|app: 0|req: 1064/2119] 10.222.0.82 () {60 vars in 1144 bytes} [Tue Aug 2 10:17:28 2022] POST /identity/v3/users => generated 293 bytes in 143 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 02 10:17:28.579464 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92c3531c-b5d7-4b5c-84bf-ec423c88806c admin admin] Authenticating user token {{(pid=80377) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:17:28.629951 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92c3531c-b5d7-4b5c-84bf-ec423c88806c admin admin] Validating token access rules against request {{(pid=80377) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:17:28.713358 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99061ddc-4cbd-45c7-839c-3e30cac3a592 admin admin] Validating token access rules against request {{(pid=80378) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:17:28.732148 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3fd9bb2-b1f8-4c3d-a336-77761da112a4 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1023de281ae74e9d8a6d6c312c1c97a7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f1b38ebb804c41c69772c998aae72c41', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80378) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:17:28.732802 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3fd9bb2-b1f8-4c3d-a336-77761da112a4 service cinder] REQUEST_METHOD: `GET` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:17:28.733086 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3fd9bb2-b1f8-4c3d-a336-77761da112a4 service cinder] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:17:28.733365 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3fd9bb2-b1f8-4c3d-a336-77761da112a4 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:17:28.741958 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.fernet_utils [None req-f3fd9bb2-b1f8-4c3d-a336-77761da112a4 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80378) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 10:17:28.840654 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43272654-6270-4446-84f5-3ca72884f86b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '891005fa5ff74041b75747d16cc66798', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7047912c5acd4adeba5d87c3bf895e39', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80377) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:17:28.865757 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43272654-6270-4446-84f5-3ca72884f86b admin admin] REQUEST_METHOD: `POST` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:17:28.866798 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43272654-6270-4446-84f5-3ca72884f86b admin admin] SCRIPT_NAME: `/identity` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:17:28.867094 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43272654-6270-4446-84f5-3ca72884f86b admin admin] PATH_INFO: `/v3/users` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:17:28.874060 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43272654-6270-4446-84f5-3ca72884f86b admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:17:28.893678 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43272654-6270-4446-84f5-3ca72884f86b admin admin] RBAC: Authorization granted {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:17:28.903679 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3fd9bb2-b1f8-4c3d-a336-77761da112a4 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:17:28.906286 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3fd9bb2-b1f8-4c3d-a336-77761da112a4 service cinder] RBAC: Authorization granted {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:17:28.918685 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 1056/2120] 10.222.0.82 () {60 vars in 1365 bytes} [Tue Aug 2 10:17:28 2022] GET /identity/v3/auth/tokens => generated 3252 bytes in 401 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 02 10:17:28.919735 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3fd9bb2-b1f8-4c3d-a336-77761da112a4 service cinder] Authenticating user token {{(pid=80378) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:17:28.934507 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3fd9bb2-b1f8-4c3d-a336-77761da112a4 service cinder] Validating token access rules against request {{(pid=80378) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:17:28.952775 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-525d4af8-56f0-4e2a-ab33-63bb12eff53a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '891005fa5ff74041b75747d16cc66798', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7047912c5acd4adeba5d87c3bf895e39', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80378) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:17:28.953539 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-525d4af8-56f0-4e2a-ab33-63bb12eff53a admin admin] REQUEST_METHOD: `GET` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:17:28.955612 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-525d4af8-56f0-4e2a-ab33-63bb12eff53a admin admin] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:17:28.955894 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-525d4af8-56f0-4e2a-ab33-63bb12eff53a admin admin] PATH_INFO: `/v3/roles` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:17:28.957385 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-525d4af8-56f0-4e2a-ab33-63bb12eff53a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:17:28.965016 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-525d4af8-56f0-4e2a-ab33-63bb12eff53a admin admin] RBAC: Authorization granted {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:17:28.994801 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 1057/2121] 10.222.0.82 () {58 vars in 1122 bytes} [Tue Aug 2 10:17:28 2022] GET /identity/v3/roles => generated 1420 bytes in 76 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 02 10:17:29.000253 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-525d4af8-56f0-4e2a-ab33-63bb12eff53a admin admin] Authenticating user token {{(pid=80378) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:17:29.016685 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: [pid: 80377|app: 0|req: 1065/2122] 10.222.0.82 () {60 vars in 1144 bytes} [Tue Aug 2 10:17:28 2022] POST /identity/v3/users => generated 298 bytes in 438 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 02 10:17:29.020867 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43272654-6270-4446-84f5-3ca72884f86b admin admin] Authenticating user token {{(pid=80377) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:17:29.030449 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-525d4af8-56f0-4e2a-ab33-63bb12eff53a admin admin] Validating token access rules against request {{(pid=80378) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:17:29.044246 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-544d0df0-ffd0-4014-8956-0894d0e5f9a0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '891005fa5ff74041b75747d16cc66798', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7047912c5acd4adeba5d87c3bf895e39', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80378) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:17:29.045274 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-544d0df0-ffd0-4014-8956-0894d0e5f9a0 admin admin] REQUEST_METHOD: `GET` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:17:29.045823 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-544d0df0-ffd0-4014-8956-0894d0e5f9a0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:17:29.047142 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-544d0df0-ffd0-4014-8956-0894d0e5f9a0 admin admin] PATH_INFO: `/v3/roles` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:17:29.047142 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-544d0df0-ffd0-4014-8956-0894d0e5f9a0 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:17:29.051749 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-544d0df0-ffd0-4014-8956-0894d0e5f9a0 admin admin] RBAC: Authorization granted {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:17:29.076810 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 1058/2123] 10.222.0.82 () {58 vars in 1122 bytes} [Tue Aug 2 10:17:28 2022] GET /identity/v3/roles => generated 1420 bytes in 77 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 02 10:17:29.081886 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-544d0df0-ffd0-4014-8956-0894d0e5f9a0 admin admin] Authenticating user token {{(pid=80378) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:17:29.082973 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43272654-6270-4446-84f5-3ca72884f86b admin admin] Validating token access rules against request {{(pid=80377) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:17:29.096863 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-653b62b4-134a-4740-ad60-44a302282a5d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '891005fa5ff74041b75747d16cc66798', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7047912c5acd4adeba5d87c3bf895e39', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80377) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:17:29.097471 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-653b62b4-134a-4740-ad60-44a302282a5d admin admin] REQUEST_METHOD: `GET` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:17:29.097726 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-653b62b4-134a-4740-ad60-44a302282a5d admin admin] SCRIPT_NAME: `/identity` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:17:29.098062 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-653b62b4-134a-4740-ad60-44a302282a5d admin admin] PATH_INFO: `/v3/roles` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:17:29.098793 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-653b62b4-134a-4740-ad60-44a302282a5d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:17:29.101743 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-653b62b4-134a-4740-ad60-44a302282a5d admin admin] RBAC: Authorization granted {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:17:29.105745 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-544d0df0-ffd0-4014-8956-0894d0e5f9a0 admin admin] Validating token access rules against request {{(pid=80378) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:17:29.107773 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG dbcounter [-] [80377] Writing DB stats keystone:SELECT=94,keystone:INSERT=6 {{(pid=80377) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 10:17:29.120487 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b881fc7-9b25-4b73-ba57-af2013f4a6e7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '891005fa5ff74041b75747d16cc66798', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7047912c5acd4adeba5d87c3bf895e39', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80378) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:17:29.121429 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b881fc7-9b25-4b73-ba57-af2013f4a6e7 admin admin] REQUEST_METHOD: `PUT` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:17:29.121834 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b881fc7-9b25-4b73-ba57-af2013f4a6e7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:17:29.122207 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b881fc7-9b25-4b73-ba57-af2013f4a6e7 admin admin] PATH_INFO: `/v3/projects/4d054c07cbf041ea984a4eb93e81733f/users/e4197dceb5de49d0919b88d1010ec076/roles/e1c04b19fc4543b8b54f58bbcffbc598` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:17:29.128172 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: [pid: 80377|app: 0|req: 1066/2124] 10.222.0.82 () {58 vars in 1122 bytes} [Tue Aug 2 10:17:29 2022] GET /identity/v3/roles => generated 1420 bytes in 108 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 02 10:17:29.132454 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-653b62b4-134a-4740-ad60-44a302282a5d admin admin] Authenticating user token {{(pid=80377) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:17:29.162710 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b881fc7-9b25-4b73-ba57-af2013f4a6e7 admin admin] RBAC: Authorizing `identity:create_grant(project_id=4d054c07cbf041ea984a4eb93e81733f, user_id=e4197dceb5de49d0919b88d1010ec076, role_id=e1c04b19fc4543b8b54f58bbcffbc598)` {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:17:29.163743 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-653b62b4-134a-4740-ad60-44a302282a5d admin admin] Validating token access rules against request {{(pid=80377) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:17:29.166181 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b881fc7-9b25-4b73-ba57-af2013f4a6e7 admin admin] RBAC: Authorization granted {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:17:29.189753 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e223e396-ec42-4f1e-ba14-7ac45363d12f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '891005fa5ff74041b75747d16cc66798', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7047912c5acd4adeba5d87c3bf895e39', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80377) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:17:29.191436 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e223e396-ec42-4f1e-ba14-7ac45363d12f admin admin] REQUEST_METHOD: `PUT` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:17:29.192231 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e223e396-ec42-4f1e-ba14-7ac45363d12f admin admin] SCRIPT_NAME: `/identity` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:17:29.192657 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e223e396-ec42-4f1e-ba14-7ac45363d12f admin admin] PATH_INFO: `/v3/projects/48943a69c8854fcebfd985ee35c6643e/users/ad4d220ace114ad09070199661f3c6fc/roles/237966442f05411191105bf336bc14f3` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:17:29.196169 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 1059/2125] 10.222.0.82 () {60 vars in 1483 bytes} [Tue Aug 2 10:17:29 2022] PUT /identity/v3/projects/4d054c07cbf041ea984a4eb93e81733f/users/e4197dceb5de49d0919b88d1010ec076/roles/e1c04b19fc4543b8b54f58bbcffbc598 => generated 0 bytes in 115 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 02 10:17:29.209836 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc958ad3-7028-4bca-ada8-cdbf32e629cc None None] REQUEST_METHOD: `POST` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:17:29.210278 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc958ad3-7028-4bca-ada8-cdbf32e629cc None None] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:17:29.210699 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc958ad3-7028-4bca-ada8-cdbf32e629cc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:17:29.227195 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e223e396-ec42-4f1e-ba14-7ac45363d12f admin admin] RBAC: Authorizing `identity:create_grant(project_id=48943a69c8854fcebfd985ee35c6643e, user_id=ad4d220ace114ad09070199661f3c6fc, role_id=237966442f05411191105bf336bc14f3)` {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:17:29.231300 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e223e396-ec42-4f1e-ba14-7ac45363d12f admin admin] RBAC: Authorization granted {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:17:29.258284 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.auth.core [None req-fc958ad3-7028-4bca-ada8-cdbf32e629cc None None] MFA Rules not processed for user `e4197dceb5de49d0919b88d1010ec076`. Rule list: `[]` (Enabled: `True`). {{(pid=80378) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 02 10:17:29.264837 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.fernet_utils [None req-fc958ad3-7028-4bca-ada8-cdbf32e629cc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80378) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 10:17:29.270077 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: [pid: 80377|app: 0|req: 1067/2126] 10.222.0.82 () {60 vars in 1483 bytes} [Tue Aug 2 10:17:29 2022] PUT /identity/v3/projects/48943a69c8854fcebfd985ee35c6643e/users/ad4d220ace114ad09070199661f3c6fc/roles/237966442f05411191105bf336bc14f3 => generated 0 bytes in 138 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 02 10:17:29.274555 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e223e396-ec42-4f1e-ba14-7ac45363d12f admin admin] Authenticating user token {{(pid=80377) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:17:29.354839 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG dbcounter [-] [80378] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80378) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 10:17:29.442502 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e223e396-ec42-4f1e-ba14-7ac45363d12f admin admin] Validating token access rules against request {{(pid=80377) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:17:29.443729 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 1060/2127] 10.222.0.82 () {58 vars in 958 bytes} [Tue Aug 2 10:17:29 2022] POST /identity/v3/auth/tokens => generated 3262 bytes in 235 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 02 10:17:29.454088 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d972437-7dcb-4977-9757-43de6239e791 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '891005fa5ff74041b75747d16cc66798', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7047912c5acd4adeba5d87c3bf895e39', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80377) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:17:29.455015 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d972437-7dcb-4977-9757-43de6239e791 admin admin] REQUEST_METHOD: `GET` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:17:29.455206 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d972437-7dcb-4977-9757-43de6239e791 admin admin] SCRIPT_NAME: `/identity` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:17:29.455490 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d972437-7dcb-4977-9757-43de6239e791 admin admin] PATH_INFO: `/v3/roles` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:17:29.456243 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d972437-7dcb-4977-9757-43de6239e791 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:17:29.457995 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d972437-7dcb-4977-9757-43de6239e791 admin admin] RBAC: Authorization granted {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:17:29.469146 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc958ad3-7028-4bca-ada8-cdbf32e629cc None None] Authenticating user token {{(pid=80378) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:17:29.470284 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: [pid: 80377|app: 0|req: 1068/2128] 10.222.0.82 () {58 vars in 1122 bytes} [Tue Aug 2 10:17:29 2022] GET /identity/v3/roles => generated 1420 bytes in 197 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 02 10:17:29.471359 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.fernet_utils [None req-fc958ad3-7028-4bca-ada8-cdbf32e629cc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80378) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 10:17:29.478783 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d972437-7dcb-4977-9757-43de6239e791 admin admin] Authenticating user token {{(pid=80377) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:17:29.499829 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d972437-7dcb-4977-9757-43de6239e791 admin admin] Validating token access rules against request {{(pid=80377) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:17:29.506929 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e7060c3-da80-4588-b5d7-422132a564d3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '891005fa5ff74041b75747d16cc66798', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7047912c5acd4adeba5d87c3bf895e39', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80377) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:17:29.507791 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e7060c3-da80-4588-b5d7-422132a564d3 admin admin] REQUEST_METHOD: `PUT` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:17:29.508177 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e7060c3-da80-4588-b5d7-422132a564d3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:17:29.508601 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e7060c3-da80-4588-b5d7-422132a564d3 admin admin] PATH_INFO: `/v3/projects/48943a69c8854fcebfd985ee35c6643e/users/ad4d220ace114ad09070199661f3c6fc/roles/237966442f05411191105bf336bc14f3` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:17:29.510921 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e7060c3-da80-4588-b5d7-422132a564d3 admin admin] RBAC: Authorizing `identity:create_grant(project_id=48943a69c8854fcebfd985ee35c6643e, user_id=ad4d220ace114ad09070199661f3c6fc, role_id=237966442f05411191105bf336bc14f3)` {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:17:29.514230 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e7060c3-da80-4588-b5d7-422132a564d3 admin admin] RBAC: Authorization granted {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:17:29.532032 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: [pid: 80377|app: 0|req: 1069/2129] 10.222.0.82 () {60 vars in 1483 bytes} [Tue Aug 2 10:17:29 2022] PUT /identity/v3/projects/48943a69c8854fcebfd985ee35c6643e/users/ad4d220ace114ad09070199661f3c6fc/roles/237966442f05411191105bf336bc14f3 => generated 0 bytes in 56 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 02 10:17:29.544216 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ee25b5d-9fbc-4d63-8c6f-84b7822c3b2d None None] REQUEST_METHOD: `POST` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:17:29.545251 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ee25b5d-9fbc-4d63-8c6f-84b7822c3b2d None None] SCRIPT_NAME: `/identity` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:17:29.545640 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ee25b5d-9fbc-4d63-8c6f-84b7822c3b2d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:17:29.619272 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.auth.core [None req-9ee25b5d-9fbc-4d63-8c6f-84b7822c3b2d None None] MFA Rules not processed for user `ad4d220ace114ad09070199661f3c6fc`. Rule list: `[]` (Enabled: `True`). {{(pid=80377) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 02 10:17:29.621215 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.fernet_utils [None req-9ee25b5d-9fbc-4d63-8c6f-84b7822c3b2d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80377) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 10:17:29.693402 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc958ad3-7028-4bca-ada8-cdbf32e629cc None None] Validating token access rules against request {{(pid=80378) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:17:29.798913 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2d5ba77-738d-4e71-9012-2f867f0b577f service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1023de281ae74e9d8a6d6c312c1c97a7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f1b38ebb804c41c69772c998aae72c41', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80378) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:17:29.799876 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2d5ba77-738d-4e71-9012-2f867f0b577f service cinder] REQUEST_METHOD: `GET` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:17:29.802173 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2d5ba77-738d-4e71-9012-2f867f0b577f service cinder] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:17:29.802636 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2d5ba77-738d-4e71-9012-2f867f0b577f service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:17:29.805939 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: [pid: 80377|app: 0|req: 1070/2130] 10.222.0.82 () {58 vars in 958 bytes} [Tue Aug 2 10:17:29 2022] POST /identity/v3/auth/tokens => generated 3327 bytes in 263 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 02 10:17:29.816155 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ee25b5d-9fbc-4d63-8c6f-84b7822c3b2d None None] Authenticating user token {{(pid=80377) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:17:29.842149 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ee25b5d-9fbc-4d63-8c6f-84b7822c3b2d None None] Validating token access rules against request {{(pid=80377) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:17:29.854274 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69c2e686-c3f6-4fad-99a4-a2fd3bc96366 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1023de281ae74e9d8a6d6c312c1c97a7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f1b38ebb804c41c69772c998aae72c41', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80377) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:17:29.854917 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69c2e686-c3f6-4fad-99a4-a2fd3bc96366 service cinder] REQUEST_METHOD: `GET` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:17:29.857707 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69c2e686-c3f6-4fad-99a4-a2fd3bc96366 service cinder] SCRIPT_NAME: `/identity` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:17:29.858054 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69c2e686-c3f6-4fad-99a4-a2fd3bc96366 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:17:29.878409 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69c2e686-c3f6-4fad-99a4-a2fd3bc96366 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:17:29.880919 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69c2e686-c3f6-4fad-99a4-a2fd3bc96366 service cinder] RBAC: Authorization granted {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:17:29.895344 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: [pid: 80377|app: 0|req: 1071/2131] 10.222.0.82 () {60 vars in 1365 bytes} [Tue Aug 2 10:17:29 2022] GET /identity/v3/auth/tokens => generated 3327 bytes in 80 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 02 10:17:29.911193 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2d5ba77-738d-4e71-9012-2f867f0b577f service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:17:29.912962 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2d5ba77-738d-4e71-9012-2f867f0b577f service cinder] RBAC: Authorization granted {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:17:29.936385 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 1061/2132] 10.222.0.82 () {60 vars in 1365 bytes} [Tue Aug 2 10:17:29 2022] GET /identity/v3/auth/tokens => generated 3262 bytes in 468 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 02 10:17:29.954861 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69c2e686-c3f6-4fad-99a4-a2fd3bc96366 service cinder] Authenticating user token {{(pid=80377) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:17:30.017518 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2d5ba77-738d-4e71-9012-2f867f0b577f service cinder] Authenticating user token {{(pid=80378) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:17:30.027125 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG dbcounter [-] [80377] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80377) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 10:17:30.030909 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2d5ba77-738d-4e71-9012-2f867f0b577f service cinder] Validating token access rules against request {{(pid=80378) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:17:30.039134 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25f14d13-e634-428e-8bef-2cc98ef9a54b service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1023de281ae74e9d8a6d6c312c1c97a7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f1b38ebb804c41c69772c998aae72c41', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80378) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:17:30.040048 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25f14d13-e634-428e-8bef-2cc98ef9a54b service cinder] REQUEST_METHOD: `GET` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:17:30.040409 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25f14d13-e634-428e-8bef-2cc98ef9a54b service cinder] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:17:30.040816 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25f14d13-e634-428e-8bef-2cc98ef9a54b service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:17:30.087730 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69c2e686-c3f6-4fad-99a4-a2fd3bc96366 service cinder] Validating token access rules against request {{(pid=80377) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:17:30.105859 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee06bfef-b8ee-4319-9fa1-cb44a441d47b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '891005fa5ff74041b75747d16cc66798', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7047912c5acd4adeba5d87c3bf895e39', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80377) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:17:30.109014 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee06bfef-b8ee-4319-9fa1-cb44a441d47b admin admin] REQUEST_METHOD: `DELETE` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:17:30.109972 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee06bfef-b8ee-4319-9fa1-cb44a441d47b admin admin] SCRIPT_NAME: `/identity` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:17:30.110471 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee06bfef-b8ee-4319-9fa1-cb44a441d47b admin admin] PATH_INFO: `/v3/users/e4197dceb5de49d0919b88d1010ec076` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:17:30.114398 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee06bfef-b8ee-4319-9fa1-cb44a441d47b admin admin] RBAC: Authorizing `identity:delete_user(user_id=e4197dceb5de49d0919b88d1010ec076)` {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:17:30.119372 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee06bfef-b8ee-4319-9fa1-cb44a441d47b admin admin] RBAC: Authorization granted {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:17:30.168206 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG dbcounter [-] [80378] Writing DB stats keystone:SELECT=100 {{(pid=80378) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 10:17:30.206304 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25f14d13-e634-428e-8bef-2cc98ef9a54b service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:17:30.210692 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25f14d13-e634-428e-8bef-2cc98ef9a54b service cinder] RBAC: Authorization granted {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:17:30.222300 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.notifications [None req-ee06bfef-b8ee-4319-9fa1-cb44a441d47b admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'e4197dceb5de49d0919b88d1010ec076'} {{(pid=80377) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 10:17:30.234576 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.notifications [None req-ee06bfef-b8ee-4319-9fa1-cb44a441d47b admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'e4197dceb5de49d0919b88d1010ec076'} {{(pid=80377) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 10:17:30.247044 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 1062/2133] 10.222.0.82 () {60 vars in 1365 bytes} [Tue Aug 2 10:17:30 2022] GET /identity/v3/auth/tokens => generated 3258 bytes in 230 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 02 10:17:30.251218 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.notifications [None req-ee06bfef-b8ee-4319-9fa1-cb44a441d47b admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'e4197dceb5de49d0919b88d1010ec076'} {{(pid=80377) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 10:17:30.252237 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.notifications [None req-ee06bfef-b8ee-4319-9fa1-cb44a441d47b admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'e4197dceb5de49d0919b88d1010ec076'} {{(pid=80377) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 10:17:30.279748 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.notifications [None req-ee06bfef-b8ee-4319-9fa1-cb44a441d47b admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'e4197dceb5de49d0919b88d1010ec076'} {{(pid=80377) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 10:17:30.320565 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: [pid: 80377|app: 0|req: 1072/2134] 10.222.0.82 () {58 vars in 1224 bytes} [Tue Aug 2 10:17:29 2022] DELETE /identity/v3/users/e4197dceb5de49d0919b88d1010ec076 => generated 0 bytes in 368 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 02 10:17:30.327667 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af707748-87c7-4402-a1cd-6d2b08d6a113 None None] REQUEST_METHOD: `POST` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:17:30.328102 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af707748-87c7-4402-a1cd-6d2b08d6a113 None None] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:17:30.328482 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af707748-87c7-4402-a1cd-6d2b08d6a113 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:17:30.370675 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.auth.core [None req-af707748-87c7-4402-a1cd-6d2b08d6a113 None None] MFA Rules not processed for user `891005fa5ff74041b75747d16cc66798`. Rule list: `[]` (Enabled: `True`). {{(pid=80378) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 02 10:17:30.373808 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.fernet_utils [None req-af707748-87c7-4402-a1cd-6d2b08d6a113 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80378) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 10:17:30.565244 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 1063/2135] 10.222.0.82 () {58 vars in 958 bytes} [Tue Aug 2 10:17:30 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 237 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 02 10:17:30.619379 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee06bfef-b8ee-4319-9fa1-cb44a441d47b admin admin] Authenticating user token {{(pid=80377) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:17:30.621221 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.fernet_utils [None req-ee06bfef-b8ee-4319-9fa1-cb44a441d47b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80377) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 10:17:30.764164 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee06bfef-b8ee-4319-9fa1-cb44a441d47b admin admin] Validating token access rules against request {{(pid=80377) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:17:30.774980 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b86ea874-2fc4-4adb-b63c-fdd7150455c8 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '33b87170f0634b67bce60750847b5cbe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f1b38ebb804c41c69772c998aae72c41', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80377) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:17:30.775635 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b86ea874-2fc4-4adb-b63c-fdd7150455c8 service neutron] REQUEST_METHOD: `GET` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:17:30.775918 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b86ea874-2fc4-4adb-b63c-fdd7150455c8 service neutron] SCRIPT_NAME: `/identity` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:17:30.776183 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b86ea874-2fc4-4adb-b63c-fdd7150455c8 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:17:30.795817 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b86ea874-2fc4-4adb-b63c-fdd7150455c8 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:17:30.799096 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b86ea874-2fc4-4adb-b63c-fdd7150455c8 service neutron] RBAC: Authorization granted {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:17:30.812166 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: [pid: 80377|app: 0|req: 1073/2136] 10.222.0.82 () {60 vars in 1365 bytes} [Tue Aug 2 10:17:30 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 194 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 02 10:17:30.880657 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af707748-87c7-4402-a1cd-6d2b08d6a113 None None] Authenticating user token {{(pid=80378) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:17:30.882753 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.fernet_utils [None req-af707748-87c7-4402-a1cd-6d2b08d6a113 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80378) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 10:17:30.942548 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af707748-87c7-4402-a1cd-6d2b08d6a113 None None] Validating token access rules against request {{(pid=80378) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:17:30.988859 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-665ac523-91f4-477f-b35a-31afea707300 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '891005fa5ff74041b75747d16cc66798', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7047912c5acd4adeba5d87c3bf895e39', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80378) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:17:30.992498 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-665ac523-91f4-477f-b35a-31afea707300 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:17:30.992922 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-665ac523-91f4-477f-b35a-31afea707300 admin admin] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:17:30.993278 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-665ac523-91f4-477f-b35a-31afea707300 admin admin] PATH_INFO: `/v3/users/3d76ef766e9b4f608240cffcbf2b5906` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:17:30.999026 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-665ac523-91f4-477f-b35a-31afea707300 admin admin] RBAC: Authorizing `identity:delete_user(user_id=3d76ef766e9b4f608240cffcbf2b5906)` {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:17:31.001148 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-665ac523-91f4-477f-b35a-31afea707300 admin admin] RBAC: Authorization granted {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:17:31.109895 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.notifications [None req-665ac523-91f4-477f-b35a-31afea707300 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '3d76ef766e9b4f608240cffcbf2b5906'} {{(pid=80378) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 10:17:31.132023 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.notifications [None req-665ac523-91f4-477f-b35a-31afea707300 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '3d76ef766e9b4f608240cffcbf2b5906'} {{(pid=80378) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 10:17:31.159141 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.notifications [None req-665ac523-91f4-477f-b35a-31afea707300 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '3d76ef766e9b4f608240cffcbf2b5906'} {{(pid=80378) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 10:17:31.160400 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.notifications [None req-665ac523-91f4-477f-b35a-31afea707300 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '3d76ef766e9b4f608240cffcbf2b5906'} {{(pid=80378) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 10:17:31.226022 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.notifications [None req-665ac523-91f4-477f-b35a-31afea707300 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '3d76ef766e9b4f608240cffcbf2b5906'} {{(pid=80378) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 10:17:31.255124 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 1064/2137] 10.222.0.82 () {58 vars in 1224 bytes} [Tue Aug 2 10:17:30 2022] DELETE /identity/v3/users/3d76ef766e9b4f608240cffcbf2b5906 => generated 0 bytes in 375 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 02 10:17:31.272310 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b9f02e1-e7f8-474c-9c12-1bf763547977 None None] REQUEST_METHOD: `POST` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:17:31.272601 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b9f02e1-e7f8-474c-9c12-1bf763547977 None None] SCRIPT_NAME: `/identity` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:17:31.272935 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b9f02e1-e7f8-474c-9c12-1bf763547977 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:17:31.307065 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.auth.core [None req-6b9f02e1-e7f8-474c-9c12-1bf763547977 None None] MFA Rules not processed for user `891005fa5ff74041b75747d16cc66798`. Rule list: `[]` (Enabled: `True`). {{(pid=80377) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 02 10:17:31.311149 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.fernet_utils [None req-6b9f02e1-e7f8-474c-9c12-1bf763547977 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80377) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 10:17:31.392557 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG dbcounter [-] [80377] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80377) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 10:17:31.419850 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: [pid: 80377|app: 0|req: 1074/2138] 10.222.0.82 () {58 vars in 958 bytes} [Tue Aug 2 10:17:31 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 149 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 02 10:17:31.476332 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-665ac523-91f4-477f-b35a-31afea707300 admin admin] Authenticating user token {{(pid=80378) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:17:31.478507 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.fernet_utils [None req-665ac523-91f4-477f-b35a-31afea707300 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80378) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 10:17:31.682651 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-665ac523-91f4-477f-b35a-31afea707300 admin admin] Validating token access rules against request {{(pid=80378) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:17:31.688329 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-093b0e94-80ad-4028-ae15-0c4d58905ed8 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '33b87170f0634b67bce60750847b5cbe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f1b38ebb804c41c69772c998aae72c41', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80378) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:17:31.689233 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-093b0e94-80ad-4028-ae15-0c4d58905ed8 service neutron] REQUEST_METHOD: `GET` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:17:31.689565 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-093b0e94-80ad-4028-ae15-0c4d58905ed8 service neutron] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:17:31.692234 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-093b0e94-80ad-4028-ae15-0c4d58905ed8 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:17:31.713971 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-093b0e94-80ad-4028-ae15-0c4d58905ed8 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:17:31.716374 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-093b0e94-80ad-4028-ae15-0c4d58905ed8 service neutron] RBAC: Authorization granted {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:17:31.729293 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 1065/2139] 10.222.0.82 () {60 vars in 1365 bytes} [Tue Aug 2 10:17:31 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 255 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 02 10:17:32.367655 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b9f02e1-e7f8-474c-9c12-1bf763547977 None None] Authenticating user token {{(pid=80377) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:17:32.370632 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.fernet_utils [None req-6b9f02e1-e7f8-474c-9c12-1bf763547977 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80377) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 10:17:32.397557 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b9f02e1-e7f8-474c-9c12-1bf763547977 None None] Validating token access rules against request {{(pid=80377) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:17:32.414149 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a257cbbb-ab67-4a16-96cc-193d5f6c9844 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '891005fa5ff74041b75747d16cc66798', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7047912c5acd4adeba5d87c3bf895e39', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80377) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:17:32.414618 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a257cbbb-ab67-4a16-96cc-193d5f6c9844 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:17:32.414910 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a257cbbb-ab67-4a16-96cc-193d5f6c9844 admin admin] SCRIPT_NAME: `/identity` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:17:32.415341 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a257cbbb-ab67-4a16-96cc-193d5f6c9844 admin admin] PATH_INFO: `/v3/users/4a3080d2fc214bfc99cadab7c7e4ea89` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:17:32.420942 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a257cbbb-ab67-4a16-96cc-193d5f6c9844 admin admin] RBAC: Authorizing `identity:delete_user(user_id=4a3080d2fc214bfc99cadab7c7e4ea89)` {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:17:32.424668 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a257cbbb-ab67-4a16-96cc-193d5f6c9844 admin admin] RBAC: Authorization granted {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:17:32.506589 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-093b0e94-80ad-4028-ae15-0c4d58905ed8 service neutron] Authenticating user token {{(pid=80378) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:17:32.510428 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.fernet_utils [None req-093b0e94-80ad-4028-ae15-0c4d58905ed8 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80378) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 10:17:32.523693 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG dbcounter [-] [80378] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80378) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 10:17:32.544249 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-093b0e94-80ad-4028-ae15-0c4d58905ed8 service neutron] Validating token access rules against request {{(pid=80378) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:17:32.554598 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.notifications [None req-a257cbbb-ab67-4a16-96cc-193d5f6c9844 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '4a3080d2fc214bfc99cadab7c7e4ea89'} {{(pid=80377) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 10:17:32.561318 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3eaf4bf-57c6-441b-8f4e-b6a1e9498bea admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '891005fa5ff74041b75747d16cc66798', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7047912c5acd4adeba5d87c3bf895e39', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80378) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:17:32.562099 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3eaf4bf-57c6-441b-8f4e-b6a1e9498bea admin admin] REQUEST_METHOD: `DELETE` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:17:32.562385 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3eaf4bf-57c6-441b-8f4e-b6a1e9498bea admin admin] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:17:32.562667 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3eaf4bf-57c6-441b-8f4e-b6a1e9498bea admin admin] PATH_INFO: `/v3/projects/4d054c07cbf041ea984a4eb93e81733f` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:17:32.564386 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3eaf4bf-57c6-441b-8f4e-b6a1e9498bea admin admin] RBAC: Authorizing `identity:delete_project(project_id=4d054c07cbf041ea984a4eb93e81733f)` {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:17:32.566275 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3eaf4bf-57c6-441b-8f4e-b6a1e9498bea admin admin] RBAC: Authorization granted {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:17:32.569832 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.notifications [None req-a257cbbb-ab67-4a16-96cc-193d5f6c9844 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '4a3080d2fc214bfc99cadab7c7e4ea89'} {{(pid=80377) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 10:17:32.595831 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.notifications [None req-a257cbbb-ab67-4a16-96cc-193d5f6c9844 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '4a3080d2fc214bfc99cadab7c7e4ea89'} {{(pid=80377) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 10:17:32.596639 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.notifications [None req-a257cbbb-ab67-4a16-96cc-193d5f6c9844 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '4a3080d2fc214bfc99cadab7c7e4ea89'} {{(pid=80377) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 10:17:32.625139 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.notifications [None req-a257cbbb-ab67-4a16-96cc-193d5f6c9844 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '4a3080d2fc214bfc99cadab7c7e4ea89'} {{(pid=80377) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 10:17:32.640069 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: [pid: 80377|app: 0|req: 1075/2140] 10.222.0.82 () {58 vars in 1224 bytes} [Tue Aug 2 10:17:32 2022] DELETE /identity/v3/users/4a3080d2fc214bfc99cadab7c7e4ea89 => generated 0 bytes in 273 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 02 10:17:32.644804 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-310730d1-a70e-498b-abc5-3470a722b740 None None] REQUEST_METHOD: `POST` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:17:32.645093 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-310730d1-a70e-498b-abc5-3470a722b740 None None] SCRIPT_NAME: `/identity` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:17:32.645371 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-310730d1-a70e-498b-abc5-3470a722b740 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:17:32.681873 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.auth.core [None req-310730d1-a70e-498b-abc5-3470a722b740 None None] MFA Rules not processed for user `891005fa5ff74041b75747d16cc66798`. Rule list: `[]` (Enabled: `True`). {{(pid=80377) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 02 10:17:32.685971 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.fernet_utils [None req-310730d1-a70e-498b-abc5-3470a722b740 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80377) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 10:17:32.704727 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.notifications [None req-d3eaf4bf-57c6-441b-8f4e-b6a1e9498bea admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '4d054c07cbf041ea984a4eb93e81733f', 'request_id': 'req-d3eaf4bf-57c6-441b-8f4e-b6a1e9498bea'} {{(pid=80378) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 10:17:32.719227 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.notifications [None req-d3eaf4bf-57c6-441b-8f4e-b6a1e9498bea admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '4d054c07cbf041ea984a4eb93e81733f', 'request_id': 'req-d3eaf4bf-57c6-441b-8f4e-b6a1e9498bea'} {{(pid=80378) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 10:17:32.731014 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.notifications [None req-d3eaf4bf-57c6-441b-8f4e-b6a1e9498bea admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '4d054c07cbf041ea984a4eb93e81733f', 'request_id': 'req-d3eaf4bf-57c6-441b-8f4e-b6a1e9498bea'} {{(pid=80378) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 10:17:32.776245 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.notifications [None req-d3eaf4bf-57c6-441b-8f4e-b6a1e9498bea admin admin] The token cache is being invalidate because project 4d054c07cbf041ea984a4eb93e81733f was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80378) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 02 10:17:32.776644 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.notifications [None req-d3eaf4bf-57c6-441b-8f4e-b6a1e9498bea admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80378) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 10:17:32.777609 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.notifications [None req-d3eaf4bf-57c6-441b-8f4e-b6a1e9498bea admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80378) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 10:17:32.795195 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 1066/2141] 10.222.0.82 () {58 vars in 1233 bytes} [Tue Aug 2 10:17:32 2022] DELETE /identity/v3/projects/4d054c07cbf041ea984a4eb93e81733f => generated 0 bytes in 289 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 02 10:17:32.801043 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb7b2d8a-e129-44fb-9484-9af8b0fce6b2 None None] REQUEST_METHOD: `POST` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:17:32.801347 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb7b2d8a-e129-44fb-9484-9af8b0fce6b2 None None] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:17:32.801631 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb7b2d8a-e129-44fb-9484-9af8b0fce6b2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:17:32.835291 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.auth.core [None req-bb7b2d8a-e129-44fb-9484-9af8b0fce6b2 None None] MFA Rules not processed for user `891005fa5ff74041b75747d16cc66798`. Rule list: `[]` (Enabled: `True`). {{(pid=80378) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 02 10:17:32.842256 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.fernet_utils [None req-bb7b2d8a-e129-44fb-9484-9af8b0fce6b2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80378) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 10:17:32.908161 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: [pid: 80377|app: 0|req: 1076/2142] 10.222.0.82 () {58 vars in 958 bytes} [Tue Aug 2 10:17:32 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 264 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 02 10:17:32.910017 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-310730d1-a70e-498b-abc5-3470a722b740 None None] Authenticating user token {{(pid=80377) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:17:32.912792 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.fernet_utils [None req-310730d1-a70e-498b-abc5-3470a722b740 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80377) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 10:17:32.953934 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 1067/2143] 10.222.0.82 () {58 vars in 958 bytes} [Tue Aug 2 10:17:32 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 153 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 02 10:17:32.964103 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9d76e7e-d3a5-4916-95a8-26c0640329c7 None None] REQUEST_METHOD: `POST` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:17:32.964702 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9d76e7e-d3a5-4916-95a8-26c0640329c7 None None] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:17:32.965645 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9d76e7e-d3a5-4916-95a8-26c0640329c7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:17:32.993948 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.auth.core [None req-f9d76e7e-d3a5-4916-95a8-26c0640329c7 None None] MFA Rules not processed for user `891005fa5ff74041b75747d16cc66798`. Rule list: `[]` (Enabled: `True`). {{(pid=80378) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 02 10:17:32.995700 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.fernet_utils [None req-f9d76e7e-d3a5-4916-95a8-26c0640329c7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80378) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 10:17:33.006279 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 1068/2144] 10.222.0.82 () {58 vars in 958 bytes} [Tue Aug 2 10:17:32 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 43 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 02 10:17:33.013391 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9d76e7e-d3a5-4916-95a8-26c0640329c7 None None] Authenticating user token {{(pid=80378) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:17:33.025571 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9d76e7e-d3a5-4916-95a8-26c0640329c7 None None] Validating token access rules against request {{(pid=80378) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:17:33.031483 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b76be7b-01a4-4aba-996b-861ab81941df admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '891005fa5ff74041b75747d16cc66798', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7047912c5acd4adeba5d87c3bf895e39', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80378) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:17:33.032278 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b76be7b-01a4-4aba-996b-861ab81941df admin admin] REQUEST_METHOD: `GET` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:17:33.032785 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b76be7b-01a4-4aba-996b-861ab81941df admin admin] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:17:33.033140 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b76be7b-01a4-4aba-996b-861ab81941df admin admin] PATH_INFO: `/v3/domains` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:17:33.034168 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b76be7b-01a4-4aba-996b-861ab81941df admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80378) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 02 10:17:33.034782 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b76be7b-01a4-4aba-996b-861ab81941df admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:17:33.038317 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b76be7b-01a4-4aba-996b-861ab81941df admin admin] RBAC: Authorization granted {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:17:33.056725 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 1069/2145] 10.222.0.82 () {58 vars in 1153 bytes} [Tue Aug 2 10:17:33 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 44 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 02 10:17:33.063219 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b76be7b-01a4-4aba-996b-861ab81941df admin admin] Authenticating user token {{(pid=80378) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:17:33.084257 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-310730d1-a70e-498b-abc5-3470a722b740 None None] Validating token access rules against request {{(pid=80377) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:17:33.091234 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b76be7b-01a4-4aba-996b-861ab81941df admin admin] Validating token access rules against request {{(pid=80378) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:17:33.093010 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ed68795-a0c0-4508-80d9-24d3a8903711 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '33b87170f0634b67bce60750847b5cbe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f1b38ebb804c41c69772c998aae72c41', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80377) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:17:33.094443 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ed68795-a0c0-4508-80d9-24d3a8903711 service neutron] REQUEST_METHOD: `GET` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:17:33.094596 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ed68795-a0c0-4508-80d9-24d3a8903711 service neutron] SCRIPT_NAME: `/identity` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:17:33.094769 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ed68795-a0c0-4508-80d9-24d3a8903711 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:17:33.101829 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-deb1afcc-7a67-4188-a835-10c1fe72744c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '891005fa5ff74041b75747d16cc66798', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7047912c5acd4adeba5d87c3bf895e39', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80378) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:17:33.102508 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-deb1afcc-7a67-4188-a835-10c1fe72744c admin admin] REQUEST_METHOD: `POST` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:17:33.102787 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-deb1afcc-7a67-4188-a835-10c1fe72744c admin admin] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:17:33.103067 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-deb1afcc-7a67-4188-a835-10c1fe72744c admin admin] PATH_INFO: `/v3/projects` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:17:33.104857 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-deb1afcc-7a67-4188-a835-10c1fe72744c admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:17:33.112162 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-deb1afcc-7a67-4188-a835-10c1fe72744c admin admin] RBAC: Authorization granted {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:17:33.121351 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ed68795-a0c0-4508-80d9-24d3a8903711 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:17:33.125387 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ed68795-a0c0-4508-80d9-24d3a8903711 service neutron] RBAC: Authorization granted {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:17:33.155273 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: [pid: 80377|app: 0|req: 1077/2146] 10.222.0.82 () {60 vars in 1365 bytes} [Tue Aug 2 10:17:32 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 246 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 02 10:17:33.189126 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 1070/2147] 10.222.0.82 () {60 vars in 1153 bytes} [Tue Aug 2 10:17:33 2022] POST /identity/v3/projects => generated 373 bytes in 128 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 02 10:17:33.192779 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ed68795-a0c0-4508-80d9-24d3a8903711 service neutron] Authenticating user token {{(pid=80377) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:17:33.204765 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ed68795-a0c0-4508-80d9-24d3a8903711 service neutron] Validating token access rules against request {{(pid=80377) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:17:33.216249 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG dbcounter [-] [80377] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80377) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 10:17:33.324124 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79d5af7d-b020-4905-a459-421041931f48 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '891005fa5ff74041b75747d16cc66798', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7047912c5acd4adeba5d87c3bf895e39', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80377) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:17:33.324789 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79d5af7d-b020-4905-a459-421041931f48 admin admin] REQUEST_METHOD: `POST` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:17:33.325164 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79d5af7d-b020-4905-a459-421041931f48 admin admin] SCRIPT_NAME: `/identity` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:17:33.325870 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79d5af7d-b020-4905-a459-421041931f48 admin admin] PATH_INFO: `/v3/users` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:17:33.326658 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79d5af7d-b020-4905-a459-421041931f48 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:17:33.328910 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79d5af7d-b020-4905-a459-421041931f48 admin admin] RBAC: Authorization granted {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:17:33.369734 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: [pid: 80377|app: 0|req: 1078/2148] 10.222.0.82 () {60 vars in 1144 bytes} [Tue Aug 2 10:17:33 2022] POST /identity/v3/users => generated 287 bytes in 175 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 02 10:17:33.371808 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-deb1afcc-7a67-4188-a835-10c1fe72744c admin admin] Authenticating user token {{(pid=80378) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:17:33.388147 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-deb1afcc-7a67-4188-a835-10c1fe72744c admin admin] Validating token access rules against request {{(pid=80378) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:17:33.404189 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c52b453-7c9d-4228-8230-178c7b0fac14 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '891005fa5ff74041b75747d16cc66798', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7047912c5acd4adeba5d87c3bf895e39', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80378) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:17:33.405649 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c52b453-7c9d-4228-8230-178c7b0fac14 admin admin] REQUEST_METHOD: `GET` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:17:33.406087 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c52b453-7c9d-4228-8230-178c7b0fac14 admin admin] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:17:33.407811 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c52b453-7c9d-4228-8230-178c7b0fac14 admin admin] PATH_INFO: `/v3/roles` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:17:33.411234 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c52b453-7c9d-4228-8230-178c7b0fac14 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:17:33.416450 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c52b453-7c9d-4228-8230-178c7b0fac14 admin admin] RBAC: Authorization granted {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:17:33.444689 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 1071/2149] 10.222.0.82 () {58 vars in 1122 bytes} [Tue Aug 2 10:17:33 2022] GET /identity/v3/roles => generated 1420 bytes in 71 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 02 10:17:33.447163 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79d5af7d-b020-4905-a459-421041931f48 admin admin] Authenticating user token {{(pid=80377) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:17:33.475143 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79d5af7d-b020-4905-a459-421041931f48 admin admin] Validating token access rules against request {{(pid=80377) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:17:33.486663 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77c88731-2702-4070-9a67-b32a7182d899 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '891005fa5ff74041b75747d16cc66798', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7047912c5acd4adeba5d87c3bf895e39', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80377) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:17:33.489281 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77c88731-2702-4070-9a67-b32a7182d899 admin admin] REQUEST_METHOD: `GET` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:17:33.489818 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77c88731-2702-4070-9a67-b32a7182d899 admin admin] SCRIPT_NAME: `/identity` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:17:33.490226 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77c88731-2702-4070-9a67-b32a7182d899 admin admin] PATH_INFO: `/v3/roles` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:17:33.491411 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77c88731-2702-4070-9a67-b32a7182d899 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:17:33.494108 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77c88731-2702-4070-9a67-b32a7182d899 admin admin] RBAC: Authorization granted {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:17:33.515845 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c52b453-7c9d-4228-8230-178c7b0fac14 admin admin] Authenticating user token {{(pid=80378) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:17:33.524716 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: [pid: 80377|app: 0|req: 1079/2150] 10.222.0.82 () {58 vars in 1122 bytes} [Tue Aug 2 10:17:33 2022] GET /identity/v3/roles => generated 1420 bytes in 77 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 02 10:17:33.532355 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c52b453-7c9d-4228-8230-178c7b0fac14 admin admin] Validating token access rules against request {{(pid=80378) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:17:33.546525 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3249fe17-c01a-4410-93a7-58b459a12ae2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '891005fa5ff74041b75747d16cc66798', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7047912c5acd4adeba5d87c3bf895e39', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80378) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:17:33.547639 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3249fe17-c01a-4410-93a7-58b459a12ae2 admin admin] REQUEST_METHOD: `PUT` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:17:33.548352 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3249fe17-c01a-4410-93a7-58b459a12ae2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:17:33.549303 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3249fe17-c01a-4410-93a7-58b459a12ae2 admin admin] PATH_INFO: `/v3/projects/5704398ca48145f0920db111e88a7455/users/9210e6a584144415acee9667164fd8ff/roles/e1c04b19fc4543b8b54f58bbcffbc598` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:17:33.594673 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3249fe17-c01a-4410-93a7-58b459a12ae2 admin admin] RBAC: Authorizing `identity:create_grant(project_id=5704398ca48145f0920db111e88a7455, user_id=9210e6a584144415acee9667164fd8ff, role_id=e1c04b19fc4543b8b54f58bbcffbc598)` {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:17:33.601643 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3249fe17-c01a-4410-93a7-58b459a12ae2 admin admin] RBAC: Authorization granted {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:17:33.620798 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77c88731-2702-4070-9a67-b32a7182d899 admin admin] Authenticating user token {{(pid=80377) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:17:33.626094 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.fernet_utils [None req-77c88731-2702-4070-9a67-b32a7182d899 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80377) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 10:17:33.632706 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 1072/2151] 10.222.0.82 () {60 vars in 1483 bytes} [Tue Aug 2 10:17:33 2022] PUT /identity/v3/projects/5704398ca48145f0920db111e88a7455/users/9210e6a584144415acee9667164fd8ff/roles/e1c04b19fc4543b8b54f58bbcffbc598 => generated 0 bytes in 117 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 02 10:17:33.654781 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2df150a-4ac0-412d-b2bd-f0ad4d779443 None None] REQUEST_METHOD: `POST` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:17:33.660016 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77c88731-2702-4070-9a67-b32a7182d899 admin admin] Validating token access rules against request {{(pid=80377) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:17:33.667986 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2df150a-4ac0-412d-b2bd-f0ad4d779443 None None] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:17:33.668562 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2df150a-4ac0-412d-b2bd-f0ad4d779443 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:17:33.727703 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG dbcounter [-] [80378] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=80378) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 10:17:33.741912 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.auth.core [None req-e2df150a-4ac0-412d-b2bd-f0ad4d779443 None None] MFA Rules not processed for user `9210e6a584144415acee9667164fd8ff`. Rule list: `[]` (Enabled: `True`). {{(pid=80378) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 02 10:17:33.743872 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.fernet_utils [None req-e2df150a-4ac0-412d-b2bd-f0ad4d779443 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80378) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 10:17:33.771054 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96cb2fdc-b0bf-4f06-b5f3-a534da8fcbea admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '891005fa5ff74041b75747d16cc66798', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7047912c5acd4adeba5d87c3bf895e39', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80377) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:17:33.772000 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96cb2fdc-b0bf-4f06-b5f3-a534da8fcbea admin admin] REQUEST_METHOD: `DELETE` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:17:33.772439 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96cb2fdc-b0bf-4f06-b5f3-a534da8fcbea admin admin] SCRIPT_NAME: `/identity` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:17:33.778744 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96cb2fdc-b0bf-4f06-b5f3-a534da8fcbea admin admin] PATH_INFO: `/v3/projects/5877952b334640b6b9d704e3f1409e7b` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:17:33.782132 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96cb2fdc-b0bf-4f06-b5f3-a534da8fcbea admin admin] RBAC: Authorizing `identity:delete_project(project_id=5877952b334640b6b9d704e3f1409e7b)` {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:17:33.783981 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96cb2fdc-b0bf-4f06-b5f3-a534da8fcbea admin admin] RBAC: Authorization granted {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:17:33.858818 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 1073/2152] 10.222.0.82 () {58 vars in 958 bytes} [Tue Aug 2 10:17:33 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 209 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 02 10:17:33.871974 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2df150a-4ac0-412d-b2bd-f0ad4d779443 None None] Authenticating user token {{(pid=80378) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:17:33.874012 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.fernet_utils [None req-e2df150a-4ac0-412d-b2bd-f0ad4d779443 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80378) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 10:17:33.885431 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.notifications [None req-96cb2fdc-b0bf-4f06-b5f3-a534da8fcbea admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '5877952b334640b6b9d704e3f1409e7b', 'request_id': 'req-96cb2fdc-b0bf-4f06-b5f3-a534da8fcbea'} {{(pid=80377) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 10:17:33.893715 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.notifications [None req-96cb2fdc-b0bf-4f06-b5f3-a534da8fcbea admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '5877952b334640b6b9d704e3f1409e7b', 'request_id': 'req-96cb2fdc-b0bf-4f06-b5f3-a534da8fcbea'} {{(pid=80377) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 10:17:33.902807 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.notifications [None req-96cb2fdc-b0bf-4f06-b5f3-a534da8fcbea admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '5877952b334640b6b9d704e3f1409e7b', 'request_id': 'req-96cb2fdc-b0bf-4f06-b5f3-a534da8fcbea'} {{(pid=80377) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 10:17:33.924002 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.notifications [None req-96cb2fdc-b0bf-4f06-b5f3-a534da8fcbea admin admin] The token cache is being invalidate because project 5877952b334640b6b9d704e3f1409e7b was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80377) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 02 10:17:33.924303 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.notifications [None req-96cb2fdc-b0bf-4f06-b5f3-a534da8fcbea admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80377) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 10:17:33.925650 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.notifications [None req-96cb2fdc-b0bf-4f06-b5f3-a534da8fcbea admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80377) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 10:17:33.929064 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: [pid: 80377|app: 0|req: 1080/2153] 10.222.0.82 () {58 vars in 1233 bytes} [Tue Aug 2 10:17:33 2022] DELETE /identity/v3/projects/5877952b334640b6b9d704e3f1409e7b => generated 0 bytes in 309 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 02 10:17:33.930310 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96cb2fdc-b0bf-4f06-b5f3-a534da8fcbea admin admin] Authenticating user token {{(pid=80377) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:17:33.932158 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.fernet_utils [None req-96cb2fdc-b0bf-4f06-b5f3-a534da8fcbea admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80377) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 10:17:33.957143 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG dbcounter [-] [80377] Writing DB stats keystone:SELECT=89,keystone:INSERT=4,keystone:DELETE=7 {{(pid=80377) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 10:17:33.960912 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2df150a-4ac0-412d-b2bd-f0ad4d779443 None None] Validating token access rules against request {{(pid=80378) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:17:34.014952 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96cb2fdc-b0bf-4f06-b5f3-a534da8fcbea admin admin] Validating token access rules against request {{(pid=80377) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:17:34.032376 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f05b7d6-f1ea-4f4d-9e08-879f8441e657 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '891005fa5ff74041b75747d16cc66798', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7047912c5acd4adeba5d87c3bf895e39', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80377) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:17:34.033193 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f05b7d6-f1ea-4f4d-9e08-879f8441e657 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:17:34.033322 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f05b7d6-f1ea-4f4d-9e08-879f8441e657 admin admin] SCRIPT_NAME: `/identity` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:17:34.033604 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f05b7d6-f1ea-4f4d-9e08-879f8441e657 admin admin] PATH_INFO: `/v3/projects/cb42e40657a64cc29779eb3552b569d7` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:17:34.035351 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-138ff3e6-d22d-4140-bc4f-9ca2fcba44ee service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1023de281ae74e9d8a6d6c312c1c97a7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f1b38ebb804c41c69772c998aae72c41', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80378) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:17:34.035491 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f05b7d6-f1ea-4f4d-9e08-879f8441e657 admin admin] RBAC: Authorizing `identity:delete_project(project_id=cb42e40657a64cc29779eb3552b569d7)` {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:17:34.036276 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-138ff3e6-d22d-4140-bc4f-9ca2fcba44ee service cinder] REQUEST_METHOD: `GET` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:17:34.036883 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-138ff3e6-d22d-4140-bc4f-9ca2fcba44ee service cinder] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:17:34.037389 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-138ff3e6-d22d-4140-bc4f-9ca2fcba44ee service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:17:34.037761 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f05b7d6-f1ea-4f4d-9e08-879f8441e657 admin admin] RBAC: Authorization granted {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:17:34.040037 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.fernet_utils [None req-138ff3e6-d22d-4140-bc4f-9ca2fcba44ee service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80378) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 10:17:34.091064 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.notifications [None req-6f05b7d6-f1ea-4f4d-9e08-879f8441e657 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'cb42e40657a64cc29779eb3552b569d7', 'request_id': 'req-6f05b7d6-f1ea-4f4d-9e08-879f8441e657'} {{(pid=80377) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 10:17:34.098181 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.notifications [None req-6f05b7d6-f1ea-4f4d-9e08-879f8441e657 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'cb42e40657a64cc29779eb3552b569d7', 'request_id': 'req-6f05b7d6-f1ea-4f4d-9e08-879f8441e657'} {{(pid=80377) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 10:17:34.105716 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.notifications [None req-6f05b7d6-f1ea-4f4d-9e08-879f8441e657 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'cb42e40657a64cc29779eb3552b569d7', 'request_id': 'req-6f05b7d6-f1ea-4f4d-9e08-879f8441e657'} {{(pid=80377) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 10:17:34.121984 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-138ff3e6-d22d-4140-bc4f-9ca2fcba44ee service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:17:34.123476 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-138ff3e6-d22d-4140-bc4f-9ca2fcba44ee service cinder] RBAC: Authorization granted {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:17:34.129357 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.notifications [None req-6f05b7d6-f1ea-4f4d-9e08-879f8441e657 admin admin] The token cache is being invalidate because project cb42e40657a64cc29779eb3552b569d7 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80377) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 02 10:17:34.129674 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.notifications [None req-6f05b7d6-f1ea-4f4d-9e08-879f8441e657 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80377) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 10:17:34.130728 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.notifications [None req-6f05b7d6-f1ea-4f4d-9e08-879f8441e657 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80377) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 10:17:34.135952 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: [pid: 80377|app: 0|req: 1081/2154] 10.222.0.82 () {58 vars in 1233 bytes} [Tue Aug 2 10:17:33 2022] DELETE /identity/v3/projects/cb42e40657a64cc29779eb3552b569d7 => generated 0 bytes in 203 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 02 10:17:34.135952 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f05b7d6-f1ea-4f4d-9e08-879f8441e657 admin admin] Authenticating user token {{(pid=80377) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:17:34.136248 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 1074/2155] 10.222.0.82 () {60 vars in 1365 bytes} [Tue Aug 2 10:17:33 2022] GET /identity/v3/auth/tokens => generated 3250 bytes in 264 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 02 10:17:34.137550 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.fernet_utils [None req-6f05b7d6-f1ea-4f4d-9e08-879f8441e657 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80377) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 10:17:34.139764 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-138ff3e6-d22d-4140-bc4f-9ca2fcba44ee service cinder] Authenticating user token {{(pid=80378) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:17:34.158375 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-138ff3e6-d22d-4140-bc4f-9ca2fcba44ee service cinder] Validating token access rules against request {{(pid=80378) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:17:34.163699 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG dbcounter [-] [80378] Writing DB stats keystone:SELECT=100 {{(pid=80378) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 10:17:34.235206 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5efe0c73-d967-48b1-8963-07c7100a8bce admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '891005fa5ff74041b75747d16cc66798', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7047912c5acd4adeba5d87c3bf895e39', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80378) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:17:34.235206 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5efe0c73-d967-48b1-8963-07c7100a8bce admin admin] REQUEST_METHOD: `DELETE` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:17:34.235510 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5efe0c73-d967-48b1-8963-07c7100a8bce admin admin] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:17:34.235756 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5efe0c73-d967-48b1-8963-07c7100a8bce admin admin] PATH_INFO: `/v3/users/ad4d220ace114ad09070199661f3c6fc` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:17:34.237028 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5efe0c73-d967-48b1-8963-07c7100a8bce admin admin] RBAC: Authorizing `identity:delete_user(user_id=ad4d220ace114ad09070199661f3c6fc)` {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:17:34.239453 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5efe0c73-d967-48b1-8963-07c7100a8bce admin admin] RBAC: Authorization granted {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:17:34.249469 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f05b7d6-f1ea-4f4d-9e08-879f8441e657 admin admin] Validating token access rules against request {{(pid=80377) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:17:34.258794 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d54e80cc-43dc-4bad-ab01-b0dd4c1e4fe7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '891005fa5ff74041b75747d16cc66798', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7047912c5acd4adeba5d87c3bf895e39', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80377) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:17:34.259525 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d54e80cc-43dc-4bad-ab01-b0dd4c1e4fe7 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:17:34.259993 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d54e80cc-43dc-4bad-ab01-b0dd4c1e4fe7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:17:34.260276 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d54e80cc-43dc-4bad-ab01-b0dd4c1e4fe7 admin admin] PATH_INFO: `/v3/users/7e6fcf8a332c4e81b40ac1cc3eb747c6` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:17:34.261976 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d54e80cc-43dc-4bad-ab01-b0dd4c1e4fe7 admin admin] RBAC: Authorizing `identity:delete_user(user_id=7e6fcf8a332c4e81b40ac1cc3eb747c6)` {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:17:34.264883 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d54e80cc-43dc-4bad-ab01-b0dd4c1e4fe7 admin admin] RBAC: Authorization granted {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:17:34.311570 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.notifications [None req-5efe0c73-d967-48b1-8963-07c7100a8bce admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'ad4d220ace114ad09070199661f3c6fc'} {{(pid=80378) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 10:17:34.320570 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.notifications [None req-5efe0c73-d967-48b1-8963-07c7100a8bce admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'ad4d220ace114ad09070199661f3c6fc'} {{(pid=80378) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 10:17:34.333949 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.notifications [None req-5efe0c73-d967-48b1-8963-07c7100a8bce admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'ad4d220ace114ad09070199661f3c6fc'} {{(pid=80378) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 10:17:34.335832 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.notifications [None req-5efe0c73-d967-48b1-8963-07c7100a8bce admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'ad4d220ace114ad09070199661f3c6fc'} {{(pid=80378) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 10:17:34.342069 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.notifications [None req-d54e80cc-43dc-4bad-ab01-b0dd4c1e4fe7 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '7e6fcf8a332c4e81b40ac1cc3eb747c6'} {{(pid=80377) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 10:17:34.346707 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG dbcounter [-] [80377] Writing DB stats keystone:SELECT=85,keystone:DELETE=14,keystone:INSERT=1 {{(pid=80377) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 10:17:34.356183 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.notifications [None req-d54e80cc-43dc-4bad-ab01-b0dd4c1e4fe7 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '7e6fcf8a332c4e81b40ac1cc3eb747c6'} {{(pid=80377) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 10:17:34.357013 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.notifications [None req-5efe0c73-d967-48b1-8963-07c7100a8bce admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'ad4d220ace114ad09070199661f3c6fc'} {{(pid=80378) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 10:17:34.373323 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 1075/2156] 10.222.0.82 () {58 vars in 1224 bytes} [Tue Aug 2 10:17:34 2022] DELETE /identity/v3/users/ad4d220ace114ad09070199661f3c6fc => generated 0 bytes in 230 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 02 10:17:34.374876 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5efe0c73-d967-48b1-8963-07c7100a8bce admin admin] Authenticating user token {{(pid=80378) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:17:34.376695 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.fernet_utils [None req-5efe0c73-d967-48b1-8963-07c7100a8bce admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80378) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 10:17:34.379940 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.notifications [None req-d54e80cc-43dc-4bad-ab01-b0dd4c1e4fe7 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '7e6fcf8a332c4e81b40ac1cc3eb747c6'} {{(pid=80377) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 10:17:34.381550 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.notifications [None req-d54e80cc-43dc-4bad-ab01-b0dd4c1e4fe7 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '7e6fcf8a332c4e81b40ac1cc3eb747c6'} {{(pid=80377) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 10:17:34.402047 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.notifications [None req-d54e80cc-43dc-4bad-ab01-b0dd4c1e4fe7 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '7e6fcf8a332c4e81b40ac1cc3eb747c6'} {{(pid=80377) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 10:17:34.411855 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: [pid: 80377|app: 0|req: 1082/2157] 10.222.0.82 () {58 vars in 1224 bytes} [Tue Aug 2 10:17:34 2022] DELETE /identity/v3/users/7e6fcf8a332c4e81b40ac1cc3eb747c6 => generated 0 bytes in 278 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 02 10:17:34.479081 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5efe0c73-d967-48b1-8963-07c7100a8bce admin admin] Validating token access rules against request {{(pid=80378) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:17:34.538794 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG dbcounter [-] [80378] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80378) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 10:17:34.571082 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c261fe48-7564-42df-8c83-344cca22fe89 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '891005fa5ff74041b75747d16cc66798', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7047912c5acd4adeba5d87c3bf895e39', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80378) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:17:34.572102 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c261fe48-7564-42df-8c83-344cca22fe89 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:17:34.572612 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c261fe48-7564-42df-8c83-344cca22fe89 admin admin] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:17:34.573930 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c261fe48-7564-42df-8c83-344cca22fe89 admin admin] PATH_INFO: `/v3/users/9210e6a584144415acee9667164fd8ff` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:17:34.574732 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c261fe48-7564-42df-8c83-344cca22fe89 admin admin] RBAC: Authorizing `identity:delete_user(user_id=9210e6a584144415acee9667164fd8ff)` {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:17:34.576750 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c261fe48-7564-42df-8c83-344cca22fe89 admin admin] RBAC: Authorization granted {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:17:34.684343 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.notifications [None req-c261fe48-7564-42df-8c83-344cca22fe89 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '9210e6a584144415acee9667164fd8ff'} {{(pid=80378) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 10:17:34.696200 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.notifications [None req-c261fe48-7564-42df-8c83-344cca22fe89 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '9210e6a584144415acee9667164fd8ff'} {{(pid=80378) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 10:17:34.706644 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.notifications [None req-c261fe48-7564-42df-8c83-344cca22fe89 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '9210e6a584144415acee9667164fd8ff'} {{(pid=80378) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 10:17:34.707686 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.notifications [None req-c261fe48-7564-42df-8c83-344cca22fe89 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '9210e6a584144415acee9667164fd8ff'} {{(pid=80378) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 10:17:34.727287 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.notifications [None req-c261fe48-7564-42df-8c83-344cca22fe89 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '9210e6a584144415acee9667164fd8ff'} {{(pid=80378) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 10:17:34.737058 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 1076/2158] 10.222.0.82 () {58 vars in 1224 bytes} [Tue Aug 2 10:17:34 2022] DELETE /identity/v3/users/9210e6a584144415acee9667164fd8ff => generated 0 bytes in 363 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 02 10:17:34.741429 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37d62703-a56b-4658-a582-30fafca666b7 None None] REQUEST_METHOD: `POST` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:17:34.741717 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37d62703-a56b-4658-a582-30fafca666b7 None None] SCRIPT_NAME: `/identity` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:17:34.742014 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37d62703-a56b-4658-a582-30fafca666b7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:17:34.766746 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.auth.core [None req-37d62703-a56b-4658-a582-30fafca666b7 None None] MFA Rules not processed for user `891005fa5ff74041b75747d16cc66798`. Rule list: `[]` (Enabled: `True`). {{(pid=80377) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 02 10:17:34.768139 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.fernet_utils [None req-37d62703-a56b-4658-a582-30fafca666b7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80377) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 10:17:34.803083 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c261fe48-7564-42df-8c83-344cca22fe89 admin admin] Authenticating user token {{(pid=80378) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:17:34.804953 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.fernet_utils [None req-c261fe48-7564-42df-8c83-344cca22fe89 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80378) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 10:17:34.861742 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: [pid: 80377|app: 0|req: 1083/2159] 10.222.0.82 () {58 vars in 958 bytes} [Tue Aug 2 10:17:34 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 118 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 02 10:17:34.869011 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37d62703-a56b-4658-a582-30fafca666b7 None None] Authenticating user token {{(pid=80377) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:17:34.870518 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.fernet_utils [None req-37d62703-a56b-4658-a582-30fafca666b7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80377) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 10:17:34.912306 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c261fe48-7564-42df-8c83-344cca22fe89 admin admin] Validating token access rules against request {{(pid=80378) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:17:34.917362 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3cbc738d-6fea-4811-8975-0c5b5b4d94fa admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '891005fa5ff74041b75747d16cc66798', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7047912c5acd4adeba5d87c3bf895e39', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80378) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:17:34.918118 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cbc738d-6fea-4811-8975-0c5b5b4d94fa admin admin] REQUEST_METHOD: `DELETE` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:17:34.918467 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cbc738d-6fea-4811-8975-0c5b5b4d94fa admin admin] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:17:34.918752 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cbc738d-6fea-4811-8975-0c5b5b4d94fa admin admin] PATH_INFO: `/v3/projects/48943a69c8854fcebfd985ee35c6643e` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:17:34.920664 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3cbc738d-6fea-4811-8975-0c5b5b4d94fa admin admin] RBAC: Authorizing `identity:delete_project(project_id=48943a69c8854fcebfd985ee35c6643e)` {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:17:34.922685 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3cbc738d-6fea-4811-8975-0c5b5b4d94fa admin admin] RBAC: Authorization granted {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:17:34.944678 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37d62703-a56b-4658-a582-30fafca666b7 None None] Validating token access rules against request {{(pid=80377) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:17:34.949151 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dac8825e-0114-49c6-95b6-774f5f84a0de service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '33b87170f0634b67bce60750847b5cbe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f1b38ebb804c41c69772c998aae72c41', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80377) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:17:34.950013 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dac8825e-0114-49c6-95b6-774f5f84a0de service neutron] REQUEST_METHOD: `GET` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:17:34.950216 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dac8825e-0114-49c6-95b6-774f5f84a0de service neutron] SCRIPT_NAME: `/identity` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:17:34.951009 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dac8825e-0114-49c6-95b6-774f5f84a0de service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:17:34.964676 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dac8825e-0114-49c6-95b6-774f5f84a0de service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:17:34.967367 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dac8825e-0114-49c6-95b6-774f5f84a0de service neutron] RBAC: Authorization granted {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:17:34.979645 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.notifications [None req-3cbc738d-6fea-4811-8975-0c5b5b4d94fa admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '48943a69c8854fcebfd985ee35c6643e', 'request_id': 'req-3cbc738d-6fea-4811-8975-0c5b5b4d94fa'} {{(pid=80378) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 10:17:34.984193 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: [pid: 80377|app: 0|req: 1084/2160] 10.222.0.82 () {60 vars in 1365 bytes} [Tue Aug 2 10:17:34 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 116 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 02 10:17:34.990831 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.notifications [None req-3cbc738d-6fea-4811-8975-0c5b5b4d94fa admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '48943a69c8854fcebfd985ee35c6643e', 'request_id': 'req-3cbc738d-6fea-4811-8975-0c5b5b4d94fa'} {{(pid=80378) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 10:17:34.999203 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG dbcounter [-] [80378] Writing DB stats keystone:SELECT=84,keystone:DELETE=15,keystone:INSERT=1 {{(pid=80378) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 10:17:35.012500 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.notifications [None req-3cbc738d-6fea-4811-8975-0c5b5b4d94fa admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '48943a69c8854fcebfd985ee35c6643e', 'request_id': 'req-3cbc738d-6fea-4811-8975-0c5b5b4d94fa'} {{(pid=80378) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 10:17:35.031209 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.notifications [None req-3cbc738d-6fea-4811-8975-0c5b5b4d94fa admin admin] The token cache is being invalidate because project 48943a69c8854fcebfd985ee35c6643e was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80378) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 02 10:17:35.031805 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.notifications [None req-3cbc738d-6fea-4811-8975-0c5b5b4d94fa admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80378) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 10:17:35.032774 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.notifications [None req-3cbc738d-6fea-4811-8975-0c5b5b4d94fa admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80378) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 10:17:35.034906 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 1077/2161] 10.222.0.82 () {58 vars in 1233 bytes} [Tue Aug 2 10:17:34 2022] DELETE /identity/v3/projects/48943a69c8854fcebfd985ee35c6643e => generated 0 bytes in 232 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 02 10:17:35.040721 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9f1a103-a302-4e31-8007-78d0fffd8970 None None] REQUEST_METHOD: `POST` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:17:35.041091 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9f1a103-a302-4e31-8007-78d0fffd8970 None None] SCRIPT_NAME: `/identity` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:17:35.041437 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9f1a103-a302-4e31-8007-78d0fffd8970 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:17:35.085222 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.auth.core [None req-a9f1a103-a302-4e31-8007-78d0fffd8970 None None] MFA Rules not processed for user `891005fa5ff74041b75747d16cc66798`. Rule list: `[]` (Enabled: `True`). {{(pid=80377) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 02 10:17:35.086626 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.fernet_utils [None req-a9f1a103-a302-4e31-8007-78d0fffd8970 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80377) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 10:17:35.166903 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: [pid: 80377|app: 0|req: 1085/2162] 10.222.0.82 () {58 vars in 958 bytes} [Tue Aug 2 10:17:35 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 128 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 02 10:17:35.176217 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f41cb99a-93d3-4e8c-8f6d-814961b04191 None None] REQUEST_METHOD: `POST` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:17:35.176608 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f41cb99a-93d3-4e8c-8f6d-814961b04191 None None] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:17:35.176958 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f41cb99a-93d3-4e8c-8f6d-814961b04191 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:17:35.203325 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.auth.core [None req-f41cb99a-93d3-4e8c-8f6d-814961b04191 None None] MFA Rules not processed for user `891005fa5ff74041b75747d16cc66798`. Rule list: `[]` (Enabled: `True`). {{(pid=80378) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 02 10:17:35.204693 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.fernet_utils [None req-f41cb99a-93d3-4e8c-8f6d-814961b04191 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80378) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 10:17:35.220393 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 1078/2163] 10.222.0.82 () {58 vars in 958 bytes} [Tue Aug 2 10:17:35 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 45 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 02 10:17:35.221187 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9f1a103-a302-4e31-8007-78d0fffd8970 None None] Authenticating user token {{(pid=80377) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:17:35.235158 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9f1a103-a302-4e31-8007-78d0fffd8970 None None] Validating token access rules against request {{(pid=80377) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:17:35.241235 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85506c9d-bfc6-48cd-8503-4789bf51ca91 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '891005fa5ff74041b75747d16cc66798', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7047912c5acd4adeba5d87c3bf895e39', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80377) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:17:35.242184 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85506c9d-bfc6-48cd-8503-4789bf51ca91 admin admin] REQUEST_METHOD: `GET` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:17:35.242484 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85506c9d-bfc6-48cd-8503-4789bf51ca91 admin admin] SCRIPT_NAME: `/identity` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:17:35.242946 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85506c9d-bfc6-48cd-8503-4789bf51ca91 admin admin] PATH_INFO: `/v3/domains` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:17:35.243737 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85506c9d-bfc6-48cd-8503-4789bf51ca91 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80377) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 02 10:17:35.244127 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85506c9d-bfc6-48cd-8503-4789bf51ca91 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:17:35.246070 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85506c9d-bfc6-48cd-8503-4789bf51ca91 admin admin] RBAC: Authorization granted {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:17:35.249174 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG dbcounter [-] [80377] Writing DB stats keystone:INSERT=1,keystone:DELETE=3,keystone:SELECT=96 {{(pid=80377) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 10:17:35.265875 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: [pid: 80377|app: 0|req: 1086/2164] 10.222.0.82 () {58 vars in 1153 bytes} [Tue Aug 2 10:17:35 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 45 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 02 10:17:35.270628 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f41cb99a-93d3-4e8c-8f6d-814961b04191 None None] Authenticating user token {{(pid=80378) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:17:35.283810 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f41cb99a-93d3-4e8c-8f6d-814961b04191 None None] Validating token access rules against request {{(pid=80378) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:17:35.292497 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac929c14-3faa-468b-a9cb-34f8f29acd71 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '891005fa5ff74041b75747d16cc66798', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7047912c5acd4adeba5d87c3bf895e39', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80378) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:17:35.294368 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac929c14-3faa-468b-a9cb-34f8f29acd71 admin admin] REQUEST_METHOD: `POST` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:17:35.294807 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac929c14-3faa-468b-a9cb-34f8f29acd71 admin admin] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:17:35.295265 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac929c14-3faa-468b-a9cb-34f8f29acd71 admin admin] PATH_INFO: `/v3/projects` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:17:35.296551 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac929c14-3faa-468b-a9cb-34f8f29acd71 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:17:35.298966 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac929c14-3faa-468b-a9cb-34f8f29acd71 admin admin] RBAC: Authorization granted {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:17:35.385816 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85506c9d-bfc6-48cd-8503-4789bf51ca91 admin admin] Authenticating user token {{(pid=80377) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:17:35.389398 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.fernet_utils [None req-85506c9d-bfc6-48cd-8503-4789bf51ca91 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80377) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 10:17:35.395832 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 1079/2165] 10.222.0.82 () {60 vars in 1153 bytes} [Tue Aug 2 10:17:35 2022] POST /identity/v3/projects => generated 377 bytes in 122 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 02 10:17:35.397040 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac929c14-3faa-468b-a9cb-34f8f29acd71 admin admin] Authenticating user token {{(pid=80378) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:17:35.428696 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85506c9d-bfc6-48cd-8503-4789bf51ca91 admin admin] Validating token access rules against request {{(pid=80377) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:17:35.522577 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac929c14-3faa-468b-a9cb-34f8f29acd71 admin admin] Validating token access rules against request {{(pid=80378) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:17:35.530685 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8e2c13d-e696-4f2c-a89f-477925239baa admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '891005fa5ff74041b75747d16cc66798', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7047912c5acd4adeba5d87c3bf895e39', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80378) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:17:35.531481 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8e2c13d-e696-4f2c-a89f-477925239baa admin admin] REQUEST_METHOD: `POST` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:17:35.531769 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8e2c13d-e696-4f2c-a89f-477925239baa admin admin] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:17:35.532294 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8e2c13d-e696-4f2c-a89f-477925239baa admin admin] PATH_INFO: `/v3/users` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:17:35.533035 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8e2c13d-e696-4f2c-a89f-477925239baa admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:17:35.537830 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8e2c13d-e696-4f2c-a89f-477925239baa admin admin] RBAC: Authorization granted {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:17:35.560025 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a34992f-0209-4334-8146-15d72a9ad7d4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '891005fa5ff74041b75747d16cc66798', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7047912c5acd4adeba5d87c3bf895e39', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80377) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:17:35.561171 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a34992f-0209-4334-8146-15d72a9ad7d4 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:17:35.561957 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a34992f-0209-4334-8146-15d72a9ad7d4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:17:35.562409 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a34992f-0209-4334-8146-15d72a9ad7d4 admin admin] PATH_INFO: `/v3/projects/5704398ca48145f0920db111e88a7455` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:17:35.563838 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a34992f-0209-4334-8146-15d72a9ad7d4 admin admin] RBAC: Authorizing `identity:delete_project(project_id=5704398ca48145f0920db111e88a7455)` {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:17:35.565254 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 1080/2166] 10.222.0.82 () {60 vars in 1144 bytes} [Tue Aug 2 10:17:35 2022] POST /identity/v3/users => generated 289 bytes in 169 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 02 10:17:35.566797 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a34992f-0209-4334-8146-15d72a9ad7d4 admin admin] RBAC: Authorization granted {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:17:35.569413 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8e2c13d-e696-4f2c-a89f-477925239baa admin admin] Authenticating user token {{(pid=80378) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:17:35.616340 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8e2c13d-e696-4f2c-a89f-477925239baa admin admin] Validating token access rules against request {{(pid=80378) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:17:35.636713 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-547362c6-b1fc-4ba7-a320-4fbab39ac075 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '891005fa5ff74041b75747d16cc66798', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7047912c5acd4adeba5d87c3bf895e39', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80378) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:17:35.637497 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-547362c6-b1fc-4ba7-a320-4fbab39ac075 admin admin] REQUEST_METHOD: `GET` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:17:35.637900 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-547362c6-b1fc-4ba7-a320-4fbab39ac075 admin admin] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:17:35.638763 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-547362c6-b1fc-4ba7-a320-4fbab39ac075 admin admin] PATH_INFO: `/v3/roles` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:17:35.639539 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-547362c6-b1fc-4ba7-a320-4fbab39ac075 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:17:35.641514 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-547362c6-b1fc-4ba7-a320-4fbab39ac075 admin admin] RBAC: Authorization granted {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:17:35.644446 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.notifications [None req-2a34992f-0209-4334-8146-15d72a9ad7d4 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '5704398ca48145f0920db111e88a7455', 'request_id': 'req-2a34992f-0209-4334-8146-15d72a9ad7d4'} {{(pid=80377) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 10:17:35.657237 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 1081/2167] 10.222.0.82 () {58 vars in 1122 bytes} [Tue Aug 2 10:17:35 2022] GET /identity/v3/roles => generated 1420 bytes in 88 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 02 10:17:35.667487 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.notifications [None req-2a34992f-0209-4334-8146-15d72a9ad7d4 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '5704398ca48145f0920db111e88a7455', 'request_id': 'req-2a34992f-0209-4334-8146-15d72a9ad7d4'} {{(pid=80377) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 10:17:35.673343 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-547362c6-b1fc-4ba7-a320-4fbab39ac075 admin admin] Authenticating user token {{(pid=80378) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:17:35.682294 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.notifications [None req-2a34992f-0209-4334-8146-15d72a9ad7d4 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '5704398ca48145f0920db111e88a7455', 'request_id': 'req-2a34992f-0209-4334-8146-15d72a9ad7d4'} {{(pid=80377) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 10:17:35.691076 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-547362c6-b1fc-4ba7-a320-4fbab39ac075 admin admin] Validating token access rules against request {{(pid=80378) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:17:35.707138 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.notifications [None req-2a34992f-0209-4334-8146-15d72a9ad7d4 admin admin] The token cache is being invalidate because project 5704398ca48145f0920db111e88a7455 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80377) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 02 10:17:35.707947 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.notifications [None req-2a34992f-0209-4334-8146-15d72a9ad7d4 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80377) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 10:17:35.709211 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.notifications [None req-2a34992f-0209-4334-8146-15d72a9ad7d4 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80377) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 10:17:35.712077 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: [pid: 80377|app: 0|req: 1087/2168] 10.222.0.82 () {58 vars in 1233 bytes} [Tue Aug 2 10:17:35 2022] DELETE /identity/v3/projects/5704398ca48145f0920db111e88a7455 => generated 0 bytes in 327 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 02 10:17:35.717155 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41b91260-f5a3-4772-8108-93f753c23df5 None None] REQUEST_METHOD: `POST` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:17:35.717414 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41b91260-f5a3-4772-8108-93f753c23df5 None None] SCRIPT_NAME: `/identity` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:17:35.717698 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41b91260-f5a3-4772-8108-93f753c23df5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:17:35.742573 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.auth.core [None req-41b91260-f5a3-4772-8108-93f753c23df5 None None] MFA Rules not processed for user `891005fa5ff74041b75747d16cc66798`. Rule list: `[]` (Enabled: `True`). {{(pid=80377) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 02 10:17:35.743808 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.fernet_utils [None req-41b91260-f5a3-4772-8108-93f753c23df5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80377) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 10:17:35.799804 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8667243-3b8a-4c4e-93b9-95c34548442e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '891005fa5ff74041b75747d16cc66798', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7047912c5acd4adeba5d87c3bf895e39', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80378) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:17:35.800447 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8667243-3b8a-4c4e-93b9-95c34548442e admin admin] REQUEST_METHOD: `GET` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:17:35.800730 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8667243-3b8a-4c4e-93b9-95c34548442e admin admin] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:17:35.801005 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8667243-3b8a-4c4e-93b9-95c34548442e admin admin] PATH_INFO: `/v3/roles` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:17:35.801822 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8667243-3b8a-4c4e-93b9-95c34548442e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:17:35.803513 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8667243-3b8a-4c4e-93b9-95c34548442e admin admin] RBAC: Authorization granted {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:17:35.830068 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 1082/2169] 10.222.0.82 () {58 vars in 1122 bytes} [Tue Aug 2 10:17:35 2022] GET /identity/v3/roles => generated 1420 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 02 10:17:35.830957 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8667243-3b8a-4c4e-93b9-95c34548442e admin admin] Authenticating user token {{(pid=80378) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:17:35.832713 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.fernet_utils [None req-e8667243-3b8a-4c4e-93b9-95c34548442e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80378) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 10:17:35.857344 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8667243-3b8a-4c4e-93b9-95c34548442e admin admin] Validating token access rules against request {{(pid=80378) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:17:35.863170 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87715a3a-c642-45a5-b4cf-239a9f6631f2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '891005fa5ff74041b75747d16cc66798', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7047912c5acd4adeba5d87c3bf895e39', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80378) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:17:35.864004 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87715a3a-c642-45a5-b4cf-239a9f6631f2 admin admin] REQUEST_METHOD: `PUT` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:17:35.864465 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87715a3a-c642-45a5-b4cf-239a9f6631f2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:17:35.864774 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87715a3a-c642-45a5-b4cf-239a9f6631f2 admin admin] PATH_INFO: `/v3/projects/92ec411d93024682ae13872739095d2a/users/56af49202378415f88d3310ebfe4faab/roles/e1c04b19fc4543b8b54f58bbcffbc598` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:17:35.870342 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: [pid: 80377|app: 0|req: 1088/2170] 10.222.0.82 () {58 vars in 958 bytes} [Tue Aug 2 10:17:35 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 154 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 02 10:17:35.874169 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7557618f-f5e5-4d8a-b801-499c64e88ad9 None None] REQUEST_METHOD: `POST` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:17:35.875183 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7557618f-f5e5-4d8a-b801-499c64e88ad9 None None] SCRIPT_NAME: `/identity` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:17:35.875474 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7557618f-f5e5-4d8a-b801-499c64e88ad9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:17:35.892228 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87715a3a-c642-45a5-b4cf-239a9f6631f2 admin admin] RBAC: Authorizing `identity:create_grant(project_id=92ec411d93024682ae13872739095d2a, user_id=56af49202378415f88d3310ebfe4faab, role_id=e1c04b19fc4543b8b54f58bbcffbc598)` {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:17:35.895012 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87715a3a-c642-45a5-b4cf-239a9f6631f2 admin admin] RBAC: Authorization granted {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:17:35.905641 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.auth.core [None req-7557618f-f5e5-4d8a-b801-499c64e88ad9 None None] MFA Rules not processed for user `891005fa5ff74041b75747d16cc66798`. Rule list: `[]` (Enabled: `True`). {{(pid=80377) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 02 10:17:35.906908 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.fernet_utils [None req-7557618f-f5e5-4d8a-b801-499c64e88ad9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80377) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 10:17:35.922034 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: [pid: 80377|app: 0|req: 1089/2171] 10.222.0.82 () {58 vars in 958 bytes} [Tue Aug 2 10:17:35 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 51 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 02 10:17:35.923179 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7557618f-f5e5-4d8a-b801-499c64e88ad9 None None] Authenticating user token {{(pid=80377) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:17:35.924823 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 1083/2172] 10.222.0.82 () {60 vars in 1483 bytes} [Tue Aug 2 10:17:35 2022] PUT /identity/v3/projects/92ec411d93024682ae13872739095d2a/users/56af49202378415f88d3310ebfe4faab/roles/e1c04b19fc4543b8b54f58bbcffbc598 => generated 0 bytes in 94 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 02 10:17:35.933375 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87715a3a-c642-45a5-b4cf-239a9f6631f2 admin admin] Authenticating user token {{(pid=80378) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:17:35.933665 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7557618f-f5e5-4d8a-b801-499c64e88ad9 None None] Validating token access rules against request {{(pid=80377) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:17:35.944329 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.fernet_utils [None req-87715a3a-c642-45a5-b4cf-239a9f6631f2 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80378) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 10:17:35.963581 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG dbcounter [-] [80378] Writing DB stats keystone:SELECT=92,keystone:DELETE=2,keystone:INSERT=6 {{(pid=80378) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 10:17:36.017049 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG dbcounter [-] [80377] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80377) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 10:17:36.074703 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87715a3a-c642-45a5-b4cf-239a9f6631f2 admin admin] Validating token access rules against request {{(pid=80378) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:17:36.082557 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-899a6044-558a-46a1-86b6-55d929500a70 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '891005fa5ff74041b75747d16cc66798', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7047912c5acd4adeba5d87c3bf895e39', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80378) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:17:36.084583 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-899a6044-558a-46a1-86b6-55d929500a70 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:17:36.085097 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-899a6044-558a-46a1-86b6-55d929500a70 admin admin] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:17:36.085596 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-899a6044-558a-46a1-86b6-55d929500a70 admin admin] PATH_INFO: `/v3/projects/13848405aa654dfa96cc1d7e8b18d8ac` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:17:36.088286 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-899a6044-558a-46a1-86b6-55d929500a70 admin admin] RBAC: Authorizing `identity:delete_project(project_id=13848405aa654dfa96cc1d7e8b18d8ac)` {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:17:36.089099 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fb48320-7bb2-4953-a79a-126c1ced9900 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '891005fa5ff74041b75747d16cc66798', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7047912c5acd4adeba5d87c3bf895e39', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80377) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:17:36.089266 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fb48320-7bb2-4953-a79a-126c1ced9900 admin admin] REQUEST_METHOD: `GET` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:17:36.089516 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fb48320-7bb2-4953-a79a-126c1ced9900 admin admin] SCRIPT_NAME: `/identity` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:17:36.089816 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fb48320-7bb2-4953-a79a-126c1ced9900 admin admin] PATH_INFO: `/v3/domains` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:17:36.090626 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5fb48320-7bb2-4953-a79a-126c1ced9900 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80377) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 02 10:17:36.091026 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5fb48320-7bb2-4953-a79a-126c1ced9900 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:17:36.092677 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5fb48320-7bb2-4953-a79a-126c1ced9900 admin admin] RBAC: Authorization granted {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:17:36.094101 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-899a6044-558a-46a1-86b6-55d929500a70 admin admin] RBAC: Authorization granted {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:17:36.109069 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: [pid: 80377|app: 0|req: 1090/2173] 10.222.0.82 () {58 vars in 1153 bytes} [Tue Aug 2 10:17:35 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 186 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 02 10:17:36.110661 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66d49207-288a-4001-a787-2b92f05b195c None None] REQUEST_METHOD: `POST` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:17:36.111022 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66d49207-288a-4001-a787-2b92f05b195c None None] SCRIPT_NAME: `/identity` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:17:36.111312 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66d49207-288a-4001-a787-2b92f05b195c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:17:36.149126 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.auth.core [None req-66d49207-288a-4001-a787-2b92f05b195c None None] MFA Rules not processed for user `56af49202378415f88d3310ebfe4faab`. Rule list: `[]` (Enabled: `True`). {{(pid=80377) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 02 10:17:36.150531 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.fernet_utils [None req-66d49207-288a-4001-a787-2b92f05b195c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80377) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 10:17:36.173999 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.notifications [None req-899a6044-558a-46a1-86b6-55d929500a70 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '13848405aa654dfa96cc1d7e8b18d8ac', 'request_id': 'req-899a6044-558a-46a1-86b6-55d929500a70'} {{(pid=80378) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 10:17:36.185757 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.notifications [None req-899a6044-558a-46a1-86b6-55d929500a70 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '13848405aa654dfa96cc1d7e8b18d8ac', 'request_id': 'req-899a6044-558a-46a1-86b6-55d929500a70'} {{(pid=80378) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 10:17:36.196146 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.notifications [None req-899a6044-558a-46a1-86b6-55d929500a70 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '13848405aa654dfa96cc1d7e8b18d8ac', 'request_id': 'req-899a6044-558a-46a1-86b6-55d929500a70'} {{(pid=80378) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 10:17:36.221220 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.notifications [None req-899a6044-558a-46a1-86b6-55d929500a70 admin admin] The token cache is being invalidate because project 13848405aa654dfa96cc1d7e8b18d8ac was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80378) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 02 10:17:36.221722 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.notifications [None req-899a6044-558a-46a1-86b6-55d929500a70 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80378) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 10:17:36.222504 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.notifications [None req-899a6044-558a-46a1-86b6-55d929500a70 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80378) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 10:17:36.224254 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 1084/2174] 10.222.0.82 () {58 vars in 1233 bytes} [Tue Aug 2 10:17:35 2022] DELETE /identity/v3/projects/13848405aa654dfa96cc1d7e8b18d8ac => generated 0 bytes in 292 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 02 10:17:36.225143 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-899a6044-558a-46a1-86b6-55d929500a70 admin admin] Authenticating user token {{(pid=80378) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:17:36.226899 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.fernet_utils [None req-899a6044-558a-46a1-86b6-55d929500a70 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80378) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 10:17:36.273542 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: [pid: 80377|app: 0|req: 1091/2175] 10.222.0.82 () {58 vars in 958 bytes} [Tue Aug 2 10:17:36 2022] POST /identity/v3/auth/tokens => generated 3254 bytes in 164 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 02 10:17:36.275152 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9dc874c9-b217-4f73-bff4-dda159648a35 None None] REQUEST_METHOD: `POST` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:17:36.275445 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9dc874c9-b217-4f73-bff4-dda159648a35 None None] SCRIPT_NAME: `/identity` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:17:36.275725 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9dc874c9-b217-4f73-bff4-dda159648a35 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:17:36.301069 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.auth.core [None req-9dc874c9-b217-4f73-bff4-dda159648a35 None None] MFA Rules not processed for user `891005fa5ff74041b75747d16cc66798`. Rule list: `[]` (Enabled: `True`). {{(pid=80377) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 02 10:17:36.302347 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.fernet_utils [None req-9dc874c9-b217-4f73-bff4-dda159648a35 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80377) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 10:17:36.336834 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-899a6044-558a-46a1-86b6-55d929500a70 admin admin] Validating token access rules against request {{(pid=80378) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:17:36.348152 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74b68b9b-1dcb-4744-91eb-e8dc755d9fdc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '891005fa5ff74041b75747d16cc66798', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7047912c5acd4adeba5d87c3bf895e39', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80378) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:17:36.348815 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74b68b9b-1dcb-4744-91eb-e8dc755d9fdc admin admin] REQUEST_METHOD: `POST` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:17:36.349104 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74b68b9b-1dcb-4744-91eb-e8dc755d9fdc admin admin] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:17:36.349375 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74b68b9b-1dcb-4744-91eb-e8dc755d9fdc admin admin] PATH_INFO: `/v3/projects` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:17:36.350106 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74b68b9b-1dcb-4744-91eb-e8dc755d9fdc admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:17:36.352113 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74b68b9b-1dcb-4744-91eb-e8dc755d9fdc admin admin] RBAC: Authorization granted {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:17:36.384203 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 1085/2176] 10.222.0.82 () {60 vars in 1153 bytes} [Tue Aug 2 10:17:36 2022] POST /identity/v3/projects => generated 375 bytes in 159 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 02 10:17:36.388973 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74b68b9b-1dcb-4744-91eb-e8dc755d9fdc admin admin] Authenticating user token {{(pid=80378) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:17:36.391580 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.fernet_utils [None req-74b68b9b-1dcb-4744-91eb-e8dc755d9fdc admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80378) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 10:17:36.395337 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: [pid: 80377|app: 0|req: 1092/2177] 10.222.0.82 () {58 vars in 958 bytes} [Tue Aug 2 10:17:36 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 121 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 02 10:17:36.398346 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9dc874c9-b217-4f73-bff4-dda159648a35 None None] Authenticating user token {{(pid=80377) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:17:36.409876 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9dc874c9-b217-4f73-bff4-dda159648a35 None None] Validating token access rules against request {{(pid=80377) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:17:36.430179 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG dbcounter [-] [80378] Writing DB stats keystone:SELECT=91,keystone:DELETE=7,keystone:INSERT=2 {{(pid=80378) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 10:17:36.474360 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG dbcounter [-] [80377] Writing DB stats keystone:SELECT=100 {{(pid=80377) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 10:17:36.496338 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93bdfacc-d193-4ed7-93a0-693fd63e2274 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '891005fa5ff74041b75747d16cc66798', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7047912c5acd4adeba5d87c3bf895e39', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80377) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:17:36.496599 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74b68b9b-1dcb-4744-91eb-e8dc755d9fdc admin admin] Validating token access rules against request {{(pid=80378) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:17:36.497167 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93bdfacc-d193-4ed7-93a0-693fd63e2274 admin admin] REQUEST_METHOD: `POST` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:17:36.497670 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93bdfacc-d193-4ed7-93a0-693fd63e2274 admin admin] SCRIPT_NAME: `/identity` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:17:36.498241 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93bdfacc-d193-4ed7-93a0-693fd63e2274 admin admin] PATH_INFO: `/v3/users` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:17:36.499031 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93bdfacc-d193-4ed7-93a0-693fd63e2274 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:17:36.500840 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-318e096c-4a10-48e0-9bec-bc6297b5b509 service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa964af53a2b4fb9ace6afb544d54d40', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f1b38ebb804c41c69772c998aae72c41', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80378) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:17:36.501410 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93bdfacc-d193-4ed7-93a0-693fd63e2274 admin admin] RBAC: Authorization granted {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:17:36.501522 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-318e096c-4a10-48e0-9bec-bc6297b5b509 service glance] REQUEST_METHOD: `GET` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:17:36.501760 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-318e096c-4a10-48e0-9bec-bc6297b5b509 service glance] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:17:36.502088 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-318e096c-4a10-48e0-9bec-bc6297b5b509 service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:17:36.535166 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: [pid: 80377|app: 0|req: 1093/2178] 10.222.0.82 () {60 vars in 1144 bytes} [Tue Aug 2 10:17:36 2022] POST /identity/v3/users => generated 288 bytes in 136 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 02 10:17:36.535166 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d631811-0067-4f2f-80ed-8d4677560fe8 None None] REQUEST_METHOD: `POST` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:17:36.535166 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d631811-0067-4f2f-80ed-8d4677560fe8 None None] SCRIPT_NAME: `/identity` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:17:36.535166 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d631811-0067-4f2f-80ed-8d4677560fe8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:17:36.564293 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.auth.core [None req-7d631811-0067-4f2f-80ed-8d4677560fe8 None None] MFA Rules not processed for user `891005fa5ff74041b75747d16cc66798`. Rule list: `[]` (Enabled: `True`). {{(pid=80377) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 02 10:17:36.565383 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.fernet_utils [None req-7d631811-0067-4f2f-80ed-8d4677560fe8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80377) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 10:17:36.566951 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-318e096c-4a10-48e0-9bec-bc6297b5b509 service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:17:36.568485 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-318e096c-4a10-48e0-9bec-bc6297b5b509 service glance] RBAC: Authorization granted {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:17:36.579269 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: [pid: 80377|app: 0|req: 1094/2179] 10.222.0.82 () {58 vars in 958 bytes} [Tue Aug 2 10:17:36 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 47 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 02 10:17:36.579541 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 1086/2180] 10.222.0.82 () {60 vars in 1365 bytes} [Tue Aug 2 10:17:36 2022] GET /identity/v3/auth/tokens => generated 3254 bytes in 193 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 02 10:17:36.580282 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d631811-0067-4f2f-80ed-8d4677560fe8 None None] Authenticating user token {{(pid=80377) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:17:36.584792 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-318e096c-4a10-48e0-9bec-bc6297b5b509 service glance] Authenticating user token {{(pid=80378) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:17:36.590481 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d631811-0067-4f2f-80ed-8d4677560fe8 None None] Validating token access rules against request {{(pid=80377) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:17:36.596208 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3198bb40-c287-44c9-bc58-b84cb5610417 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '891005fa5ff74041b75747d16cc66798', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7047912c5acd4adeba5d87c3bf895e39', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80377) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:17:36.598051 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3198bb40-c287-44c9-bc58-b84cb5610417 admin admin] REQUEST_METHOD: `GET` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:17:36.598660 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3198bb40-c287-44c9-bc58-b84cb5610417 admin admin] SCRIPT_NAME: `/identity` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:17:36.599038 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3198bb40-c287-44c9-bc58-b84cb5610417 admin admin] PATH_INFO: `/v3/roles` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:17:36.599543 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-318e096c-4a10-48e0-9bec-bc6297b5b509 service glance] Validating token access rules against request {{(pid=80378) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:17:36.599964 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3198bb40-c287-44c9-bc58-b84cb5610417 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:17:36.601980 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3198bb40-c287-44c9-bc58-b84cb5610417 admin admin] RBAC: Authorization granted {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:17:36.605756 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e424b257-6e63-4e5a-a7e3-1e9ca34cd3bd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '891005fa5ff74041b75747d16cc66798', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7047912c5acd4adeba5d87c3bf895e39', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80378) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:17:36.606418 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e424b257-6e63-4e5a-a7e3-1e9ca34cd3bd admin admin] REQUEST_METHOD: `GET` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:17:36.606701 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e424b257-6e63-4e5a-a7e3-1e9ca34cd3bd admin admin] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:17:36.607522 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e424b257-6e63-4e5a-a7e3-1e9ca34cd3bd admin admin] PATH_INFO: `/v3/domains` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:17:36.608288 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e424b257-6e63-4e5a-a7e3-1e9ca34cd3bd admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80378) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 02 10:17:36.608935 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e424b257-6e63-4e5a-a7e3-1e9ca34cd3bd admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:17:36.610621 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e424b257-6e63-4e5a-a7e3-1e9ca34cd3bd admin admin] RBAC: Authorization granted {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:17:36.624723 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: [pid: 80377|app: 0|req: 1095/2181] 10.222.0.82 () {58 vars in 1122 bytes} [Tue Aug 2 10:17:36 2022] GET /identity/v3/roles => generated 1420 bytes in 45 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 02 10:17:36.628421 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 1087/2182] 10.222.0.82 () {58 vars in 1153 bytes} [Tue Aug 2 10:17:36 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 41 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 02 10:17:36.629425 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e424b257-6e63-4e5a-a7e3-1e9ca34cd3bd admin admin] Authenticating user token {{(pid=80378) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:17:36.629686 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3198bb40-c287-44c9-bc58-b84cb5610417 admin admin] Authenticating user token {{(pid=80377) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:17:36.655336 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3198bb40-c287-44c9-bc58-b84cb5610417 admin admin] Validating token access rules against request {{(pid=80377) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:17:36.663119 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e424b257-6e63-4e5a-a7e3-1e9ca34cd3bd admin admin] Validating token access rules against request {{(pid=80378) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:17:36.668542 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed9ff4b8-cc77-4b42-a8b5-e9fe0419bb02 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '891005fa5ff74041b75747d16cc66798', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7047912c5acd4adeba5d87c3bf895e39', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80377) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:17:36.670235 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed9ff4b8-cc77-4b42-a8b5-e9fe0419bb02 admin admin] REQUEST_METHOD: `GET` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:17:36.670624 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed9ff4b8-cc77-4b42-a8b5-e9fe0419bb02 admin admin] SCRIPT_NAME: `/identity` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:17:36.671001 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed9ff4b8-cc77-4b42-a8b5-e9fe0419bb02 admin admin] PATH_INFO: `/v3/roles` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:17:36.671963 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed9ff4b8-cc77-4b42-a8b5-e9fe0419bb02 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:17:36.673829 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed9ff4b8-cc77-4b42-a8b5-e9fe0419bb02 admin admin] RBAC: Authorization granted {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:17:36.686903 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ddaf122-8f30-48d9-9389-9abf7b8f99ec admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '891005fa5ff74041b75747d16cc66798', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7047912c5acd4adeba5d87c3bf895e39', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80378) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:17:36.687376 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: [pid: 80377|app: 0|req: 1096/2183] 10.222.0.82 () {58 vars in 1122 bytes} [Tue Aug 2 10:17:36 2022] GET /identity/v3/roles => generated 1420 bytes in 59 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 02 10:17:36.687545 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ddaf122-8f30-48d9-9389-9abf7b8f99ec admin admin] REQUEST_METHOD: `POST` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:17:36.687844 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ddaf122-8f30-48d9-9389-9abf7b8f99ec admin admin] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:17:36.688120 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ddaf122-8f30-48d9-9389-9abf7b8f99ec admin admin] PATH_INFO: `/v3/projects` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:17:36.688799 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ddaf122-8f30-48d9-9389-9abf7b8f99ec admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:17:36.690751 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed9ff4b8-cc77-4b42-a8b5-e9fe0419bb02 admin admin] Authenticating user token {{(pid=80377) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:17:36.692950 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ddaf122-8f30-48d9-9389-9abf7b8f99ec admin admin] RBAC: Authorization granted {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:17:36.703914 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed9ff4b8-cc77-4b42-a8b5-e9fe0419bb02 admin admin] Validating token access rules against request {{(pid=80377) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:17:36.721798 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a90c8139-4976-4d05-9ddf-b8f159ec792e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '891005fa5ff74041b75747d16cc66798', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7047912c5acd4adeba5d87c3bf895e39', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80377) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:17:36.723046 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a90c8139-4976-4d05-9ddf-b8f159ec792e admin admin] REQUEST_METHOD: `PUT` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:17:36.723046 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a90c8139-4976-4d05-9ddf-b8f159ec792e admin admin] SCRIPT_NAME: `/identity` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:17:36.723432 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a90c8139-4976-4d05-9ddf-b8f159ec792e admin admin] PATH_INFO: `/v3/projects/b4a22005113e4ac9bcca471ab0f1d0a6/users/087d4ddfd8a3479f81e6d06af67ca7a3/roles/e1c04b19fc4543b8b54f58bbcffbc598` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:17:36.749839 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 1088/2184] 10.222.0.82 () {60 vars in 1153 bytes} [Tue Aug 2 10:17:36 2022] POST /identity/v3/projects => generated 373 bytes in 119 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 02 10:17:36.752098 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ddaf122-8f30-48d9-9389-9abf7b8f99ec admin admin] Authenticating user token {{(pid=80378) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:17:36.755587 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a90c8139-4976-4d05-9ddf-b8f159ec792e admin admin] RBAC: Authorizing `identity:create_grant(project_id=b4a22005113e4ac9bcca471ab0f1d0a6, user_id=087d4ddfd8a3479f81e6d06af67ca7a3, role_id=e1c04b19fc4543b8b54f58bbcffbc598)` {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:17:36.780174 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a90c8139-4976-4d05-9ddf-b8f159ec792e admin admin] RBAC: Authorization granted {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:17:36.838043 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: [pid: 80377|app: 0|req: 1097/2185] 10.222.0.82 () {60 vars in 1483 bytes} [Tue Aug 2 10:17:36 2022] PUT /identity/v3/projects/b4a22005113e4ac9bcca471ab0f1d0a6/users/087d4ddfd8a3479f81e6d06af67ca7a3/roles/e1c04b19fc4543b8b54f58bbcffbc598 => generated 0 bytes in 138 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 02 10:17:36.840627 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90495bdd-9f1c-46cb-8dcc-7a102820f440 None None] REQUEST_METHOD: `POST` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:17:36.840997 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90495bdd-9f1c-46cb-8dcc-7a102820f440 None None] SCRIPT_NAME: `/identity` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:17:36.841978 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90495bdd-9f1c-46cb-8dcc-7a102820f440 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:17:36.897367 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.auth.core [None req-90495bdd-9f1c-46cb-8dcc-7a102820f440 None None] MFA Rules not processed for user `087d4ddfd8a3479f81e6d06af67ca7a3`. Rule list: `[]` (Enabled: `True`). {{(pid=80377) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 02 10:17:36.898884 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.fernet_utils [None req-90495bdd-9f1c-46cb-8dcc-7a102820f440 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80377) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 10:17:36.903403 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ddaf122-8f30-48d9-9389-9abf7b8f99ec admin admin] Validating token access rules against request {{(pid=80378) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:17:36.947991 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG dbcounter [-] [80378] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80378) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 10:17:37.012865 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dff52402-b640-4551-ba60-9b4696922033 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '891005fa5ff74041b75747d16cc66798', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7047912c5acd4adeba5d87c3bf895e39', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80378) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:17:37.014085 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dff52402-b640-4551-ba60-9b4696922033 admin admin] REQUEST_METHOD: `POST` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:17:37.014374 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dff52402-b640-4551-ba60-9b4696922033 admin admin] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:17:37.014785 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dff52402-b640-4551-ba60-9b4696922033 admin admin] PATH_INFO: `/v3/users` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:17:37.014894 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: [pid: 80377|app: 0|req: 1098/2186] 10.222.0.82 () {58 vars in 958 bytes} [Tue Aug 2 10:17:36 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 176 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 02 10:17:37.015484 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dff52402-b640-4551-ba60-9b4696922033 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:17:37.017667 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dff52402-b640-4551-ba60-9b4696922033 admin admin] RBAC: Authorization granted {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:17:37.019547 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90495bdd-9f1c-46cb-8dcc-7a102820f440 None None] Authenticating user token {{(pid=80377) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:17:37.044172 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90495bdd-9f1c-46cb-8dcc-7a102820f440 None None] Validating token access rules against request {{(pid=80377) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:17:37.051010 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-506c365f-0761-4978-af92-ccc712778a65 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '891005fa5ff74041b75747d16cc66798', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7047912c5acd4adeba5d87c3bf895e39', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80377) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:17:37.052044 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-506c365f-0761-4978-af92-ccc712778a65 admin admin] REQUEST_METHOD: `POST` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:17:37.052349 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-506c365f-0761-4978-af92-ccc712778a65 admin admin] SCRIPT_NAME: `/identity` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:17:37.052664 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-506c365f-0761-4978-af92-ccc712778a65 admin admin] PATH_INFO: `/v3/projects` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:17:37.053436 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-506c365f-0761-4978-af92-ccc712778a65 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:17:37.054627 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 1089/2187] 10.222.0.82 () {60 vars in 1144 bytes} [Tue Aug 2 10:17:36 2022] POST /identity/v3/users => generated 287 bytes in 303 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 02 10:17:37.059348 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dff52402-b640-4551-ba60-9b4696922033 admin admin] Authenticating user token {{(pid=80378) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:17:37.060984 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-506c365f-0761-4978-af92-ccc712778a65 admin admin] RBAC: Authorization granted {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:17:37.083114 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dff52402-b640-4551-ba60-9b4696922033 admin admin] Validating token access rules against request {{(pid=80378) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:17:37.107074 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: [pid: 80377|app: 0|req: 1099/2188] 10.222.0.82 () {60 vars in 1153 bytes} [Tue Aug 2 10:17:37 2022] POST /identity/v3/projects => generated 375 bytes in 88 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 02 10:17:37.112214 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-506c365f-0761-4978-af92-ccc712778a65 admin admin] Authenticating user token {{(pid=80377) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:17:37.124333 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c9ff263-a6c4-4d87-8479-4dba392e68c1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '891005fa5ff74041b75747d16cc66798', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7047912c5acd4adeba5d87c3bf895e39', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80378) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:17:37.125134 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c9ff263-a6c4-4d87-8479-4dba392e68c1 admin admin] REQUEST_METHOD: `GET` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:17:37.125574 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c9ff263-a6c4-4d87-8479-4dba392e68c1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:17:37.125891 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c9ff263-a6c4-4d87-8479-4dba392e68c1 admin admin] PATH_INFO: `/v3/roles` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:17:37.126610 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c9ff263-a6c4-4d87-8479-4dba392e68c1 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:17:37.128303 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c9ff263-a6c4-4d87-8479-4dba392e68c1 admin admin] RBAC: Authorization granted {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:17:37.173825 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 1090/2189] 10.222.0.82 () {58 vars in 1122 bytes} [Tue Aug 2 10:17:37 2022] GET /identity/v3/roles => generated 1420 bytes in 115 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 02 10:17:37.179840 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c9ff263-a6c4-4d87-8479-4dba392e68c1 admin admin] Authenticating user token {{(pid=80378) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:17:37.200098 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c9ff263-a6c4-4d87-8479-4dba392e68c1 admin admin] Validating token access rules against request {{(pid=80378) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:17:37.225330 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG dbcounter [-] [80377] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80377) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 10:17:37.260183 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-506c365f-0761-4978-af92-ccc712778a65 admin admin] Validating token access rules against request {{(pid=80377) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:17:37.273087 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ffcdb3c-f946-4d62-8d3f-976939b5e577 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '891005fa5ff74041b75747d16cc66798', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7047912c5acd4adeba5d87c3bf895e39', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80377) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:17:37.275867 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ffcdb3c-f946-4d62-8d3f-976939b5e577 admin admin] REQUEST_METHOD: `POST` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:17:37.276838 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ffcdb3c-f946-4d62-8d3f-976939b5e577 admin admin] SCRIPT_NAME: `/identity` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:17:37.277406 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ffcdb3c-f946-4d62-8d3f-976939b5e577 admin admin] PATH_INFO: `/v3/users` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:17:37.280301 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ffcdb3c-f946-4d62-8d3f-976939b5e577 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:17:37.284541 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ffcdb3c-f946-4d62-8d3f-976939b5e577 admin admin] RBAC: Authorization granted {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:17:37.303311 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9630b35d-5b54-4fc5-88c1-a7ef1aa203d2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '891005fa5ff74041b75747d16cc66798', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7047912c5acd4adeba5d87c3bf895e39', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80378) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:17:37.304293 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9630b35d-5b54-4fc5-88c1-a7ef1aa203d2 admin admin] REQUEST_METHOD: `GET` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:17:37.304578 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9630b35d-5b54-4fc5-88c1-a7ef1aa203d2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:17:37.304853 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9630b35d-5b54-4fc5-88c1-a7ef1aa203d2 admin admin] PATH_INFO: `/v3/roles` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:17:37.305550 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9630b35d-5b54-4fc5-88c1-a7ef1aa203d2 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:17:37.307184 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9630b35d-5b54-4fc5-88c1-a7ef1aa203d2 admin admin] RBAC: Authorization granted {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:17:37.334677 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: [pid: 80377|app: 0|req: 1100/2190] 10.222.0.82 () {60 vars in 1144 bytes} [Tue Aug 2 10:17:37 2022] POST /identity/v3/users => generated 288 bytes in 221 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 02 10:17:37.335431 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 1091/2191] 10.222.0.82 () {58 vars in 1122 bytes} [Tue Aug 2 10:17:37 2022] GET /identity/v3/roles => generated 1420 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 02 10:17:37.336076 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ffcdb3c-f946-4d62-8d3f-976939b5e577 admin admin] Authenticating user token {{(pid=80377) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:17:37.336839 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9630b35d-5b54-4fc5-88c1-a7ef1aa203d2 admin admin] Authenticating user token {{(pid=80378) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:17:37.352772 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9630b35d-5b54-4fc5-88c1-a7ef1aa203d2 admin admin] Validating token access rules against request {{(pid=80378) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:17:37.360373 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-858ae041-6fc5-4c2c-be44-496310f0f26e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '891005fa5ff74041b75747d16cc66798', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7047912c5acd4adeba5d87c3bf895e39', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80378) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:17:37.361187 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-858ae041-6fc5-4c2c-be44-496310f0f26e admin admin] REQUEST_METHOD: `PUT` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:17:37.361571 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-858ae041-6fc5-4c2c-be44-496310f0f26e admin admin] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:17:37.361952 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-858ae041-6fc5-4c2c-be44-496310f0f26e admin admin] PATH_INFO: `/v3/projects/99a92325ab3d4ec2bbecffb9a6e609d5/users/7b672b80961445c3a3a9b2d784f64208/roles/e1c04b19fc4543b8b54f58bbcffbc598` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:17:37.367351 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ffcdb3c-f946-4d62-8d3f-976939b5e577 admin admin] Validating token access rules against request {{(pid=80377) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:17:37.387349 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-858ae041-6fc5-4c2c-be44-496310f0f26e admin admin] RBAC: Authorizing `identity:create_grant(project_id=99a92325ab3d4ec2bbecffb9a6e609d5, user_id=7b672b80961445c3a3a9b2d784f64208, role_id=e1c04b19fc4543b8b54f58bbcffbc598)` {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:17:37.391684 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24046de9-4807-4736-be1e-0a3bafbe906f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '891005fa5ff74041b75747d16cc66798', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7047912c5acd4adeba5d87c3bf895e39', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80377) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:17:37.392333 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24046de9-4807-4736-be1e-0a3bafbe906f admin admin] REQUEST_METHOD: `GET` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:17:37.392626 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24046de9-4807-4736-be1e-0a3bafbe906f admin admin] SCRIPT_NAME: `/identity` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:17:37.392905 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24046de9-4807-4736-be1e-0a3bafbe906f admin admin] PATH_INFO: `/v3/roles` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:17:37.393687 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24046de9-4807-4736-be1e-0a3bafbe906f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:17:37.395229 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-858ae041-6fc5-4c2c-be44-496310f0f26e admin admin] RBAC: Authorization granted {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:17:37.395386 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24046de9-4807-4736-be1e-0a3bafbe906f admin admin] RBAC: Authorization granted {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:17:37.419600 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: [pid: 80377|app: 0|req: 1101/2192] 10.222.0.82 () {58 vars in 1122 bytes} [Tue Aug 2 10:17:37 2022] GET /identity/v3/roles => generated 1420 bytes in 84 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 02 10:17:37.423579 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24046de9-4807-4736-be1e-0a3bafbe906f admin admin] Authenticating user token {{(pid=80377) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:17:37.426941 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 1092/2193] 10.222.0.82 () {60 vars in 1483 bytes} [Tue Aug 2 10:17:37 2022] PUT /identity/v3/projects/99a92325ab3d4ec2bbecffb9a6e609d5/users/7b672b80961445c3a3a9b2d784f64208/roles/e1c04b19fc4543b8b54f58bbcffbc598 => generated 0 bytes in 91 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 02 10:17:37.432421 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e59c100-c6df-4c7d-83d7-a9379c5c9718 None None] REQUEST_METHOD: `POST` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:17:37.432923 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e59c100-c6df-4c7d-83d7-a9379c5c9718 None None] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:17:37.433760 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e59c100-c6df-4c7d-83d7-a9379c5c9718 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:17:37.457812 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24046de9-4807-4736-be1e-0a3bafbe906f admin admin] Validating token access rules against request {{(pid=80377) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:17:37.470501 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.auth.core [None req-3e59c100-c6df-4c7d-83d7-a9379c5c9718 None None] MFA Rules not processed for user `891005fa5ff74041b75747d16cc66798`. Rule list: `[]` (Enabled: `True`). {{(pid=80378) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 02 10:17:37.472260 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.fernet_utils [None req-3e59c100-c6df-4c7d-83d7-a9379c5c9718 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80378) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 10:17:37.543533 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efd7159b-224b-443e-a570-7b31d98e9800 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '891005fa5ff74041b75747d16cc66798', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7047912c5acd4adeba5d87c3bf895e39', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80377) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:17:37.544222 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efd7159b-224b-443e-a570-7b31d98e9800 admin admin] REQUEST_METHOD: `GET` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:17:37.545047 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efd7159b-224b-443e-a570-7b31d98e9800 admin admin] SCRIPT_NAME: `/identity` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:17:37.545338 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efd7159b-224b-443e-a570-7b31d98e9800 admin admin] PATH_INFO: `/v3/roles` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:17:37.546183 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-efd7159b-224b-443e-a570-7b31d98e9800 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:17:37.547959 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-efd7159b-224b-443e-a570-7b31d98e9800 admin admin] RBAC: Authorization granted {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:17:37.564842 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: [pid: 80377|app: 0|req: 1102/2194] 10.222.0.82 () {58 vars in 1122 bytes} [Tue Aug 2 10:17:37 2022] GET /identity/v3/roles => generated 1420 bytes in 139 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 02 10:17:37.567861 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efd7159b-224b-443e-a570-7b31d98e9800 admin admin] Authenticating user token {{(pid=80377) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:17:37.581052 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efd7159b-224b-443e-a570-7b31d98e9800 admin admin] Validating token access rules against request {{(pid=80377) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:17:37.586113 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fa9e29d-b22b-4866-b280-9603c1a23ac1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '891005fa5ff74041b75747d16cc66798', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7047912c5acd4adeba5d87c3bf895e39', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80377) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:17:37.586846 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fa9e29d-b22b-4866-b280-9603c1a23ac1 admin admin] REQUEST_METHOD: `PUT` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:17:37.587134 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fa9e29d-b22b-4866-b280-9603c1a23ac1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:17:37.587418 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fa9e29d-b22b-4866-b280-9603c1a23ac1 admin admin] PATH_INFO: `/v3/projects/8a6d0cf631f54fef94a079c69c4b6ca2/users/17057dba1f78414c940890ccb96b64e3/roles/e1c04b19fc4543b8b54f58bbcffbc598` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:17:37.602487 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 1093/2195] 10.222.0.82 () {58 vars in 958 bytes} [Tue Aug 2 10:17:37 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 169 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 02 10:17:37.606534 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e59c100-c6df-4c7d-83d7-a9379c5c9718 None None] Authenticating user token {{(pid=80378) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:17:37.608668 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.fernet_utils [None req-3e59c100-c6df-4c7d-83d7-a9379c5c9718 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80378) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 10:17:37.609534 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5fa9e29d-b22b-4866-b280-9603c1a23ac1 admin admin] RBAC: Authorizing `identity:create_grant(project_id=8a6d0cf631f54fef94a079c69c4b6ca2, user_id=17057dba1f78414c940890ccb96b64e3, role_id=e1c04b19fc4543b8b54f58bbcffbc598)` {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:17:37.613025 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5fa9e29d-b22b-4866-b280-9603c1a23ac1 admin admin] RBAC: Authorization granted {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:17:37.632106 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: [pid: 80377|app: 0|req: 1103/2196] 10.222.0.82 () {60 vars in 1483 bytes} [Tue Aug 2 10:17:37 2022] PUT /identity/v3/projects/8a6d0cf631f54fef94a079c69c4b6ca2/users/17057dba1f78414c940890ccb96b64e3/roles/e1c04b19fc4543b8b54f58bbcffbc598 => generated 0 bytes in 65 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 02 10:17:37.636439 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG dbcounter [-] [80378] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=80378) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 10:17:37.643549 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86746070-214e-4d89-8902-517ef7fc2941 None None] REQUEST_METHOD: `POST` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:17:37.644344 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86746070-214e-4d89-8902-517ef7fc2941 None None] SCRIPT_NAME: `/identity` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:17:37.644790 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86746070-214e-4d89-8902-517ef7fc2941 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:17:37.682287 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.auth.core [None req-86746070-214e-4d89-8902-517ef7fc2941 None None] MFA Rules not processed for user `17057dba1f78414c940890ccb96b64e3`. Rule list: `[]` (Enabled: `True`). {{(pid=80377) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 02 10:17:37.684258 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.fernet_utils [None req-86746070-214e-4d89-8902-517ef7fc2941 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80377) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 10:17:37.720132 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e59c100-c6df-4c7d-83d7-a9379c5c9718 None None] Validating token access rules against request {{(pid=80378) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:17:37.784526 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce161ec6-3c08-4b3b-bdd5-16238c427220 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '33b87170f0634b67bce60750847b5cbe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f1b38ebb804c41c69772c998aae72c41', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80378) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:17:37.785375 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce161ec6-3c08-4b3b-bdd5-16238c427220 service neutron] REQUEST_METHOD: `GET` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:17:37.785756 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce161ec6-3c08-4b3b-bdd5-16238c427220 service neutron] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:17:37.786157 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce161ec6-3c08-4b3b-bdd5-16238c427220 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:17:37.802982 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: [pid: 80377|app: 0|req: 1104/2197] 10.222.0.82 () {58 vars in 958 bytes} [Tue Aug 2 10:17:37 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 157 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 02 10:17:37.805407 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86746070-214e-4d89-8902-517ef7fc2941 None None] Authenticating user token {{(pid=80377) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:17:37.871920 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG dbcounter [-] [80377] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=80377) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 10:17:37.906815 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce161ec6-3c08-4b3b-bdd5-16238c427220 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:17:37.908332 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce161ec6-3c08-4b3b-bdd5-16238c427220 service neutron] RBAC: Authorization granted {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:17:37.920944 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 1094/2198] 10.222.0.82 () {60 vars in 1365 bytes} [Tue Aug 2 10:17:37 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 315 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 02 10:17:37.940610 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86746070-214e-4d89-8902-517ef7fc2941 None None] Validating token access rules against request {{(pid=80377) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:17:37.946092 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7530221-985b-40bd-996d-c581e7de788f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '891005fa5ff74041b75747d16cc66798', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7047912c5acd4adeba5d87c3bf895e39', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80377) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:17:37.946918 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7530221-985b-40bd-996d-c581e7de788f admin admin] REQUEST_METHOD: `POST` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:17:37.947379 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7530221-985b-40bd-996d-c581e7de788f admin admin] SCRIPT_NAME: `/identity` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:17:37.947790 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7530221-985b-40bd-996d-c581e7de788f admin admin] PATH_INFO: `/v3/projects` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:17:37.948625 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7530221-985b-40bd-996d-c581e7de788f admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:17:37.951080 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7530221-985b-40bd-996d-c581e7de788f admin admin] RBAC: Authorization granted {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:17:37.994236 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: [pid: 80377|app: 0|req: 1105/2199] 10.222.0.82 () {60 vars in 1153 bytes} [Tue Aug 2 10:17:37 2022] POST /identity/v3/projects => generated 377 bytes in 190 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 02 10:17:38.001118 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce161ec6-3c08-4b3b-bdd5-16238c427220 service neutron] Authenticating user token {{(pid=80378) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:17:38.025901 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce161ec6-3c08-4b3b-bdd5-16238c427220 service neutron] Validating token access rules against request {{(pid=80378) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:17:38.186326 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58a30297-1896-421b-8d26-490476d530cd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '891005fa5ff74041b75747d16cc66798', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7047912c5acd4adeba5d87c3bf895e39', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80378) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:17:38.187794 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58a30297-1896-421b-8d26-490476d530cd admin admin] REQUEST_METHOD: `POST` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:17:38.188157 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58a30297-1896-421b-8d26-490476d530cd admin admin] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:17:38.188663 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58a30297-1896-421b-8d26-490476d530cd admin admin] PATH_INFO: `/v3/users` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:17:38.189132 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58a30297-1896-421b-8d26-490476d530cd admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:17:38.191258 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58a30297-1896-421b-8d26-490476d530cd admin admin] RBAC: Authorization granted {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:17:38.241447 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 1095/2200] 10.222.0.82 () {60 vars in 1144 bytes} [Tue Aug 2 10:17:37 2022] POST /identity/v3/users => generated 295 bytes in 244 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 02 10:17:38.245608 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7530221-985b-40bd-996d-c581e7de788f admin admin] Authenticating user token {{(pid=80377) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:17:38.263904 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7530221-985b-40bd-996d-c581e7de788f admin admin] Validating token access rules against request {{(pid=80377) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:17:38.271137 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-224de708-6ac4-4e00-ac8b-3469840135a4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '891005fa5ff74041b75747d16cc66798', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7047912c5acd4adeba5d87c3bf895e39', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80377) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:17:38.271788 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-224de708-6ac4-4e00-ac8b-3469840135a4 admin admin] REQUEST_METHOD: `GET` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:17:38.272072 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-224de708-6ac4-4e00-ac8b-3469840135a4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:17:38.272348 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-224de708-6ac4-4e00-ac8b-3469840135a4 admin admin] PATH_INFO: `/v3/roles` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:17:38.276854 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-224de708-6ac4-4e00-ac8b-3469840135a4 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:17:38.282082 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-224de708-6ac4-4e00-ac8b-3469840135a4 admin admin] RBAC: Authorization granted {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:17:38.306087 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: [pid: 80377|app: 0|req: 1106/2201] 10.222.0.82 () {58 vars in 1122 bytes} [Tue Aug 2 10:17:38 2022] GET /identity/v3/roles => generated 1420 bytes in 60 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 02 10:17:38.307195 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58a30297-1896-421b-8d26-490476d530cd admin admin] Authenticating user token {{(pid=80378) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:17:38.326248 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58a30297-1896-421b-8d26-490476d530cd admin admin] Validating token access rules against request {{(pid=80378) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:17:38.333569 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b267d1e-a22d-44de-b823-e0f420204db6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '891005fa5ff74041b75747d16cc66798', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7047912c5acd4adeba5d87c3bf895e39', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80378) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:17:38.334140 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b267d1e-a22d-44de-b823-e0f420204db6 admin admin] REQUEST_METHOD: `PUT` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:17:38.334775 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b267d1e-a22d-44de-b823-e0f420204db6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:17:38.335993 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b267d1e-a22d-44de-b823-e0f420204db6 admin admin] PATH_INFO: `/v3/projects/33eb10ad74fe440da7f9bc1704b3b570/users/f3e913d3ba1a48f1a28ed62953d96af7/roles/237966442f05411191105bf336bc14f3` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:17:38.358980 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG dbcounter [-] [80378] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80378) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 10:17:38.363376 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b267d1e-a22d-44de-b823-e0f420204db6 admin admin] RBAC: Authorizing `identity:create_grant(project_id=33eb10ad74fe440da7f9bc1704b3b570, user_id=f3e913d3ba1a48f1a28ed62953d96af7, role_id=237966442f05411191105bf336bc14f3)` {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:17:38.366196 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b267d1e-a22d-44de-b823-e0f420204db6 admin admin] RBAC: Authorization granted {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:17:38.385954 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 1096/2202] 10.222.0.82 () {60 vars in 1483 bytes} [Tue Aug 2 10:17:38 2022] PUT /identity/v3/projects/33eb10ad74fe440da7f9bc1704b3b570/users/f3e913d3ba1a48f1a28ed62953d96af7/roles/237966442f05411191105bf336bc14f3 => generated 0 bytes in 78 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 02 10:17:38.387405 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-224de708-6ac4-4e00-ac8b-3469840135a4 admin admin] Authenticating user token {{(pid=80377) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:17:38.401721 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-224de708-6ac4-4e00-ac8b-3469840135a4 admin admin] Validating token access rules against request {{(pid=80377) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:17:38.497518 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fe5ffe0-89b0-408e-af18-95b1f92c86dd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '891005fa5ff74041b75747d16cc66798', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7047912c5acd4adeba5d87c3bf895e39', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80377) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:17:38.497518 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fe5ffe0-89b0-408e-af18-95b1f92c86dd admin admin] REQUEST_METHOD: `GET` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:17:38.497518 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fe5ffe0-89b0-408e-af18-95b1f92c86dd admin admin] SCRIPT_NAME: `/identity` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:17:38.497518 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fe5ffe0-89b0-408e-af18-95b1f92c86dd admin admin] PATH_INFO: `/v3/roles` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:17:38.498124 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1fe5ffe0-89b0-408e-af18-95b1f92c86dd admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:17:38.501877 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1fe5ffe0-89b0-408e-af18-95b1f92c86dd admin admin] RBAC: Authorization granted {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:17:38.534599 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: [pid: 80377|app: 0|req: 1107/2203] 10.222.0.82 () {58 vars in 1122 bytes} [Tue Aug 2 10:17:38 2022] GET /identity/v3/roles => generated 1420 bytes in 145 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 02 10:17:38.536126 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b267d1e-a22d-44de-b823-e0f420204db6 admin admin] Authenticating user token {{(pid=80378) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:17:38.554244 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b267d1e-a22d-44de-b823-e0f420204db6 admin admin] Validating token access rules against request {{(pid=80378) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:17:38.562877 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1024300-3b1b-4e6e-93eb-4305f4560e89 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '891005fa5ff74041b75747d16cc66798', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7047912c5acd4adeba5d87c3bf895e39', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80378) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:17:38.563577 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1024300-3b1b-4e6e-93eb-4305f4560e89 admin admin] REQUEST_METHOD: `PUT` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:17:38.563952 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1024300-3b1b-4e6e-93eb-4305f4560e89 admin admin] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:17:38.564228 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1024300-3b1b-4e6e-93eb-4305f4560e89 admin admin] PATH_INFO: `/v3/projects/33eb10ad74fe440da7f9bc1704b3b570/users/f3e913d3ba1a48f1a28ed62953d96af7/roles/237966442f05411191105bf336bc14f3` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:17:38.569748 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1024300-3b1b-4e6e-93eb-4305f4560e89 admin admin] RBAC: Authorizing `identity:create_grant(project_id=33eb10ad74fe440da7f9bc1704b3b570, user_id=f3e913d3ba1a48f1a28ed62953d96af7, role_id=237966442f05411191105bf336bc14f3)` {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:17:38.572456 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1024300-3b1b-4e6e-93eb-4305f4560e89 admin admin] RBAC: Authorization granted {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:17:38.610978 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 1097/2204] 10.222.0.82 () {60 vars in 1483 bytes} [Tue Aug 2 10:17:38 2022] PUT /identity/v3/projects/33eb10ad74fe440da7f9bc1704b3b570/users/f3e913d3ba1a48f1a28ed62953d96af7/roles/237966442f05411191105bf336bc14f3 => generated 0 bytes in 67 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 02 10:17:38.616312 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84ad6a01-b68a-4a57-82a1-1614058ff66d None None] REQUEST_METHOD: `POST` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:17:38.616604 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84ad6a01-b68a-4a57-82a1-1614058ff66d None None] SCRIPT_NAME: `/identity` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:17:38.616940 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84ad6a01-b68a-4a57-82a1-1614058ff66d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:17:38.702134 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.auth.core [None req-84ad6a01-b68a-4a57-82a1-1614058ff66d None None] MFA Rules not processed for user `f3e913d3ba1a48f1a28ed62953d96af7`. Rule list: `[]` (Enabled: `True`). {{(pid=80377) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 02 10:17:38.704778 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.fernet_utils [None req-84ad6a01-b68a-4a57-82a1-1614058ff66d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80377) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 10:17:38.844788 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: [pid: 80377|app: 0|req: 1108/2205] 10.222.0.82 () {58 vars in 958 bytes} [Tue Aug 2 10:17:38 2022] POST /identity/v3/auth/tokens => generated 3321 bytes in 230 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 02 10:17:38.862392 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1024300-3b1b-4e6e-93eb-4305f4560e89 admin admin] Authenticating user token {{(pid=80378) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:17:38.864500 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.fernet_utils [None req-b1024300-3b1b-4e6e-93eb-4305f4560e89 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80378) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 10:17:39.004765 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1024300-3b1b-4e6e-93eb-4305f4560e89 admin admin] Validating token access rules against request {{(pid=80378) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:17:39.010395 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-010d1f72-f7f5-4416-b81b-73d2c06e5a73 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1023de281ae74e9d8a6d6c312c1c97a7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f1b38ebb804c41c69772c998aae72c41', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80378) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:17:39.011511 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-010d1f72-f7f5-4416-b81b-73d2c06e5a73 service cinder] REQUEST_METHOD: `GET` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:17:39.011953 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-010d1f72-f7f5-4416-b81b-73d2c06e5a73 service cinder] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:17:39.012389 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-010d1f72-f7f5-4416-b81b-73d2c06e5a73 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:17:39.142332 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-010d1f72-f7f5-4416-b81b-73d2c06e5a73 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:17:39.147936 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-010d1f72-f7f5-4416-b81b-73d2c06e5a73 service cinder] RBAC: Authorization granted {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:17:39.174683 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 1098/2206] 10.222.0.82 () {60 vars in 1365 bytes} [Tue Aug 2 10:17:38 2022] GET /identity/v3/auth/tokens => generated 3252 bytes in 316 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 02 10:17:40.043321 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b375f8d1-9d93-48f9-8e90-cbd4755256a5 None None] REQUEST_METHOD: `POST` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:17:40.044257 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b375f8d1-9d93-48f9-8e90-cbd4755256a5 None None] SCRIPT_NAME: `/identity` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:17:40.045230 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b375f8d1-9d93-48f9-8e90-cbd4755256a5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:17:40.086283 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.auth.core [None req-b375f8d1-9d93-48f9-8e90-cbd4755256a5 None None] MFA Rules not processed for user `8db28092597b454dbfb040c0d8f35a96`. Rule list: `[]` (Enabled: `True`). {{(pid=80377) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 02 10:17:40.088036 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.fernet_utils [None req-b375f8d1-9d93-48f9-8e90-cbd4755256a5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80377) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 10:17:40.104071 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG dbcounter [-] [80377] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80377) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 10:17:40.194728 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: [pid: 80377|app: 0|req: 1109/2207] 10.222.0.82 () {58 vars in 958 bytes} [Tue Aug 2 10:17:40 2022] POST /identity/v3/auth/tokens => generated 3325 bytes in 153 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 02 10:17:40.195547 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-010d1f72-f7f5-4416-b81b-73d2c06e5a73 service cinder] Authenticating user token {{(pid=80378) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:17:40.283430 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-010d1f72-f7f5-4416-b81b-73d2c06e5a73 service cinder] Validating token access rules against request {{(pid=80378) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:17:40.287904 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e2f772f-66c2-4a2c-bcb6-3be51914c4df service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa964af53a2b4fb9ace6afb544d54d40', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f1b38ebb804c41c69772c998aae72c41', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80378) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:17:40.289429 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e2f772f-66c2-4a2c-bcb6-3be51914c4df service glance] REQUEST_METHOD: `GET` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:17:40.289830 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e2f772f-66c2-4a2c-bcb6-3be51914c4df service glance] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:17:40.290462 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e2f772f-66c2-4a2c-bcb6-3be51914c4df service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:17:40.292293 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.fernet_utils [None req-0e2f772f-66c2-4a2c-bcb6-3be51914c4df service glance] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80378) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 10:17:40.368936 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG dbcounter [-] [80378] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80378) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 10:17:40.387988 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e2f772f-66c2-4a2c-bcb6-3be51914c4df service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:17:40.389735 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e2f772f-66c2-4a2c-bcb6-3be51914c4df service glance] RBAC: Authorization granted {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:17:40.410041 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 1099/2208] 10.222.0.82 () {60 vars in 1365 bytes} [Tue Aug 2 10:17:40 2022] GET /identity/v3/auth/tokens => generated 3262 bytes in 215 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 02 10:17:40.632376 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e5d0401-88e3-40c6-ab58-f8b46897b23c None None] REQUEST_METHOD: `POST` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:17:40.632670 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e5d0401-88e3-40c6-ab58-f8b46897b23c None None] SCRIPT_NAME: `/identity` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:17:40.632958 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e5d0401-88e3-40c6-ab58-f8b46897b23c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:17:40.660120 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.auth.core [None req-5e5d0401-88e3-40c6-ab58-f8b46897b23c None None] MFA Rules not processed for user `bd8dd72086db48b8aaac2ed0058872df`. Rule list: `[]` (Enabled: `True`). {{(pid=80377) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 02 10:17:40.661673 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.fernet_utils [None req-5e5d0401-88e3-40c6-ab58-f8b46897b23c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80377) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 10:17:40.755692 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: [pid: 80377|app: 0|req: 1110/2209] 10.222.0.82 () {58 vars in 1009 bytes} [Tue Aug 2 10:17:40 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 124 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 02 10:17:40.830493 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e2f772f-66c2-4a2c-bcb6-3be51914c4df service glance] Authenticating user token {{(pid=80378) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:17:40.832020 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.fernet_utils [None req-0e2f772f-66c2-4a2c-bcb6-3be51914c4df service glance] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80378) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 10:17:41.045741 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e2f772f-66c2-4a2c-bcb6-3be51914c4df service glance] Validating token access rules against request {{(pid=80378) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:17:41.056337 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41d3a933-3475-4b91-9b13-6456fb0886a6 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bef1b2e714cb45c4a51eb6d3e15aad6d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f1b38ebb804c41c69772c998aae72c41', 'project_domain_id': 'default', 'roles': ['admin', 'service', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80378) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:17:41.057235 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41d3a933-3475-4b91-9b13-6456fb0886a6 service swift] REQUEST_METHOD: `GET` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:17:41.057643 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41d3a933-3475-4b91-9b13-6456fb0886a6 service swift] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:17:41.058050 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41d3a933-3475-4b91-9b13-6456fb0886a6 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:17:41.076792 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41d3a933-3475-4b91-9b13-6456fb0886a6 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:17:41.078686 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41d3a933-3475-4b91-9b13-6456fb0886a6 service swift] RBAC: Authorization granted {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:17:41.113064 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 1100/2210] 10.222.0.82 () {60 vars in 1384 bytes} [Tue Aug 2 10:17:40 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 283 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Aug 02 10:17:41.386009 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e5d0401-88e3-40c6-ab58-f8b46897b23c None None] Authenticating user token {{(pid=80377) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:17:41.399337 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e5d0401-88e3-40c6-ab58-f8b46897b23c None None] Validating token access rules against request {{(pid=80377) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:17:41.403416 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f814b6c2-0cc7-4603-b42c-56bc7acdd9fb service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1023de281ae74e9d8a6d6c312c1c97a7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f1b38ebb804c41c69772c998aae72c41', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80377) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:17:41.404707 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f814b6c2-0cc7-4603-b42c-56bc7acdd9fb service cinder] REQUEST_METHOD: `GET` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:17:41.404996 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f814b6c2-0cc7-4603-b42c-56bc7acdd9fb service cinder] SCRIPT_NAME: `/identity` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:17:41.405272 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f814b6c2-0cc7-4603-b42c-56bc7acdd9fb service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:17:41.495140 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f814b6c2-0cc7-4603-b42c-56bc7acdd9fb service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:17:41.497853 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f814b6c2-0cc7-4603-b42c-56bc7acdd9fb service cinder] RBAC: Authorization granted {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:17:41.522666 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: [pid: 80377|app: 0|req: 1111/2211] 10.222.0.82 () {60 vars in 1365 bytes} [Tue Aug 2 10:17:41 2022] GET /identity/v3/auth/tokens => generated 3252 bytes in 137 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 02 10:17:42.610240 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41d3a933-3475-4b91-9b13-6456fb0886a6 service swift] Authenticating user token {{(pid=80378) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:17:42.625219 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41d3a933-3475-4b91-9b13-6456fb0886a6 service swift] Validating token access rules against request {{(pid=80378) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:17:42.648963 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d069202-7002-4231-8200-0e6616bb3c2f service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1023de281ae74e9d8a6d6c312c1c97a7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f1b38ebb804c41c69772c998aae72c41', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80378) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:17:42.652177 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d069202-7002-4231-8200-0e6616bb3c2f service cinder] REQUEST_METHOD: `GET` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:17:42.652777 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d069202-7002-4231-8200-0e6616bb3c2f service cinder] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:17:42.655059 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d069202-7002-4231-8200-0e6616bb3c2f service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:17:42.687588 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d069202-7002-4231-8200-0e6616bb3c2f service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:17:42.689300 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d069202-7002-4231-8200-0e6616bb3c2f service cinder] RBAC: Authorization granted {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:17:42.725984 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 1101/2212] 10.222.0.82 () {60 vars in 1365 bytes} [Tue Aug 2 10:17:42 2022] GET /identity/v3/auth/tokens => generated 3321 bytes in 115 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 02 10:17:46.199926 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9521af71-86e8-4921-8b0a-1b61c74867a3 None None] REQUEST_METHOD: `POST` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:17:46.200480 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9521af71-86e8-4921-8b0a-1b61c74867a3 None None] SCRIPT_NAME: `/identity` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:17:46.201626 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9521af71-86e8-4921-8b0a-1b61c74867a3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:17:46.311965 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.auth.core [None req-9521af71-86e8-4921-8b0a-1b61c74867a3 None None] MFA Rules not processed for user `7b672b80961445c3a3a9b2d784f64208`. Rule list: `[]` (Enabled: `True`). {{(pid=80377) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 02 10:17:46.315519 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.fernet_utils [None req-9521af71-86e8-4921-8b0a-1b61c74867a3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80377) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 10:17:46.432911 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG dbcounter [-] [80377] Writing DB stats keystone:SELECT=100 {{(pid=80377) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 10:17:46.512235 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: [pid: 80377|app: 0|req: 1112/2213] 10.222.0.82 () {58 vars in 958 bytes} [Tue Aug 2 10:17:46 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 314 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 02 10:17:46.517465 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d069202-7002-4231-8200-0e6616bb3c2f service cinder] Authenticating user token {{(pid=80378) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:17:46.690494 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d069202-7002-4231-8200-0e6616bb3c2f service cinder] Validating token access rules against request {{(pid=80378) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:17:46.707813 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ab8d216-b111-4207-ab31-6867952a3277 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '891005fa5ff74041b75747d16cc66798', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7047912c5acd4adeba5d87c3bf895e39', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80378) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:17:46.709009 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ab8d216-b111-4207-ab31-6867952a3277 admin admin] REQUEST_METHOD: `POST` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:17:46.712275 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ab8d216-b111-4207-ab31-6867952a3277 admin admin] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:17:46.713410 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ab8d216-b111-4207-ab31-6867952a3277 admin admin] PATH_INFO: `/v3/projects` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:17:46.715134 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ab8d216-b111-4207-ab31-6867952a3277 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:17:46.731599 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ab8d216-b111-4207-ab31-6867952a3277 admin admin] RBAC: Authorization granted {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:17:46.914250 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9521af71-86e8-4921-8b0a-1b61c74867a3 None None] Authenticating user token {{(pid=80377) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:17:46.914648 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 1102/2214] 10.222.0.82 () {60 vars in 1153 bytes} [Tue Aug 2 10:17:46 2022] POST /identity/v3/projects => generated 371 bytes in 398 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 02 10:17:46.957355 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9521af71-86e8-4921-8b0a-1b61c74867a3 None None] Validating token access rules against request {{(pid=80377) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:17:47.070978 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a307ed3a-f80f-4cf2-a38f-0903f2969e51 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '891005fa5ff74041b75747d16cc66798', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7047912c5acd4adeba5d87c3bf895e39', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80377) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:17:47.071820 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a307ed3a-f80f-4cf2-a38f-0903f2969e51 admin admin] REQUEST_METHOD: `POST` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:17:47.072230 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a307ed3a-f80f-4cf2-a38f-0903f2969e51 admin admin] SCRIPT_NAME: `/identity` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:17:47.072620 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a307ed3a-f80f-4cf2-a38f-0903f2969e51 admin admin] PATH_INFO: `/v3/users` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:17:47.073416 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a307ed3a-f80f-4cf2-a38f-0903f2969e51 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:17:47.075813 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a307ed3a-f80f-4cf2-a38f-0903f2969e51 admin admin] RBAC: Authorization granted {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:17:47.112883 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: [pid: 80377|app: 0|req: 1113/2215] 10.222.0.82 () {60 vars in 1144 bytes} [Tue Aug 2 10:17:46 2022] POST /identity/v3/users => generated 292 bytes in 205 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 02 10:17:47.116393 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ab8d216-b111-4207-ab31-6867952a3277 admin admin] Authenticating user token {{(pid=80378) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:17:47.132571 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ab8d216-b111-4207-ab31-6867952a3277 admin admin] Validating token access rules against request {{(pid=80378) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:17:47.138148 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94b8e640-c42b-45fc-a0dc-a71dd389a1e3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '891005fa5ff74041b75747d16cc66798', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7047912c5acd4adeba5d87c3bf895e39', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80378) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:17:47.138817 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94b8e640-c42b-45fc-a0dc-a71dd389a1e3 admin admin] REQUEST_METHOD: `GET` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:17:47.139513 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94b8e640-c42b-45fc-a0dc-a71dd389a1e3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:17:47.139797 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94b8e640-c42b-45fc-a0dc-a71dd389a1e3 admin admin] PATH_INFO: `/v3/roles` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:17:47.140645 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94b8e640-c42b-45fc-a0dc-a71dd389a1e3 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:17:47.142450 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94b8e640-c42b-45fc-a0dc-a71dd389a1e3 admin admin] RBAC: Authorization granted {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:17:47.170263 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a307ed3a-f80f-4cf2-a38f-0903f2969e51 admin admin] Authenticating user token {{(pid=80377) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:17:47.170578 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 1103/2216] 10.222.0.82 () {58 vars in 1122 bytes} [Tue Aug 2 10:17:47 2022] GET /identity/v3/roles => generated 1420 bytes in 55 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 02 10:17:47.184993 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a307ed3a-f80f-4cf2-a38f-0903f2969e51 admin admin] Validating token access rules against request {{(pid=80377) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:17:47.196898 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42ee74be-f418-4d2d-8d7b-9aeeac6e4ac1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '891005fa5ff74041b75747d16cc66798', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7047912c5acd4adeba5d87c3bf895e39', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80377) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:17:47.198380 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42ee74be-f418-4d2d-8d7b-9aeeac6e4ac1 admin admin] REQUEST_METHOD: `PUT` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:17:47.198919 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42ee74be-f418-4d2d-8d7b-9aeeac6e4ac1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:17:47.199431 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42ee74be-f418-4d2d-8d7b-9aeeac6e4ac1 admin admin] PATH_INFO: `/v3/projects/e12bfff080094a6ca25073c5f68987e1/users/3d378bdfcbc5436f81a0e3ad04822ebe/roles/237966442f05411191105bf336bc14f3` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:17:47.256322 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42ee74be-f418-4d2d-8d7b-9aeeac6e4ac1 admin admin] RBAC: Authorizing `identity:create_grant(project_id=e12bfff080094a6ca25073c5f68987e1, user_id=3d378bdfcbc5436f81a0e3ad04822ebe, role_id=237966442f05411191105bf336bc14f3)` {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:17:47.259534 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42ee74be-f418-4d2d-8d7b-9aeeac6e4ac1 admin admin] RBAC: Authorization granted {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:17:47.315978 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: [pid: 80377|app: 0|req: 1114/2217] 10.222.0.82 () {60 vars in 1483 bytes} [Tue Aug 2 10:17:47 2022] PUT /identity/v3/projects/e12bfff080094a6ca25073c5f68987e1/users/3d378bdfcbc5436f81a0e3ad04822ebe/roles/237966442f05411191105bf336bc14f3 => generated 0 bytes in 147 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 02 10:17:47.326883 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94b8e640-c42b-45fc-a0dc-a71dd389a1e3 admin admin] Authenticating user token {{(pid=80378) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:17:47.343006 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94b8e640-c42b-45fc-a0dc-a71dd389a1e3 admin admin] Validating token access rules against request {{(pid=80378) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:17:47.429948 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG dbcounter [-] [80378] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80378) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 10:17:47.545216 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c946932-1910-4471-b420-e9370fab447c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '891005fa5ff74041b75747d16cc66798', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7047912c5acd4adeba5d87c3bf895e39', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80378) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:17:47.545996 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c946932-1910-4471-b420-e9370fab447c admin admin] REQUEST_METHOD: `GET` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:17:47.546300 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c946932-1910-4471-b420-e9370fab447c admin admin] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:17:47.546586 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c946932-1910-4471-b420-e9370fab447c admin admin] PATH_INFO: `/v3/roles` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:17:47.547308 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c946932-1910-4471-b420-e9370fab447c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:17:47.548946 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c946932-1910-4471-b420-e9370fab447c admin admin] RBAC: Authorization granted {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:17:47.567084 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 1104/2218] 10.222.0.82 () {58 vars in 1122 bytes} [Tue Aug 2 10:17:47 2022] GET /identity/v3/roles => generated 1420 bytes in 242 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 02 10:17:47.570948 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42ee74be-f418-4d2d-8d7b-9aeeac6e4ac1 admin admin] Authenticating user token {{(pid=80377) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:17:47.602523 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42ee74be-f418-4d2d-8d7b-9aeeac6e4ac1 admin admin] Validating token access rules against request {{(pid=80377) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:17:47.614497 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d62c5ee0-91fa-4c40-85c4-cb0f784550ab admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '891005fa5ff74041b75747d16cc66798', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7047912c5acd4adeba5d87c3bf895e39', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80377) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:17:47.615196 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d62c5ee0-91fa-4c40-85c4-cb0f784550ab admin admin] REQUEST_METHOD: `PUT` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:17:47.615510 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d62c5ee0-91fa-4c40-85c4-cb0f784550ab admin admin] SCRIPT_NAME: `/identity` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:17:47.615743 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d62c5ee0-91fa-4c40-85c4-cb0f784550ab admin admin] PATH_INFO: `/v3/projects/e12bfff080094a6ca25073c5f68987e1/users/3d378bdfcbc5436f81a0e3ad04822ebe/roles/237966442f05411191105bf336bc14f3` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:17:47.625016 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d62c5ee0-91fa-4c40-85c4-cb0f784550ab admin admin] RBAC: Authorizing `identity:create_grant(project_id=e12bfff080094a6ca25073c5f68987e1, user_id=3d378bdfcbc5436f81a0e3ad04822ebe, role_id=237966442f05411191105bf336bc14f3)` {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:17:47.647536 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d62c5ee0-91fa-4c40-85c4-cb0f784550ab admin admin] RBAC: Authorization granted {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:17:47.673423 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: [pid: 80377|app: 0|req: 1115/2219] 10.222.0.82 () {60 vars in 1483 bytes} [Tue Aug 2 10:17:47 2022] PUT /identity/v3/projects/e12bfff080094a6ca25073c5f68987e1/users/3d378bdfcbc5436f81a0e3ad04822ebe/roles/237966442f05411191105bf336bc14f3 => generated 0 bytes in 102 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 02 10:17:48.142834 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c946932-1910-4471-b420-e9370fab447c admin admin] Authenticating user token {{(pid=80378) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:17:48.163680 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c946932-1910-4471-b420-e9370fab447c admin admin] Validating token access rules against request {{(pid=80378) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:17:48.340992 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-674440fc-fc42-43d5-bb85-8b18702799fe admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '891005fa5ff74041b75747d16cc66798', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7047912c5acd4adeba5d87c3bf895e39', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80378) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:17:48.342602 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-674440fc-fc42-43d5-bb85-8b18702799fe admin admin] REQUEST_METHOD: `DELETE` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:17:48.342939 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-674440fc-fc42-43d5-bb85-8b18702799fe admin admin] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:17:48.344644 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-674440fc-fc42-43d5-bb85-8b18702799fe admin admin] PATH_INFO: `/v3/users/087d4ddfd8a3479f81e6d06af67ca7a3` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:17:48.348494 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-674440fc-fc42-43d5-bb85-8b18702799fe admin admin] RBAC: Authorizing `identity:delete_user(user_id=087d4ddfd8a3479f81e6d06af67ca7a3)` {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:17:48.355802 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-674440fc-fc42-43d5-bb85-8b18702799fe admin admin] RBAC: Authorization granted {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:17:48.448466 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.notifications [None req-674440fc-fc42-43d5-bb85-8b18702799fe admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '087d4ddfd8a3479f81e6d06af67ca7a3'} {{(pid=80378) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 10:17:48.464716 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.notifications [None req-674440fc-fc42-43d5-bb85-8b18702799fe admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '087d4ddfd8a3479f81e6d06af67ca7a3'} {{(pid=80378) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 10:17:48.482736 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.notifications [None req-674440fc-fc42-43d5-bb85-8b18702799fe admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '087d4ddfd8a3479f81e6d06af67ca7a3'} {{(pid=80378) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 10:17:48.484390 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.notifications [None req-674440fc-fc42-43d5-bb85-8b18702799fe admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '087d4ddfd8a3479f81e6d06af67ca7a3'} {{(pid=80378) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 10:17:48.519926 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.notifications [None req-674440fc-fc42-43d5-bb85-8b18702799fe admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '087d4ddfd8a3479f81e6d06af67ca7a3'} {{(pid=80378) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 10:17:48.528044 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 1105/2220] 10.222.0.82 () {58 vars in 1224 bytes} [Tue Aug 2 10:17:48 2022] DELETE /identity/v3/users/087d4ddfd8a3479f81e6d06af67ca7a3 => generated 0 bytes in 386 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 02 10:17:48.533312 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b30c03e3-b506-4742-b727-8b78dac92725 None None] REQUEST_METHOD: `POST` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:17:48.533885 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b30c03e3-b506-4742-b727-8b78dac92725 None None] SCRIPT_NAME: `/identity` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:17:48.533885 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b30c03e3-b506-4742-b727-8b78dac92725 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:17:48.563601 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.auth.core [None req-b30c03e3-b506-4742-b727-8b78dac92725 None None] MFA Rules not processed for user `891005fa5ff74041b75747d16cc66798`. Rule list: `[]` (Enabled: `True`). {{(pid=80377) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 02 10:17:48.566995 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.fernet_utils [None req-b30c03e3-b506-4742-b727-8b78dac92725 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80377) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 10:17:48.654949 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: [pid: 80377|app: 0|req: 1116/2221] 10.222.0.82 () {58 vars in 958 bytes} [Tue Aug 2 10:17:48 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 123 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 02 10:17:48.683534 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-674440fc-fc42-43d5-bb85-8b18702799fe admin admin] Authenticating user token {{(pid=80378) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:17:48.683534 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.fernet_utils [None req-674440fc-fc42-43d5-bb85-8b18702799fe admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80378) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 10:17:48.783428 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-674440fc-fc42-43d5-bb85-8b18702799fe admin admin] Validating token access rules against request {{(pid=80378) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:17:48.787435 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d98a8cf-8a70-47c1-8856-92f1fd13ed8d service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '33b87170f0634b67bce60750847b5cbe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f1b38ebb804c41c69772c998aae72c41', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80378) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:17:48.788100 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d98a8cf-8a70-47c1-8856-92f1fd13ed8d service neutron] REQUEST_METHOD: `GET` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:17:48.788381 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d98a8cf-8a70-47c1-8856-92f1fd13ed8d service neutron] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:17:48.788655 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d98a8cf-8a70-47c1-8856-92f1fd13ed8d service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:17:48.820282 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG dbcounter [-] [80378] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80378) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 10:17:48.829801 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d98a8cf-8a70-47c1-8856-92f1fd13ed8d service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:17:48.834054 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d98a8cf-8a70-47c1-8856-92f1fd13ed8d service neutron] RBAC: Authorization granted {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:17:48.860752 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 1106/2222] 10.222.0.82 () {60 vars in 1365 bytes} [Tue Aug 2 10:17:48 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 188 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 02 10:17:49.703912 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b30c03e3-b506-4742-b727-8b78dac92725 None None] Authenticating user token {{(pid=80377) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:17:49.717629 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.fernet_utils [None req-b30c03e3-b506-4742-b727-8b78dac92725 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80377) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 10:17:49.756099 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b30c03e3-b506-4742-b727-8b78dac92725 None None] Validating token access rules against request {{(pid=80377) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:17:49.767945 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2f757e3-32fc-4694-a3ad-d0d7c532b802 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '891005fa5ff74041b75747d16cc66798', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7047912c5acd4adeba5d87c3bf895e39', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80377) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:17:49.768914 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2f757e3-32fc-4694-a3ad-d0d7c532b802 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:17:49.769367 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2f757e3-32fc-4694-a3ad-d0d7c532b802 admin admin] SCRIPT_NAME: `/identity` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:17:49.769831 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2f757e3-32fc-4694-a3ad-d0d7c532b802 admin admin] PATH_INFO: `/v3/projects/b4a22005113e4ac9bcca471ab0f1d0a6` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:17:49.774045 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2f757e3-32fc-4694-a3ad-d0d7c532b802 admin admin] RBAC: Authorizing `identity:delete_project(project_id=b4a22005113e4ac9bcca471ab0f1d0a6)` {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:17:49.776586 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2f757e3-32fc-4694-a3ad-d0d7c532b802 admin admin] RBAC: Authorization granted {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:17:49.863597 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG dbcounter [-] [80377] Writing DB stats keystone:SELECT=93,keystone:INSERT=5,keystone:DELETE=2 {{(pid=80377) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 10:17:49.901081 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.notifications [None req-f2f757e3-32fc-4694-a3ad-d0d7c532b802 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'b4a22005113e4ac9bcca471ab0f1d0a6', 'request_id': 'req-f2f757e3-32fc-4694-a3ad-d0d7c532b802'} {{(pid=80377) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 10:17:49.940133 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.notifications [None req-f2f757e3-32fc-4694-a3ad-d0d7c532b802 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'b4a22005113e4ac9bcca471ab0f1d0a6', 'request_id': 'req-f2f757e3-32fc-4694-a3ad-d0d7c532b802'} {{(pid=80377) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 10:17:49.959892 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.notifications [None req-f2f757e3-32fc-4694-a3ad-d0d7c532b802 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'b4a22005113e4ac9bcca471ab0f1d0a6', 'request_id': 'req-f2f757e3-32fc-4694-a3ad-d0d7c532b802'} {{(pid=80377) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 10:17:50.000774 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.notifications [None req-f2f757e3-32fc-4694-a3ad-d0d7c532b802 admin admin] The token cache is being invalidate because project b4a22005113e4ac9bcca471ab0f1d0a6 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80377) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 02 10:17:50.001058 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.notifications [None req-f2f757e3-32fc-4694-a3ad-d0d7c532b802 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80377) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 10:17:50.004866 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.notifications [None req-f2f757e3-32fc-4694-a3ad-d0d7c532b802 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80377) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 10:17:50.007860 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: [pid: 80377|app: 0|req: 1117/2223] 10.222.0.82 () {58 vars in 1233 bytes} [Tue Aug 2 10:17:49 2022] DELETE /identity/v3/projects/b4a22005113e4ac9bcca471ab0f1d0a6 => generated 0 bytes in 307 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 02 10:17:50.011687 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d98a8cf-8a70-47c1-8856-92f1fd13ed8d service neutron] Authenticating user token {{(pid=80378) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:17:50.028031 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d98a8cf-8a70-47c1-8856-92f1fd13ed8d service neutron] Validating token access rules against request {{(pid=80378) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:17:50.127386 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3ee104c-6c3b-424e-9aba-7fd63e689693 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '891005fa5ff74041b75747d16cc66798', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7047912c5acd4adeba5d87c3bf895e39', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80378) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:17:50.128450 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3ee104c-6c3b-424e-9aba-7fd63e689693 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:17:50.128966 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3ee104c-6c3b-424e-9aba-7fd63e689693 admin admin] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:17:50.129991 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3ee104c-6c3b-424e-9aba-7fd63e689693 admin admin] PATH_INFO: `/v3/users/17057dba1f78414c940890ccb96b64e3` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:17:50.132196 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3ee104c-6c3b-424e-9aba-7fd63e689693 admin admin] RBAC: Authorizing `identity:delete_user(user_id=17057dba1f78414c940890ccb96b64e3)` {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:17:50.135214 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3ee104c-6c3b-424e-9aba-7fd63e689693 admin admin] RBAC: Authorization granted {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:17:50.227608 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.notifications [None req-f3ee104c-6c3b-424e-9aba-7fd63e689693 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '17057dba1f78414c940890ccb96b64e3'} {{(pid=80378) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 10:17:50.244626 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.notifications [None req-f3ee104c-6c3b-424e-9aba-7fd63e689693 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '17057dba1f78414c940890ccb96b64e3'} {{(pid=80378) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 10:17:50.284364 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.notifications [None req-f3ee104c-6c3b-424e-9aba-7fd63e689693 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '17057dba1f78414c940890ccb96b64e3'} {{(pid=80378) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 10:17:50.286449 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.notifications [None req-f3ee104c-6c3b-424e-9aba-7fd63e689693 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '17057dba1f78414c940890ccb96b64e3'} {{(pid=80378) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 10:17:50.354926 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.notifications [None req-f3ee104c-6c3b-424e-9aba-7fd63e689693 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '17057dba1f78414c940890ccb96b64e3'} {{(pid=80378) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 10:17:50.391544 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 1107/2224] 10.222.0.82 () {58 vars in 1224 bytes} [Tue Aug 2 10:17:50 2022] DELETE /identity/v3/users/17057dba1f78414c940890ccb96b64e3 => generated 0 bytes in 380 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 02 10:17:51.704897 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2f757e3-32fc-4694-a3ad-d0d7c532b802 admin admin] Authenticating user token {{(pid=80377) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:17:51.706534 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.fernet_utils [None req-f2f757e3-32fc-4694-a3ad-d0d7c532b802 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80377) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 10:17:51.740758 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2f757e3-32fc-4694-a3ad-d0d7c532b802 admin admin] Validating token access rules against request {{(pid=80377) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:17:51.909047 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fbcfc96-402b-42cb-a7ea-9f01c3bc8141 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '891005fa5ff74041b75747d16cc66798', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7047912c5acd4adeba5d87c3bf895e39', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80377) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:17:51.909833 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fbcfc96-402b-42cb-a7ea-9f01c3bc8141 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:17:51.910318 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fbcfc96-402b-42cb-a7ea-9f01c3bc8141 admin admin] SCRIPT_NAME: `/identity` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:17:51.910901 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fbcfc96-402b-42cb-a7ea-9f01c3bc8141 admin admin] PATH_INFO: `/v3/projects/8a6d0cf631f54fef94a079c69c4b6ca2` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:17:51.913714 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8fbcfc96-402b-42cb-a7ea-9f01c3bc8141 admin admin] RBAC: Authorizing `identity:delete_project(project_id=8a6d0cf631f54fef94a079c69c4b6ca2)` {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:17:51.916666 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8fbcfc96-402b-42cb-a7ea-9f01c3bc8141 admin admin] RBAC: Authorization granted {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:17:52.071688 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.notifications [None req-8fbcfc96-402b-42cb-a7ea-9f01c3bc8141 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '8a6d0cf631f54fef94a079c69c4b6ca2', 'request_id': 'req-8fbcfc96-402b-42cb-a7ea-9f01c3bc8141'} {{(pid=80377) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 10:17:52.094191 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.notifications [None req-8fbcfc96-402b-42cb-a7ea-9f01c3bc8141 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '8a6d0cf631f54fef94a079c69c4b6ca2', 'request_id': 'req-8fbcfc96-402b-42cb-a7ea-9f01c3bc8141'} {{(pid=80377) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 10:17:52.104323 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.notifications [None req-8fbcfc96-402b-42cb-a7ea-9f01c3bc8141 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '8a6d0cf631f54fef94a079c69c4b6ca2', 'request_id': 'req-8fbcfc96-402b-42cb-a7ea-9f01c3bc8141'} {{(pid=80377) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 10:17:52.128774 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.notifications [None req-8fbcfc96-402b-42cb-a7ea-9f01c3bc8141 admin admin] The token cache is being invalidate because project 8a6d0cf631f54fef94a079c69c4b6ca2 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80377) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 02 10:17:52.130365 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.notifications [None req-8fbcfc96-402b-42cb-a7ea-9f01c3bc8141 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80377) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 10:17:52.131277 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.notifications [None req-8fbcfc96-402b-42cb-a7ea-9f01c3bc8141 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80377) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 10:17:52.142450 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: [pid: 80377|app: 0|req: 1118/2225] 10.222.0.82 () {58 vars in 1233 bytes} [Tue Aug 2 10:17:51 2022] DELETE /identity/v3/projects/8a6d0cf631f54fef94a079c69c4b6ca2 => generated 0 bytes in 438 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 02 10:17:52.146625 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3ee104c-6c3b-424e-9aba-7fd63e689693 admin admin] Authenticating user token {{(pid=80378) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:17:52.149495 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.fernet_utils [None req-f3ee104c-6c3b-424e-9aba-7fd63e689693 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80378) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 10:17:52.183533 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3ee104c-6c3b-424e-9aba-7fd63e689693 admin admin] Validating token access rules against request {{(pid=80378) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:17:52.420716 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d279819e-c26c-4026-8cd7-1e73169781a0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '891005fa5ff74041b75747d16cc66798', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7047912c5acd4adeba5d87c3bf895e39', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80378) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:17:52.421636 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d279819e-c26c-4026-8cd7-1e73169781a0 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:17:52.422186 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d279819e-c26c-4026-8cd7-1e73169781a0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:17:52.422662 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d279819e-c26c-4026-8cd7-1e73169781a0 admin admin] PATH_INFO: `/v3/users/f3e913d3ba1a48f1a28ed62953d96af7` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:17:52.424315 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d279819e-c26c-4026-8cd7-1e73169781a0 admin admin] RBAC: Authorizing `identity:delete_user(user_id=f3e913d3ba1a48f1a28ed62953d96af7)` {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:17:52.431072 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d279819e-c26c-4026-8cd7-1e73169781a0 admin admin] RBAC: Authorization granted {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:17:52.474620 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG dbcounter [-] [80378] Writing DB stats keystone:SELECT=88,keystone:DELETE=11,keystone:INSERT=1 {{(pid=80378) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 10:17:52.560917 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.notifications [None req-d279819e-c26c-4026-8cd7-1e73169781a0 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'f3e913d3ba1a48f1a28ed62953d96af7'} {{(pid=80378) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 10:17:52.581306 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.notifications [None req-d279819e-c26c-4026-8cd7-1e73169781a0 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'f3e913d3ba1a48f1a28ed62953d96af7'} {{(pid=80378) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 10:17:52.594672 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.notifications [None req-d279819e-c26c-4026-8cd7-1e73169781a0 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'f3e913d3ba1a48f1a28ed62953d96af7'} {{(pid=80378) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 10:17:52.595685 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.notifications [None req-d279819e-c26c-4026-8cd7-1e73169781a0 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'f3e913d3ba1a48f1a28ed62953d96af7'} {{(pid=80378) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 10:17:52.634871 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.notifications [None req-d279819e-c26c-4026-8cd7-1e73169781a0 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'f3e913d3ba1a48f1a28ed62953d96af7'} {{(pid=80378) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 10:17:52.655551 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 1108/2226] 10.222.0.82 () {58 vars in 1224 bytes} [Tue Aug 2 10:17:52 2022] DELETE /identity/v3/users/f3e913d3ba1a48f1a28ed62953d96af7 => generated 0 bytes in 503 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 02 10:17:53.506814 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fbcfc96-402b-42cb-a7ea-9f01c3bc8141 admin admin] Authenticating user token {{(pid=80377) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:17:53.508462 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.fernet_utils [None req-8fbcfc96-402b-42cb-a7ea-9f01c3bc8141 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80377) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 10:17:53.532587 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fbcfc96-402b-42cb-a7ea-9f01c3bc8141 admin admin] Validating token access rules against request {{(pid=80377) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:17:53.698313 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG dbcounter [-] [80377] Writing DB stats keystone:SELECT=86,keystone:DELETE=12,keystone:INSERT=2 {{(pid=80377) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 10:17:53.731400 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc1490fb-7832-4f22-a98d-a1277196fc88 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '891005fa5ff74041b75747d16cc66798', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7047912c5acd4adeba5d87c3bf895e39', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80377) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:17:53.732134 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc1490fb-7832-4f22-a98d-a1277196fc88 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:17:53.732422 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc1490fb-7832-4f22-a98d-a1277196fc88 admin admin] SCRIPT_NAME: `/identity` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:17:53.732700 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc1490fb-7832-4f22-a98d-a1277196fc88 admin admin] PATH_INFO: `/v3/projects/33eb10ad74fe440da7f9bc1704b3b570` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:17:53.736014 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc1490fb-7832-4f22-a98d-a1277196fc88 admin admin] RBAC: Authorizing `identity:delete_project(project_id=33eb10ad74fe440da7f9bc1704b3b570)` {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:17:53.738024 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc1490fb-7832-4f22-a98d-a1277196fc88 admin admin] RBAC: Authorization granted {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:17:53.871269 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.notifications [None req-dc1490fb-7832-4f22-a98d-a1277196fc88 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '33eb10ad74fe440da7f9bc1704b3b570', 'request_id': 'req-dc1490fb-7832-4f22-a98d-a1277196fc88'} {{(pid=80377) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 10:17:53.882877 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.notifications [None req-dc1490fb-7832-4f22-a98d-a1277196fc88 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '33eb10ad74fe440da7f9bc1704b3b570', 'request_id': 'req-dc1490fb-7832-4f22-a98d-a1277196fc88'} {{(pid=80377) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 10:17:53.895841 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.notifications [None req-dc1490fb-7832-4f22-a98d-a1277196fc88 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '33eb10ad74fe440da7f9bc1704b3b570', 'request_id': 'req-dc1490fb-7832-4f22-a98d-a1277196fc88'} {{(pid=80377) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 10:17:53.924258 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.notifications [None req-dc1490fb-7832-4f22-a98d-a1277196fc88 admin admin] The token cache is being invalidate because project 33eb10ad74fe440da7f9bc1704b3b570 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80377) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 02 10:17:53.924588 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.notifications [None req-dc1490fb-7832-4f22-a98d-a1277196fc88 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80377) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 10:17:53.925232 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.notifications [None req-dc1490fb-7832-4f22-a98d-a1277196fc88 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80377) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 10:17:53.931445 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: [pid: 80377|app: 0|req: 1119/2227] 10.222.0.82 () {58 vars in 1233 bytes} [Tue Aug 2 10:17:53 2022] DELETE /identity/v3/projects/33eb10ad74fe440da7f9bc1704b3b570 => generated 0 bytes in 425 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 02 10:17:53.936723 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41ea32e4-8d89-4113-9a61-fc271c6c8c71 None None] REQUEST_METHOD: `POST` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:17:53.937120 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41ea32e4-8d89-4113-9a61-fc271c6c8c71 None None] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:17:53.937293 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41ea32e4-8d89-4113-9a61-fc271c6c8c71 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:17:53.977522 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.auth.core [None req-41ea32e4-8d89-4113-9a61-fc271c6c8c71 None None] MFA Rules not processed for user `891005fa5ff74041b75747d16cc66798`. Rule list: `[]` (Enabled: `True`). {{(pid=80378) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 02 10:17:53.979516 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.fernet_utils [None req-41ea32e4-8d89-4113-9a61-fc271c6c8c71 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80378) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 10:17:54.087239 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 1109/2228] 10.222.0.82 () {58 vars in 958 bytes} [Tue Aug 2 10:17:53 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 152 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 02 10:17:54.096040 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7591fa09-6a01-44c9-8951-6fc69e5b17f1 None None] REQUEST_METHOD: `POST` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:17:54.096345 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7591fa09-6a01-44c9-8951-6fc69e5b17f1 None None] SCRIPT_NAME: `/identity` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:17:54.096598 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7591fa09-6a01-44c9-8951-6fc69e5b17f1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:17:54.149421 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.auth.core [None req-7591fa09-6a01-44c9-8951-6fc69e5b17f1 None None] MFA Rules not processed for user `891005fa5ff74041b75747d16cc66798`. Rule list: `[]` (Enabled: `True`). {{(pid=80377) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 02 10:17:54.154528 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.fernet_utils [None req-7591fa09-6a01-44c9-8951-6fc69e5b17f1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80377) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 10:17:54.181955 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: [pid: 80377|app: 0|req: 1120/2229] 10.222.0.82 () {58 vars in 958 bytes} [Tue Aug 2 10:17:54 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 02 10:17:54.186700 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41ea32e4-8d89-4113-9a61-fc271c6c8c71 None None] Authenticating user token {{(pid=80378) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:17:54.202910 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41ea32e4-8d89-4113-9a61-fc271c6c8c71 None None] Validating token access rules against request {{(pid=80378) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:17:54.215455 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3c24f58-32cb-43dd-b211-4e99819cefc8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '891005fa5ff74041b75747d16cc66798', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7047912c5acd4adeba5d87c3bf895e39', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80378) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:17:54.228758 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3c24f58-32cb-43dd-b211-4e99819cefc8 admin admin] REQUEST_METHOD: `GET` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:17:54.229111 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3c24f58-32cb-43dd-b211-4e99819cefc8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:17:54.229397 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3c24f58-32cb-43dd-b211-4e99819cefc8 admin admin] PATH_INFO: `/v3/domains` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:17:54.230252 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3c24f58-32cb-43dd-b211-4e99819cefc8 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80378) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 02 10:17:54.230955 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3c24f58-32cb-43dd-b211-4e99819cefc8 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:17:54.235124 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3c24f58-32cb-43dd-b211-4e99819cefc8 admin admin] RBAC: Authorization granted {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:17:54.273751 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 1110/2230] 10.222.0.82 () {58 vars in 1153 bytes} [Tue Aug 2 10:17:54 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 85 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 02 10:17:54.277317 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7591fa09-6a01-44c9-8951-6fc69e5b17f1 None None] Authenticating user token {{(pid=80377) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:17:54.299684 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7591fa09-6a01-44c9-8951-6fc69e5b17f1 None None] Validating token access rules against request {{(pid=80377) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:17:54.314417 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51f329bd-bc45-4ec7-aa09-d1d8e57f9ae2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '891005fa5ff74041b75747d16cc66798', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7047912c5acd4adeba5d87c3bf895e39', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80377) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:17:54.315425 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51f329bd-bc45-4ec7-aa09-d1d8e57f9ae2 admin admin] REQUEST_METHOD: `POST` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:17:54.315932 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51f329bd-bc45-4ec7-aa09-d1d8e57f9ae2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:17:54.316422 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51f329bd-bc45-4ec7-aa09-d1d8e57f9ae2 admin admin] PATH_INFO: `/v3/projects` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:17:54.317325 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51f329bd-bc45-4ec7-aa09-d1d8e57f9ae2 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:17:54.320408 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51f329bd-bc45-4ec7-aa09-d1d8e57f9ae2 admin admin] RBAC: Authorization granted {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:17:54.383319 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: [pid: 80377|app: 0|req: 1121/2231] 10.222.0.82 () {60 vars in 1153 bytes} [Tue Aug 2 10:17:54 2022] POST /identity/v3/projects => generated 381 bytes in 106 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 02 10:17:54.388288 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3c24f58-32cb-43dd-b211-4e99819cefc8 admin admin] Authenticating user token {{(pid=80378) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:17:54.410021 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3c24f58-32cb-43dd-b211-4e99819cefc8 admin admin] Validating token access rules against request {{(pid=80378) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:17:54.545609 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9320fc2e-ba1e-4b41-b2ce-f0ebd7b70c7e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '891005fa5ff74041b75747d16cc66798', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7047912c5acd4adeba5d87c3bf895e39', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80378) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:17:54.546781 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9320fc2e-ba1e-4b41-b2ce-f0ebd7b70c7e admin admin] REQUEST_METHOD: `POST` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:17:54.547070 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9320fc2e-ba1e-4b41-b2ce-f0ebd7b70c7e admin admin] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:17:54.547922 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9320fc2e-ba1e-4b41-b2ce-f0ebd7b70c7e admin admin] PATH_INFO: `/v3/users` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:17:54.548621 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9320fc2e-ba1e-4b41-b2ce-f0ebd7b70c7e admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:17:54.551848 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9320fc2e-ba1e-4b41-b2ce-f0ebd7b70c7e admin admin] RBAC: Authorization granted {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:17:54.598187 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 1111/2232] 10.222.0.82 () {60 vars in 1144 bytes} [Tue Aug 2 10:17:54 2022] POST /identity/v3/users => generated 291 bytes in 210 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 02 10:17:54.598988 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51f329bd-bc45-4ec7-aa09-d1d8e57f9ae2 admin admin] Authenticating user token {{(pid=80377) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:17:54.616416 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51f329bd-bc45-4ec7-aa09-d1d8e57f9ae2 admin admin] Validating token access rules against request {{(pid=80377) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:17:54.631097 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82417ca9-dd53-4c7e-a6ab-1e073eb46368 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '891005fa5ff74041b75747d16cc66798', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7047912c5acd4adeba5d87c3bf895e39', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80377) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:17:54.632370 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82417ca9-dd53-4c7e-a6ab-1e073eb46368 admin admin] REQUEST_METHOD: `GET` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:17:54.632881 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82417ca9-dd53-4c7e-a6ab-1e073eb46368 admin admin] SCRIPT_NAME: `/identity` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:17:54.633951 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82417ca9-dd53-4c7e-a6ab-1e073eb46368 admin admin] PATH_INFO: `/v3/roles` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:17:54.636000 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82417ca9-dd53-4c7e-a6ab-1e073eb46368 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:17:54.640157 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82417ca9-dd53-4c7e-a6ab-1e073eb46368 admin admin] RBAC: Authorization granted {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:17:54.672284 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: [pid: 80377|app: 0|req: 1122/2233] 10.222.0.82 () {58 vars in 1122 bytes} [Tue Aug 2 10:17:54 2022] GET /identity/v3/roles => generated 1420 bytes in 74 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 02 10:17:54.676327 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9320fc2e-ba1e-4b41-b2ce-f0ebd7b70c7e admin admin] Authenticating user token {{(pid=80378) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:17:54.699490 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9320fc2e-ba1e-4b41-b2ce-f0ebd7b70c7e admin admin] Validating token access rules against request {{(pid=80378) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:17:54.726356 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2529dfe1-37b5-4cee-80ab-6ba5f0d9e92c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '891005fa5ff74041b75747d16cc66798', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7047912c5acd4adeba5d87c3bf895e39', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80378) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:17:54.731696 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2529dfe1-37b5-4cee-80ab-6ba5f0d9e92c admin admin] REQUEST_METHOD: `GET` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:17:54.731696 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2529dfe1-37b5-4cee-80ab-6ba5f0d9e92c admin admin] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:17:54.731696 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2529dfe1-37b5-4cee-80ab-6ba5f0d9e92c admin admin] PATH_INFO: `/v3/roles` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:17:54.732470 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2529dfe1-37b5-4cee-80ab-6ba5f0d9e92c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:17:54.740981 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2529dfe1-37b5-4cee-80ab-6ba5f0d9e92c admin admin] RBAC: Authorization granted {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:17:54.772883 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG dbcounter [-] [80378] Writing DB stats keystone:DELETE=9,keystone:SELECT=87,keystone:INSERT=4 {{(pid=80378) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 10:17:54.790095 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82417ca9-dd53-4c7e-a6ab-1e073eb46368 admin admin] Authenticating user token {{(pid=80377) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:17:54.798888 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 1112/2234] 10.222.0.82 () {58 vars in 1122 bytes} [Tue Aug 2 10:17:54 2022] GET /identity/v3/roles => generated 1420 bytes in 123 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 02 10:17:54.814556 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82417ca9-dd53-4c7e-a6ab-1e073eb46368 admin admin] Validating token access rules against request {{(pid=80377) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:17:54.854825 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f66337a-3058-479b-9bb8-897bfd935304 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '891005fa5ff74041b75747d16cc66798', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7047912c5acd4adeba5d87c3bf895e39', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80377) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:17:54.855799 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f66337a-3058-479b-9bb8-897bfd935304 admin admin] REQUEST_METHOD: `PUT` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:17:54.856875 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f66337a-3058-479b-9bb8-897bfd935304 admin admin] SCRIPT_NAME: `/identity` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:17:54.857515 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f66337a-3058-479b-9bb8-897bfd935304 admin admin] PATH_INFO: `/v3/projects/8f69565b334545f2903d55518686abae/users/ffe0cbc6a5f34d779966d4cee544d428/roles/e1c04b19fc4543b8b54f58bbcffbc598` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:17:54.915006 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f66337a-3058-479b-9bb8-897bfd935304 admin admin] RBAC: Authorizing `identity:create_grant(project_id=8f69565b334545f2903d55518686abae, user_id=ffe0cbc6a5f34d779966d4cee544d428, role_id=e1c04b19fc4543b8b54f58bbcffbc598)` {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:17:54.926349 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f66337a-3058-479b-9bb8-897bfd935304 admin admin] RBAC: Authorization granted {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:17:54.978030 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: [pid: 80377|app: 0|req: 1123/2235] 10.222.0.82 () {60 vars in 1483 bytes} [Tue Aug 2 10:17:54 2022] PUT /identity/v3/projects/8f69565b334545f2903d55518686abae/users/ffe0cbc6a5f34d779966d4cee544d428/roles/e1c04b19fc4543b8b54f58bbcffbc598 => generated 0 bytes in 189 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 02 10:17:54.988465 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-836ef286-96cc-4f6b-b5ae-6b7b1d27c6af None None] REQUEST_METHOD: `POST` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:17:54.989371 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-836ef286-96cc-4f6b-b5ae-6b7b1d27c6af None None] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:17:54.989658 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-836ef286-96cc-4f6b-b5ae-6b7b1d27c6af None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:17:55.050343 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.auth.core [None req-836ef286-96cc-4f6b-b5ae-6b7b1d27c6af None None] MFA Rules not processed for user `ffe0cbc6a5f34d779966d4cee544d428`. Rule list: `[]` (Enabled: `True`). {{(pid=80378) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 02 10:17:55.057933 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.fernet_utils [None req-836ef286-96cc-4f6b-b5ae-6b7b1d27c6af None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80378) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 10:17:55.248724 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 1113/2236] 10.222.0.82 () {58 vars in 958 bytes} [Tue Aug 2 10:17:54 2022] POST /identity/v3/auth/tokens => generated 3258 bytes in 261 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 02 10:17:55.253001 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f66337a-3058-479b-9bb8-897bfd935304 admin admin] Authenticating user token {{(pid=80377) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:17:55.414072 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f66337a-3058-479b-9bb8-897bfd935304 admin admin] Validating token access rules against request {{(pid=80377) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:17:55.426013 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dee03538-576b-4e7a-9a6d-44466ee9e21b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '891005fa5ff74041b75747d16cc66798', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7047912c5acd4adeba5d87c3bf895e39', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80377) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:17:55.426678 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dee03538-576b-4e7a-9a6d-44466ee9e21b admin admin] REQUEST_METHOD: `POST` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:17:55.426966 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dee03538-576b-4e7a-9a6d-44466ee9e21b admin admin] SCRIPT_NAME: `/identity` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:17:55.427634 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dee03538-576b-4e7a-9a6d-44466ee9e21b admin admin] PATH_INFO: `/v3/projects` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:17:55.428331 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dee03538-576b-4e7a-9a6d-44466ee9e21b admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:17:55.433173 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dee03538-576b-4e7a-9a6d-44466ee9e21b admin admin] RBAC: Authorization granted {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:17:55.483290 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG dbcounter [-] [80377] Writing DB stats keystone:SELECT=89,keystone:DELETE=7,keystone:INSERT=4 {{(pid=80377) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 10:17:55.496261 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: [pid: 80377|app: 0|req: 1124/2237] 10.222.0.82 () {60 vars in 1153 bytes} [Tue Aug 2 10:17:55 2022] POST /identity/v3/projects => generated 385 bytes in 244 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 02 10:17:55.504062 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-836ef286-96cc-4f6b-b5ae-6b7b1d27c6af None None] Authenticating user token {{(pid=80378) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:17:55.522171 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-836ef286-96cc-4f6b-b5ae-6b7b1d27c6af None None] Validating token access rules against request {{(pid=80378) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:17:55.648092 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9767eec1-a344-437c-93d6-8e3b951f3419 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '891005fa5ff74041b75747d16cc66798', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7047912c5acd4adeba5d87c3bf895e39', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80378) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:17:55.649253 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9767eec1-a344-437c-93d6-8e3b951f3419 admin admin] REQUEST_METHOD: `POST` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:17:55.649761 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9767eec1-a344-437c-93d6-8e3b951f3419 admin admin] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:17:55.650285 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9767eec1-a344-437c-93d6-8e3b951f3419 admin admin] PATH_INFO: `/v3/users` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:17:55.659413 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9767eec1-a344-437c-93d6-8e3b951f3419 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:17:55.662853 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9767eec1-a344-437c-93d6-8e3b951f3419 admin admin] RBAC: Authorization granted {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:17:55.712148 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 1114/2238] 10.222.0.82 () {60 vars in 1144 bytes} [Tue Aug 2 10:17:55 2022] POST /identity/v3/users => generated 293 bytes in 206 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 02 10:17:55.713813 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dee03538-576b-4e7a-9a6d-44466ee9e21b admin admin] Authenticating user token {{(pid=80377) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:17:55.730418 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dee03538-576b-4e7a-9a6d-44466ee9e21b admin admin] Validating token access rules against request {{(pid=80377) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:17:55.744607 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-838a052c-4472-4394-b3ea-b682b636f1c5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '891005fa5ff74041b75747d16cc66798', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7047912c5acd4adeba5d87c3bf895e39', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80377) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:17:55.746030 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-838a052c-4472-4394-b3ea-b682b636f1c5 admin admin] REQUEST_METHOD: `GET` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:17:55.747132 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-838a052c-4472-4394-b3ea-b682b636f1c5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:17:55.749054 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-838a052c-4472-4394-b3ea-b682b636f1c5 admin admin] PATH_INFO: `/v3/roles` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:17:55.751969 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-838a052c-4472-4394-b3ea-b682b636f1c5 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:17:55.755764 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-838a052c-4472-4394-b3ea-b682b636f1c5 admin admin] RBAC: Authorization granted {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:17:55.787430 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: [pid: 80377|app: 0|req: 1125/2239] 10.222.0.82 () {58 vars in 1122 bytes} [Tue Aug 2 10:17:55 2022] GET /identity/v3/roles => generated 1420 bytes in 70 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 02 10:17:55.787881 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9767eec1-a344-437c-93d6-8e3b951f3419 admin admin] Authenticating user token {{(pid=80378) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:17:55.801993 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9767eec1-a344-437c-93d6-8e3b951f3419 admin admin] Validating token access rules against request {{(pid=80378) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:17:55.815077 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13900253-4205-4c76-89e2-5e529472804e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '891005fa5ff74041b75747d16cc66798', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7047912c5acd4adeba5d87c3bf895e39', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80378) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:17:55.816260 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13900253-4205-4c76-89e2-5e529472804e admin admin] REQUEST_METHOD: `GET` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:17:55.817149 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13900253-4205-4c76-89e2-5e529472804e admin admin] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:17:55.817773 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13900253-4205-4c76-89e2-5e529472804e admin admin] PATH_INFO: `/v3/roles` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:17:55.819495 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13900253-4205-4c76-89e2-5e529472804e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:17:55.823434 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13900253-4205-4c76-89e2-5e529472804e admin admin] RBAC: Authorization granted {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:17:55.852066 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 1115/2240] 10.222.0.82 () {58 vars in 1122 bytes} [Tue Aug 2 10:17:55 2022] GET /identity/v3/roles => generated 1420 bytes in 61 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 02 10:17:55.852437 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-838a052c-4472-4394-b3ea-b682b636f1c5 admin admin] Authenticating user token {{(pid=80377) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:17:55.866673 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-838a052c-4472-4394-b3ea-b682b636f1c5 admin admin] Validating token access rules against request {{(pid=80377) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:17:55.881506 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37e89e5b-36ff-45d7-a185-ef7dadfd32e5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '891005fa5ff74041b75747d16cc66798', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7047912c5acd4adeba5d87c3bf895e39', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80377) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:17:55.882506 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37e89e5b-36ff-45d7-a185-ef7dadfd32e5 admin admin] REQUEST_METHOD: `PUT` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:17:55.884100 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37e89e5b-36ff-45d7-a185-ef7dadfd32e5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:17:55.884590 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37e89e5b-36ff-45d7-a185-ef7dadfd32e5 admin admin] PATH_INFO: `/v3/projects/38d139e43bae4406b94f11047bc998a4/users/2ac6c9dcc5cb4be981026eb113525afb/roles/e1c04b19fc4543b8b54f58bbcffbc598` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:17:55.924788 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37e89e5b-36ff-45d7-a185-ef7dadfd32e5 admin admin] RBAC: Authorizing `identity:create_grant(project_id=38d139e43bae4406b94f11047bc998a4, user_id=2ac6c9dcc5cb4be981026eb113525afb, role_id=e1c04b19fc4543b8b54f58bbcffbc598)` {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:17:55.930724 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37e89e5b-36ff-45d7-a185-ef7dadfd32e5 admin admin] RBAC: Authorization granted {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:17:55.963069 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: [pid: 80377|app: 0|req: 1126/2241] 10.222.0.82 () {60 vars in 1483 bytes} [Tue Aug 2 10:17:55 2022] PUT /identity/v3/projects/38d139e43bae4406b94f11047bc998a4/users/2ac6c9dcc5cb4be981026eb113525afb/roles/e1c04b19fc4543b8b54f58bbcffbc598 => generated 0 bytes in 111 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 02 10:17:55.973513 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67f72028-a908-4314-bcf4-1443ec975ca5 None None] REQUEST_METHOD: `POST` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:17:55.974541 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67f72028-a908-4314-bcf4-1443ec975ca5 None None] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:17:55.974541 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67f72028-a908-4314-bcf4-1443ec975ca5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:17:56.061827 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.auth.core [None req-67f72028-a908-4314-bcf4-1443ec975ca5 None None] MFA Rules not processed for user `2ac6c9dcc5cb4be981026eb113525afb`. Rule list: `[]` (Enabled: `True`). {{(pid=80378) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 02 10:17:56.065239 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.fernet_utils [None req-67f72028-a908-4314-bcf4-1443ec975ca5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80378) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 10:17:56.256133 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37e89e5b-36ff-45d7-a185-ef7dadfd32e5 admin admin] Authenticating user token {{(pid=80377) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:17:56.258149 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.fernet_utils [None req-37e89e5b-36ff-45d7-a185-ef7dadfd32e5 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80377) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 10:17:56.275714 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG dbcounter [-] [80378] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80378) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 10:17:56.302372 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 1116/2242] 10.222.0.82 () {58 vars in 958 bytes} [Tue Aug 2 10:17:55 2022] POST /identity/v3/auth/tokens => generated 3262 bytes in 330 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 02 10:17:56.307628 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67f72028-a908-4314-bcf4-1443ec975ca5 None None] Authenticating user token {{(pid=80378) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:17:56.386863 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37e89e5b-36ff-45d7-a185-ef7dadfd32e5 admin admin] Validating token access rules against request {{(pid=80377) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:17:56.398149 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-407f4e40-b272-4942-845c-e81588920740 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '891005fa5ff74041b75747d16cc66798', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7047912c5acd4adeba5d87c3bf895e39', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80377) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:17:56.399174 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-407f4e40-b272-4942-845c-e81588920740 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:17:56.399670 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-407f4e40-b272-4942-845c-e81588920740 admin admin] SCRIPT_NAME: `/identity` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:17:56.400166 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-407f4e40-b272-4942-845c-e81588920740 admin admin] PATH_INFO: `/v3/users/f690a9da98b548e9aee69426281ae428` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:17:56.404864 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-407f4e40-b272-4942-845c-e81588920740 admin admin] RBAC: Authorizing `identity:delete_user(user_id=f690a9da98b548e9aee69426281ae428)` {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:17:56.408036 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-407f4e40-b272-4942-845c-e81588920740 admin admin] RBAC: Authorization granted {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:17:56.491310 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67f72028-a908-4314-bcf4-1443ec975ca5 None None] Validating token access rules against request {{(pid=80378) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:17:56.506631 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19566dce-a33f-4693-b75c-5464db7bad02 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '891005fa5ff74041b75747d16cc66798', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7047912c5acd4adeba5d87c3bf895e39', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80378) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:17:56.507279 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19566dce-a33f-4693-b75c-5464db7bad02 admin admin] REQUEST_METHOD: `POST` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:17:56.507568 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19566dce-a33f-4693-b75c-5464db7bad02 admin admin] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:17:56.507847 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19566dce-a33f-4693-b75c-5464db7bad02 admin admin] PATH_INFO: `/v3/projects` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:17:56.511178 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19566dce-a33f-4693-b75c-5464db7bad02 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:17:56.513218 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19566dce-a33f-4693-b75c-5464db7bad02 admin admin] RBAC: Authorization granted {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:17:56.520827 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.notifications [None req-407f4e40-b272-4942-845c-e81588920740 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'f690a9da98b548e9aee69426281ae428'} {{(pid=80377) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 10:17:56.541843 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.notifications [None req-407f4e40-b272-4942-845c-e81588920740 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'f690a9da98b548e9aee69426281ae428'} {{(pid=80377) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 10:17:56.552766 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.notifications [None req-407f4e40-b272-4942-845c-e81588920740 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'f690a9da98b548e9aee69426281ae428'} {{(pid=80377) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 10:17:56.553520 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.notifications [None req-407f4e40-b272-4942-845c-e81588920740 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'f690a9da98b548e9aee69426281ae428'} {{(pid=80377) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 10:17:56.590183 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 1117/2243] 10.222.0.82 () {60 vars in 1153 bytes} [Tue Aug 2 10:17:56 2022] POST /identity/v3/projects => generated 385 bytes in 282 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 02 10:17:56.591225 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19566dce-a33f-4693-b75c-5464db7bad02 admin admin] Authenticating user token {{(pid=80378) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:17:56.593823 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.fernet_utils [None req-19566dce-a33f-4693-b75c-5464db7bad02 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80378) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 10:17:56.616872 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.notifications [None req-407f4e40-b272-4942-845c-e81588920740 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'f690a9da98b548e9aee69426281ae428'} {{(pid=80377) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 10:17:56.633323 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: [pid: 80377|app: 0|req: 1127/2244] 10.222.0.82 () {58 vars in 1224 bytes} [Tue Aug 2 10:17:56 2022] DELETE /identity/v3/users/f690a9da98b548e9aee69426281ae428 => generated 0 bytes in 374 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 02 10:17:56.730734 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19566dce-a33f-4693-b75c-5464db7bad02 admin admin] Validating token access rules against request {{(pid=80378) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:17:56.819921 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5f1ea31-b700-410a-968c-8593f90aa6d6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '891005fa5ff74041b75747d16cc66798', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7047912c5acd4adeba5d87c3bf895e39', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80378) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:17:56.820921 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5f1ea31-b700-410a-968c-8593f90aa6d6 admin admin] REQUEST_METHOD: `POST` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:17:56.821371 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5f1ea31-b700-410a-968c-8593f90aa6d6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:17:56.821871 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5f1ea31-b700-410a-968c-8593f90aa6d6 admin admin] PATH_INFO: `/v3/users` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:17:56.823250 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5f1ea31-b700-410a-968c-8593f90aa6d6 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:17:56.827131 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5f1ea31-b700-410a-968c-8593f90aa6d6 admin admin] RBAC: Authorization granted {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:17:56.885742 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 1118/2245] 10.222.0.82 () {60 vars in 1144 bytes} [Tue Aug 2 10:17:56 2022] POST /identity/v3/users => generated 299 bytes in 295 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 02 10:17:56.889630 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-407f4e40-b272-4942-845c-e81588920740 admin admin] Authenticating user token {{(pid=80377) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:17:56.922663 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-407f4e40-b272-4942-845c-e81588920740 admin admin] Validating token access rules against request {{(pid=80377) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:17:56.935054 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d9920f6-1f28-4346-9efe-bfea0548045f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '891005fa5ff74041b75747d16cc66798', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7047912c5acd4adeba5d87c3bf895e39', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80377) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:17:56.940979 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d9920f6-1f28-4346-9efe-bfea0548045f admin admin] REQUEST_METHOD: `GET` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:17:56.941349 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d9920f6-1f28-4346-9efe-bfea0548045f admin admin] SCRIPT_NAME: `/identity` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:17:56.941634 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d9920f6-1f28-4346-9efe-bfea0548045f admin admin] PATH_INFO: `/v3/roles` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:17:56.944242 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d9920f6-1f28-4346-9efe-bfea0548045f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:17:56.946043 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d9920f6-1f28-4346-9efe-bfea0548045f admin admin] RBAC: Authorization granted {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:17:56.969016 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: [pid: 80377|app: 0|req: 1128/2246] 10.222.0.82 () {58 vars in 1122 bytes} [Tue Aug 2 10:17:56 2022] GET /identity/v3/roles => generated 1420 bytes in 80 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 02 10:17:56.972702 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5f1ea31-b700-410a-968c-8593f90aa6d6 admin admin] Authenticating user token {{(pid=80378) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:17:57.003257 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5f1ea31-b700-410a-968c-8593f90aa6d6 admin admin] Validating token access rules against request {{(pid=80378) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:17:57.017135 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-869f4176-1fde-4cfb-8b37-632834eabcab admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '891005fa5ff74041b75747d16cc66798', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7047912c5acd4adeba5d87c3bf895e39', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80378) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:17:57.019314 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-869f4176-1fde-4cfb-8b37-632834eabcab admin admin] REQUEST_METHOD: `PUT` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:17:57.019627 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-869f4176-1fde-4cfb-8b37-632834eabcab admin admin] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:17:57.020129 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-869f4176-1fde-4cfb-8b37-632834eabcab admin admin] PATH_INFO: `/v3/projects/8c47a826f47544a5977034bf3067629f/users/870771ad4e3f4151a1ba07b305968320/roles/237966442f05411191105bf336bc14f3` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:17:57.094923 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-869f4176-1fde-4cfb-8b37-632834eabcab admin admin] RBAC: Authorizing `identity:create_grant(project_id=8c47a826f47544a5977034bf3067629f, user_id=870771ad4e3f4151a1ba07b305968320, role_id=237966442f05411191105bf336bc14f3)` {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:17:57.098478 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-869f4176-1fde-4cfb-8b37-632834eabcab admin admin] RBAC: Authorization granted {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:17:57.144680 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG dbcounter [-] [80378] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80378) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 10:17:57.156879 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 1119/2247] 10.222.0.82 () {60 vars in 1483 bytes} [Tue Aug 2 10:17:56 2022] PUT /identity/v3/projects/8c47a826f47544a5977034bf3067629f/users/870771ad4e3f4151a1ba07b305968320/roles/237966442f05411191105bf336bc14f3 => generated 0 bytes in 185 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 02 10:17:57.178281 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d9920f6-1f28-4346-9efe-bfea0548045f admin admin] Authenticating user token {{(pid=80377) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:17:57.218140 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d9920f6-1f28-4346-9efe-bfea0548045f admin admin] Validating token access rules against request {{(pid=80377) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:17:57.259203 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-869f4176-1fde-4cfb-8b37-632834eabcab admin admin] Authenticating user token {{(pid=80378) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:17:57.301625 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.fernet_utils [None req-869f4176-1fde-4cfb-8b37-632834eabcab admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80378) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 10:17:57.419290 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG dbcounter [-] [80377] Writing DB stats keystone:SELECT=88,keystone:INSERT=2,keystone:DELETE=10 {{(pid=80377) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 10:17:57.456699 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-869f4176-1fde-4cfb-8b37-632834eabcab admin admin] Validating token access rules against request {{(pid=80378) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:17:57.464891 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d81a2be7-e839-4289-bdc1-7fe75a81dc3d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '891005fa5ff74041b75747d16cc66798', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7047912c5acd4adeba5d87c3bf895e39', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80378) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:17:57.466127 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d81a2be7-e839-4289-bdc1-7fe75a81dc3d admin admin] REQUEST_METHOD: `DELETE` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:17:57.466334 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d81a2be7-e839-4289-bdc1-7fe75a81dc3d admin admin] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:17:57.467234 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d81a2be7-e839-4289-bdc1-7fe75a81dc3d admin admin] PATH_INFO: `/v3/projects/088b96c6fcf64d899fbbe739c682de7d` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:17:57.469534 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d81a2be7-e839-4289-bdc1-7fe75a81dc3d admin admin] RBAC: Authorizing `identity:delete_project(project_id=088b96c6fcf64d899fbbe739c682de7d)` {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:17:57.472059 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d81a2be7-e839-4289-bdc1-7fe75a81dc3d admin admin] RBAC: Authorization granted {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:17:57.481473 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd35170f-256a-4dcc-a630-96a22aa84799 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '891005fa5ff74041b75747d16cc66798', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7047912c5acd4adeba5d87c3bf895e39', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80377) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:17:57.482389 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd35170f-256a-4dcc-a630-96a22aa84799 admin admin] REQUEST_METHOD: `GET` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:17:57.483250 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd35170f-256a-4dcc-a630-96a22aa84799 admin admin] SCRIPT_NAME: `/identity` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:17:57.483690 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd35170f-256a-4dcc-a630-96a22aa84799 admin admin] PATH_INFO: `/v3/roles` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:17:57.484553 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd35170f-256a-4dcc-a630-96a22aa84799 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:17:57.486370 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd35170f-256a-4dcc-a630-96a22aa84799 admin admin] RBAC: Authorization granted {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:17:57.509177 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: [pid: 80377|app: 0|req: 1129/2248] 10.222.0.82 () {58 vars in 1122 bytes} [Tue Aug 2 10:17:57 2022] GET /identity/v3/roles => generated 1420 bytes in 331 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 02 10:17:57.510907 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d8e0a5a-a9b0-43a1-a2d1-98959d8ba592 None None] REQUEST_METHOD: `POST` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:17:57.511451 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d8e0a5a-a9b0-43a1-a2d1-98959d8ba592 None None] SCRIPT_NAME: `/identity` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:17:57.511967 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d8e0a5a-a9b0-43a1-a2d1-98959d8ba592 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:17:57.552078 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.auth.core [None req-4d8e0a5a-a9b0-43a1-a2d1-98959d8ba592 None None] MFA Rules not processed for user `3d378bdfcbc5436f81a0e3ad04822ebe`. Rule list: `[]` (Enabled: `True`). {{(pid=80377) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 02 10:17:57.553302 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.fernet_utils [None req-4d8e0a5a-a9b0-43a1-a2d1-98959d8ba592 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80377) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 10:17:57.569829 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.notifications [None req-d81a2be7-e839-4289-bdc1-7fe75a81dc3d admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '088b96c6fcf64d899fbbe739c682de7d', 'request_id': 'req-d81a2be7-e839-4289-bdc1-7fe75a81dc3d'} {{(pid=80378) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 10:17:57.593494 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.notifications [None req-d81a2be7-e839-4289-bdc1-7fe75a81dc3d admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '088b96c6fcf64d899fbbe739c682de7d', 'request_id': 'req-d81a2be7-e839-4289-bdc1-7fe75a81dc3d'} {{(pid=80378) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 10:17:57.607511 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.notifications [None req-d81a2be7-e839-4289-bdc1-7fe75a81dc3d admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '088b96c6fcf64d899fbbe739c682de7d', 'request_id': 'req-d81a2be7-e839-4289-bdc1-7fe75a81dc3d'} {{(pid=80378) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 10:17:57.633600 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.notifications [None req-d81a2be7-e839-4289-bdc1-7fe75a81dc3d admin admin] The token cache is being invalidate because project 088b96c6fcf64d899fbbe739c682de7d was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80378) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 02 10:17:57.633975 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.notifications [None req-d81a2be7-e839-4289-bdc1-7fe75a81dc3d admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80378) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 10:17:57.634654 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.notifications [None req-d81a2be7-e839-4289-bdc1-7fe75a81dc3d admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80378) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 10:17:57.636341 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 1120/2249] 10.222.0.82 () {58 vars in 1233 bytes} [Tue Aug 2 10:17:57 2022] DELETE /identity/v3/projects/088b96c6fcf64d899fbbe739c682de7d => generated 0 bytes in 378 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 02 10:17:57.638135 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d81a2be7-e839-4289-bdc1-7fe75a81dc3d admin admin] Authenticating user token {{(pid=80378) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:17:57.643097 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.fernet_utils [None req-d81a2be7-e839-4289-bdc1-7fe75a81dc3d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80378) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 10:17:57.753555 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: [pid: 80377|app: 0|req: 1130/2250] 10.222.0.82 () {58 vars in 958 bytes} [Tue Aug 2 10:17:57 2022] POST /identity/v3/auth/tokens => generated 3315 bytes in 244 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 02 10:17:57.762070 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06b2b2f0-d6a0-4af8-a57b-10eb1485f783 None None] REQUEST_METHOD: `POST` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:17:57.762658 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06b2b2f0-d6a0-4af8-a57b-10eb1485f783 None None] SCRIPT_NAME: `/identity` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:17:57.763209 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06b2b2f0-d6a0-4af8-a57b-10eb1485f783 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:17:57.784841 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d81a2be7-e839-4289-bdc1-7fe75a81dc3d admin admin] Validating token access rules against request {{(pid=80378) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:17:57.798826 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.auth.core [None req-06b2b2f0-d6a0-4af8-a57b-10eb1485f783 None None] MFA Rules not processed for user `891005fa5ff74041b75747d16cc66798`. Rule list: `[]` (Enabled: `True`). {{(pid=80377) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 02 10:17:57.800518 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.fernet_utils [None req-06b2b2f0-d6a0-4af8-a57b-10eb1485f783 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80377) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 10:17:57.818219 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b53a589-3959-4a80-9d9b-72a134ed74e0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '891005fa5ff74041b75747d16cc66798', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7047912c5acd4adeba5d87c3bf895e39', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80378) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:17:57.818219 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b53a589-3959-4a80-9d9b-72a134ed74e0 admin admin] REQUEST_METHOD: `PUT` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:17:57.818219 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b53a589-3959-4a80-9d9b-72a134ed74e0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:17:57.818219 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b53a589-3959-4a80-9d9b-72a134ed74e0 admin admin] PATH_INFO: `/v3/projects/8c47a826f47544a5977034bf3067629f/users/870771ad4e3f4151a1ba07b305968320/roles/237966442f05411191105bf336bc14f3` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:17:57.819925 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: [pid: 80377|app: 0|req: 1131/2251] 10.222.0.82 () {58 vars in 958 bytes} [Tue Aug 2 10:17:57 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 61 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 02 10:17:57.821085 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06b2b2f0-d6a0-4af8-a57b-10eb1485f783 None None] Authenticating user token {{(pid=80377) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:17:57.822765 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b53a589-3959-4a80-9d9b-72a134ed74e0 admin admin] RBAC: Authorizing `identity:create_grant(project_id=8c47a826f47544a5977034bf3067629f, user_id=870771ad4e3f4151a1ba07b305968320, role_id=237966442f05411191105bf336bc14f3)` {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:17:57.823203 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.fernet_utils [None req-06b2b2f0-d6a0-4af8-a57b-10eb1485f783 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80377) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 10:17:57.826504 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b53a589-3959-4a80-9d9b-72a134ed74e0 admin admin] RBAC: Authorization granted {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:17:57.851239 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 1121/2252] 10.222.0.82 () {60 vars in 1483 bytes} [Tue Aug 2 10:17:57 2022] PUT /identity/v3/projects/8c47a826f47544a5977034bf3067629f/users/870771ad4e3f4151a1ba07b305968320/roles/237966442f05411191105bf336bc14f3 => generated 0 bytes in 214 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 02 10:17:57.854886 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92dc1385-a8ec-48b8-b90e-f7a380d5a229 None None] REQUEST_METHOD: `POST` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:17:57.859805 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92dc1385-a8ec-48b8-b90e-f7a380d5a229 None None] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:17:57.860169 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92dc1385-a8ec-48b8-b90e-f7a380d5a229 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:17:57.912561 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.auth.core [None req-92dc1385-a8ec-48b8-b90e-f7a380d5a229 None None] MFA Rules not processed for user `891005fa5ff74041b75747d16cc66798`. Rule list: `[]` (Enabled: `True`). {{(pid=80378) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 02 10:17:57.914050 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.fernet_utils [None req-92dc1385-a8ec-48b8-b90e-f7a380d5a229 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80378) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 10:17:57.950866 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG dbcounter [-] [80378] Writing DB stats keystone:SELECT=91,keystone:DELETE=7,keystone:INSERT=2 {{(pid=80378) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 10:17:57.955962 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06b2b2f0-d6a0-4af8-a57b-10eb1485f783 None None] Validating token access rules against request {{(pid=80377) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:17:58.033305 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG dbcounter [-] [80377] Writing DB stats keystone:SELECT=100 {{(pid=80377) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 10:17:58.065127 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fd731c9-d832-4bdc-a9fd-d32f905be792 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88af518196674fe593ad88d30111fb7d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f1b38ebb804c41c69772c998aae72c41', 'project_domain_id': 'default', 'roles': ['admin', 'ResellerAdmin', 'reader', 'service', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80377) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:17:58.066572 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fd731c9-d832-4bdc-a9fd-d32f905be792 service nova] REQUEST_METHOD: `GET` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:17:58.067644 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fd731c9-d832-4bdc-a9fd-d32f905be792 service nova] SCRIPT_NAME: `/identity` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:17:58.068010 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fd731c9-d832-4bdc-a9fd-d32f905be792 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:17:58.070597 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.fernet_utils [None req-8fd731c9-d832-4bdc-a9fd-d32f905be792 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80377) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 10:17:58.096613 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 1122/2253] 10.222.0.82 () {58 vars in 958 bytes} [Tue Aug 2 10:17:57 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 245 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 02 10:17:58.098711 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ba48785-14de-4ab1-8516-630360f75c1c None None] REQUEST_METHOD: `POST` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:17:58.099154 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ba48785-14de-4ab1-8516-630360f75c1c None None] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:17:58.099603 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ba48785-14de-4ab1-8516-630360f75c1c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:17:58.151083 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.auth.core [None req-9ba48785-14de-4ab1-8516-630360f75c1c None None] MFA Rules not processed for user `870771ad4e3f4151a1ba07b305968320`. Rule list: `[]` (Enabled: `True`). {{(pid=80378) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 02 10:17:58.153302 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.fernet_utils [None req-9ba48785-14de-4ab1-8516-630360f75c1c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80378) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 10:17:58.193406 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8fd731c9-d832-4bdc-a9fd-d32f905be792 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:17:58.208196 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8fd731c9-d832-4bdc-a9fd-d32f905be792 service nova] RBAC: Authorization granted {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:17:58.231370 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: [pid: 80377|app: 0|req: 1132/2254] 10.222.0.82 () {60 vars in 1365 bytes} [Tue Aug 2 10:17:57 2022] GET /identity/v3/auth/tokens => generated 3250 bytes in 410 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 02 10:17:58.232124 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fd731c9-d832-4bdc-a9fd-d32f905be792 service nova] Authenticating user token {{(pid=80377) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:17:58.245474 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fd731c9-d832-4bdc-a9fd-d32f905be792 service nova] Validating token access rules against request {{(pid=80377) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:17:58.251953 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5120a083-99e5-4fef-9ed6-6469be041ba8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '891005fa5ff74041b75747d16cc66798', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7047912c5acd4adeba5d87c3bf895e39', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80377) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:17:58.253591 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5120a083-99e5-4fef-9ed6-6469be041ba8 admin admin] REQUEST_METHOD: `GET` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:17:58.254729 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5120a083-99e5-4fef-9ed6-6469be041ba8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:17:58.255102 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5120a083-99e5-4fef-9ed6-6469be041ba8 admin admin] PATH_INFO: `/v3/domains` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:17:58.257191 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5120a083-99e5-4fef-9ed6-6469be041ba8 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80377) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 02 10:17:58.258689 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5120a083-99e5-4fef-9ed6-6469be041ba8 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:17:58.262651 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5120a083-99e5-4fef-9ed6-6469be041ba8 admin admin] RBAC: Authorization granted {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:17:58.300755 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: [pid: 80377|app: 0|req: 1133/2255] 10.222.0.82 () {58 vars in 1153 bytes} [Tue Aug 2 10:17:58 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 66 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 02 10:17:58.305926 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5120a083-99e5-4fef-9ed6-6469be041ba8 admin admin] Authenticating user token {{(pid=80377) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:17:58.307466 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 1123/2256] 10.222.0.82 () {58 vars in 958 bytes} [Tue Aug 2 10:17:58 2022] POST /identity/v3/auth/tokens => generated 3329 bytes in 210 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 02 10:17:58.321514 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5120a083-99e5-4fef-9ed6-6469be041ba8 admin admin] Validating token access rules against request {{(pid=80377) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:17:58.321712 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ba48785-14de-4ab1-8516-630360f75c1c None None] Authenticating user token {{(pid=80378) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:17:58.324461 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.fernet_utils [None req-9ba48785-14de-4ab1-8516-630360f75c1c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80378) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 10:17:58.327835 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c938abb-b91f-4794-b3b8-69400652f895 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '891005fa5ff74041b75747d16cc66798', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7047912c5acd4adeba5d87c3bf895e39', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80377) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:17:58.328474 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c938abb-b91f-4794-b3b8-69400652f895 admin admin] REQUEST_METHOD: `POST` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:17:58.328754 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c938abb-b91f-4794-b3b8-69400652f895 admin admin] SCRIPT_NAME: `/identity` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:17:58.329031 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c938abb-b91f-4794-b3b8-69400652f895 admin admin] PATH_INFO: `/v3/projects` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:17:58.329707 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c938abb-b91f-4794-b3b8-69400652f895 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:17:58.332536 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c938abb-b91f-4794-b3b8-69400652f895 admin admin] RBAC: Authorization granted {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:17:58.393377 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: [pid: 80377|app: 0|req: 1134/2257] 10.222.0.82 () {60 vars in 1153 bytes} [Tue Aug 2 10:17:58 2022] POST /identity/v3/projects => generated 373 bytes in 85 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 02 10:17:58.397066 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c938abb-b91f-4794-b3b8-69400652f895 admin admin] Authenticating user token {{(pid=80377) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:17:58.498754 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c938abb-b91f-4794-b3b8-69400652f895 admin admin] Validating token access rules against request {{(pid=80377) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:17:58.509747 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ba48785-14de-4ab1-8516-630360f75c1c None None] Validating token access rules against request {{(pid=80378) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:17:58.512061 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67270647-4bd3-4b90-9f06-8e018b4a066c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '891005fa5ff74041b75747d16cc66798', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7047912c5acd4adeba5d87c3bf895e39', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80377) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:17:58.513408 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67270647-4bd3-4b90-9f06-8e018b4a066c admin admin] REQUEST_METHOD: `POST` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:17:58.515248 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67270647-4bd3-4b90-9f06-8e018b4a066c admin admin] SCRIPT_NAME: `/identity` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:17:58.515840 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67270647-4bd3-4b90-9f06-8e018b4a066c admin admin] PATH_INFO: `/v3/users` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:17:58.516834 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67270647-4bd3-4b90-9f06-8e018b4a066c admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:17:58.519030 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67270647-4bd3-4b90-9f06-8e018b4a066c admin admin] RBAC: Authorization granted {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:17:58.561851 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: [pid: 80377|app: 0|req: 1135/2258] 10.222.0.82 () {60 vars in 1144 bytes} [Tue Aug 2 10:17:58 2022] POST /identity/v3/users => generated 287 bytes in 167 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 02 10:17:58.565434 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67270647-4bd3-4b90-9f06-8e018b4a066c admin admin] Authenticating user token {{(pid=80377) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:17:58.602303 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67270647-4bd3-4b90-9f06-8e018b4a066c admin admin] Validating token access rules against request {{(pid=80377) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:17:58.617134 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b34859f4-eaf9-4eaf-9303-8b30873e18b3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '891005fa5ff74041b75747d16cc66798', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7047912c5acd4adeba5d87c3bf895e39', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80377) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:17:58.617815 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b34859f4-eaf9-4eaf-9303-8b30873e18b3 admin admin] REQUEST_METHOD: `GET` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:17:58.618093 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b34859f4-eaf9-4eaf-9303-8b30873e18b3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:17:58.618366 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b34859f4-eaf9-4eaf-9303-8b30873e18b3 admin admin] PATH_INFO: `/v3/roles` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:17:58.621057 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b34859f4-eaf9-4eaf-9303-8b30873e18b3 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:17:58.627741 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b34859f4-eaf9-4eaf-9303-8b30873e18b3 admin admin] RBAC: Authorization granted {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:17:58.629386 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10c01079-51cb-4921-bcad-264506102bc4 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1023de281ae74e9d8a6d6c312c1c97a7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f1b38ebb804c41c69772c998aae72c41', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80378) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:17:58.632514 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10c01079-51cb-4921-bcad-264506102bc4 service cinder] REQUEST_METHOD: `GET` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:17:58.632862 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10c01079-51cb-4921-bcad-264506102bc4 service cinder] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:17:58.633092 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10c01079-51cb-4921-bcad-264506102bc4 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:17:58.648057 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.fernet_utils [None req-10c01079-51cb-4921-bcad-264506102bc4 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80378) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 10:17:58.651508 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: [pid: 80377|app: 0|req: 1136/2259] 10.222.0.82 () {58 vars in 1122 bytes} [Tue Aug 2 10:17:58 2022] GET /identity/v3/roles => generated 1420 bytes in 84 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 02 10:17:58.652666 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b34859f4-eaf9-4eaf-9303-8b30873e18b3 admin admin] Authenticating user token {{(pid=80377) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:17:58.665897 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b34859f4-eaf9-4eaf-9303-8b30873e18b3 admin admin] Validating token access rules against request {{(pid=80377) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:17:58.675707 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb7e2fce-7c4e-4b63-83bc-8c74f8509164 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '891005fa5ff74041b75747d16cc66798', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7047912c5acd4adeba5d87c3bf895e39', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80377) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:17:58.677333 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb7e2fce-7c4e-4b63-83bc-8c74f8509164 admin admin] REQUEST_METHOD: `GET` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:17:58.677630 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb7e2fce-7c4e-4b63-83bc-8c74f8509164 admin admin] SCRIPT_NAME: `/identity` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:17:58.678180 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb7e2fce-7c4e-4b63-83bc-8c74f8509164 admin admin] PATH_INFO: `/v3/roles` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:17:58.678893 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb7e2fce-7c4e-4b63-83bc-8c74f8509164 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:17:58.682751 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb7e2fce-7c4e-4b63-83bc-8c74f8509164 admin admin] RBAC: Authorization granted {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:17:58.685573 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG dbcounter [-] [80378] Writing DB stats keystone:SELECT=100 {{(pid=80378) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 10:17:58.711062 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: [pid: 80377|app: 0|req: 1137/2260] 10.222.0.82 () {58 vars in 1122 bytes} [Tue Aug 2 10:17:58 2022] GET /identity/v3/roles => generated 1420 bytes in 59 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 02 10:17:58.712180 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb7e2fce-7c4e-4b63-83bc-8c74f8509164 admin admin] Authenticating user token {{(pid=80377) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:17:58.726404 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb7e2fce-7c4e-4b63-83bc-8c74f8509164 admin admin] Validating token access rules against request {{(pid=80377) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:17:58.732260 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbaad18a-97ea-4f6a-b764-262f2cc1f6cb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '891005fa5ff74041b75747d16cc66798', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7047912c5acd4adeba5d87c3bf895e39', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80377) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:17:58.732966 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbaad18a-97ea-4f6a-b764-262f2cc1f6cb admin admin] REQUEST_METHOD: `PUT` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:17:58.733242 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbaad18a-97ea-4f6a-b764-262f2cc1f6cb admin admin] SCRIPT_NAME: `/identity` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:17:58.733533 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbaad18a-97ea-4f6a-b764-262f2cc1f6cb admin admin] PATH_INFO: `/v3/projects/1a99db57b255411e8d1f1ec214f40610/users/b3bdb446b6054254b51687d7c1e37603/roles/e1c04b19fc4543b8b54f58bbcffbc598` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:17:58.747520 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG dbcounter [-] [80377] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=80377) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 10:17:58.753196 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cbaad18a-97ea-4f6a-b764-262f2cc1f6cb admin admin] RBAC: Authorizing `identity:create_grant(project_id=1a99db57b255411e8d1f1ec214f40610, user_id=b3bdb446b6054254b51687d7c1e37603, role_id=e1c04b19fc4543b8b54f58bbcffbc598)` {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:17:58.756947 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cbaad18a-97ea-4f6a-b764-262f2cc1f6cb admin admin] RBAC: Authorization granted {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:17:58.780369 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: [pid: 80377|app: 0|req: 1138/2261] 10.222.0.82 () {60 vars in 1483 bytes} [Tue Aug 2 10:17:58 2022] PUT /identity/v3/projects/1a99db57b255411e8d1f1ec214f40610/users/b3bdb446b6054254b51687d7c1e37603/roles/e1c04b19fc4543b8b54f58bbcffbc598 => generated 0 bytes in 69 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 02 10:17:58.792257 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b751a842-be62-48a1-a3b6-2beaf72dc1ea None None] REQUEST_METHOD: `POST` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:17:58.792792 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b751a842-be62-48a1-a3b6-2beaf72dc1ea None None] SCRIPT_NAME: `/identity` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:17:58.793625 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b751a842-be62-48a1-a3b6-2beaf72dc1ea None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:17:58.798113 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10c01079-51cb-4921-bcad-264506102bc4 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:17:58.799660 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10c01079-51cb-4921-bcad-264506102bc4 service cinder] RBAC: Authorization granted {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:17:58.859229 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 1124/2262] 10.222.0.82 () {60 vars in 1365 bytes} [Tue Aug 2 10:17:58 2022] GET /identity/v3/auth/tokens => generated 3258 bytes in 538 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 02 10:17:58.860819 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.auth.core [None req-b751a842-be62-48a1-a3b6-2beaf72dc1ea None None] MFA Rules not processed for user `b3bdb446b6054254b51687d7c1e37603`. Rule list: `[]` (Enabled: `True`). {{(pid=80377) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 02 10:17:58.863268 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.fernet_utils [None req-b751a842-be62-48a1-a3b6-2beaf72dc1ea None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80377) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 10:17:59.052101 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: [pid: 80377|app: 0|req: 1139/2263] 10.222.0.82 () {58 vars in 958 bytes} [Tue Aug 2 10:17:58 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 262 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 02 10:17:59.055279 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10c01079-51cb-4921-bcad-264506102bc4 service cinder] Authenticating user token {{(pid=80378) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:17:59.081310 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10c01079-51cb-4921-bcad-264506102bc4 service cinder] Validating token access rules against request {{(pid=80378) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:17:59.278507 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60911d1e-0341-4677-9ed8-18eccf967943 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '891005fa5ff74041b75747d16cc66798', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7047912c5acd4adeba5d87c3bf895e39', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80378) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:17:59.280049 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60911d1e-0341-4677-9ed8-18eccf967943 admin admin] REQUEST_METHOD: `POST` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:17:59.280598 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60911d1e-0341-4677-9ed8-18eccf967943 admin admin] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:17:59.281040 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60911d1e-0341-4677-9ed8-18eccf967943 admin admin] PATH_INFO: `/v3/projects` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:17:59.282287 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60911d1e-0341-4677-9ed8-18eccf967943 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:17:59.293169 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60911d1e-0341-4677-9ed8-18eccf967943 admin admin] RBAC: Authorization granted {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:17:59.355110 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 1125/2264] 10.222.0.82 () {60 vars in 1153 bytes} [Tue Aug 2 10:17:59 2022] POST /identity/v3/projects => generated 371 bytes in 301 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 02 10:17:59.360253 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b751a842-be62-48a1-a3b6-2beaf72dc1ea None None] Authenticating user token {{(pid=80377) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:17:59.383207 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b751a842-be62-48a1-a3b6-2beaf72dc1ea None None] Validating token access rules against request {{(pid=80377) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:17:59.583551 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be0e4f0e-f384-4c6f-ac09-b284cfabd3a6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '891005fa5ff74041b75747d16cc66798', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7047912c5acd4adeba5d87c3bf895e39', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80377) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:17:59.584390 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be0e4f0e-f384-4c6f-ac09-b284cfabd3a6 admin admin] REQUEST_METHOD: `POST` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:17:59.584774 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be0e4f0e-f384-4c6f-ac09-b284cfabd3a6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:17:59.585149 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be0e4f0e-f384-4c6f-ac09-b284cfabd3a6 admin admin] PATH_INFO: `/v3/users` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:17:59.594196 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be0e4f0e-f384-4c6f-ac09-b284cfabd3a6 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:17:59.598527 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be0e4f0e-f384-4c6f-ac09-b284cfabd3a6 admin admin] RBAC: Authorization granted {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:17:59.632797 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: [pid: 80377|app: 0|req: 1140/2265] 10.222.0.82 () {60 vars in 1144 bytes} [Tue Aug 2 10:17:59 2022] POST /identity/v3/users => generated 292 bytes in 273 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 02 10:17:59.636953 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60911d1e-0341-4677-9ed8-18eccf967943 admin admin] Authenticating user token {{(pid=80378) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:17:59.657239 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60911d1e-0341-4677-9ed8-18eccf967943 admin admin] Validating token access rules against request {{(pid=80378) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:17:59.681820 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5cfb56e6-9b2c-4ce8-96dc-0464442e5f3b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '891005fa5ff74041b75747d16cc66798', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7047912c5acd4adeba5d87c3bf895e39', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80378) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:17:59.682247 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cfb56e6-9b2c-4ce8-96dc-0464442e5f3b admin admin] REQUEST_METHOD: `GET` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:17:59.684376 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cfb56e6-9b2c-4ce8-96dc-0464442e5f3b admin admin] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:17:59.684376 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cfb56e6-9b2c-4ce8-96dc-0464442e5f3b admin admin] PATH_INFO: `/v3/roles` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:17:59.684376 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5cfb56e6-9b2c-4ce8-96dc-0464442e5f3b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:17:59.685433 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5cfb56e6-9b2c-4ce8-96dc-0464442e5f3b admin admin] RBAC: Authorization granted {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:17:59.733928 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 1126/2266] 10.222.0.82 () {58 vars in 1122 bytes} [Tue Aug 2 10:17:59 2022] GET /identity/v3/roles => generated 1420 bytes in 98 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 02 10:17:59.739663 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be0e4f0e-f384-4c6f-ac09-b284cfabd3a6 admin admin] Authenticating user token {{(pid=80377) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:17:59.762681 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be0e4f0e-f384-4c6f-ac09-b284cfabd3a6 admin admin] Validating token access rules against request {{(pid=80377) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:17:59.774963 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0f5d2b7-ef27-45ab-9d64-2bc0ad734ca4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '891005fa5ff74041b75747d16cc66798', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7047912c5acd4adeba5d87c3bf895e39', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80377) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:17:59.775727 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0f5d2b7-ef27-45ab-9d64-2bc0ad734ca4 admin admin] REQUEST_METHOD: `PUT` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:17:59.776092 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0f5d2b7-ef27-45ab-9d64-2bc0ad734ca4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:17:59.776373 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0f5d2b7-ef27-45ab-9d64-2bc0ad734ca4 admin admin] PATH_INFO: `/v3/projects/bad4aaeee3bf481aa49a73036aba3617/users/eba9240efa834724a46464813c6188ed/roles/237966442f05411191105bf336bc14f3` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:17:59.813655 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0f5d2b7-ef27-45ab-9d64-2bc0ad734ca4 admin admin] RBAC: Authorizing `identity:create_grant(project_id=bad4aaeee3bf481aa49a73036aba3617, user_id=eba9240efa834724a46464813c6188ed, role_id=237966442f05411191105bf336bc14f3)` {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:17:59.819501 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0f5d2b7-ef27-45ab-9d64-2bc0ad734ca4 admin admin] RBAC: Authorization granted {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:17:59.870867 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: [pid: 80377|app: 0|req: 1141/2267] 10.222.0.82 () {60 vars in 1483 bytes} [Tue Aug 2 10:17:59 2022] PUT /identity/v3/projects/bad4aaeee3bf481aa49a73036aba3617/users/eba9240efa834724a46464813c6188ed/roles/237966442f05411191105bf336bc14f3 => generated 0 bytes in 134 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 02 10:17:59.875786 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5cfb56e6-9b2c-4ce8-96dc-0464442e5f3b admin admin] Authenticating user token {{(pid=80378) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:17:59.914801 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5cfb56e6-9b2c-4ce8-96dc-0464442e5f3b admin admin] Validating token access rules against request {{(pid=80378) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:18:00.084295 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4093d6d-aa1e-40df-a342-6b38cb676794 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '891005fa5ff74041b75747d16cc66798', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7047912c5acd4adeba5d87c3bf895e39', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80378) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:18:00.085825 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4093d6d-aa1e-40df-a342-6b38cb676794 admin admin] REQUEST_METHOD: `GET` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:18:00.085825 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4093d6d-aa1e-40df-a342-6b38cb676794 admin admin] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:18:00.085825 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4093d6d-aa1e-40df-a342-6b38cb676794 admin admin] PATH_INFO: `/v3/roles` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:18:00.086608 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4093d6d-aa1e-40df-a342-6b38cb676794 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:18:00.088726 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4093d6d-aa1e-40df-a342-6b38cb676794 admin admin] RBAC: Authorization granted {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:18:00.104382 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 1127/2268] 10.222.0.82 () {58 vars in 1122 bytes} [Tue Aug 2 10:17:59 2022] GET /identity/v3/roles => generated 1420 bytes in 229 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 02 10:18:00.108809 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0f5d2b7-ef27-45ab-9d64-2bc0ad734ca4 admin admin] Authenticating user token {{(pid=80377) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:18:00.138186 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0f5d2b7-ef27-45ab-9d64-2bc0ad734ca4 admin admin] Validating token access rules against request {{(pid=80377) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:18:00.147054 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ad757ae-dc38-4890-82a4-d088babf2f79 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '891005fa5ff74041b75747d16cc66798', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7047912c5acd4adeba5d87c3bf895e39', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80377) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:18:00.148556 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ad757ae-dc38-4890-82a4-d088babf2f79 admin admin] REQUEST_METHOD: `PUT` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:18:00.149479 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ad757ae-dc38-4890-82a4-d088babf2f79 admin admin] SCRIPT_NAME: `/identity` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:18:00.149964 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ad757ae-dc38-4890-82a4-d088babf2f79 admin admin] PATH_INFO: `/v3/projects/bad4aaeee3bf481aa49a73036aba3617/users/eba9240efa834724a46464813c6188ed/roles/237966442f05411191105bf336bc14f3` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:18:00.153604 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ad757ae-dc38-4890-82a4-d088babf2f79 admin admin] RBAC: Authorizing `identity:create_grant(project_id=bad4aaeee3bf481aa49a73036aba3617, user_id=eba9240efa834724a46464813c6188ed, role_id=237966442f05411191105bf336bc14f3)` {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:18:00.159418 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ad757ae-dc38-4890-82a4-d088babf2f79 admin admin] RBAC: Authorization granted {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:18:00.201610 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: [pid: 80377|app: 0|req: 1142/2269] 10.222.0.82 () {60 vars in 1483 bytes} [Tue Aug 2 10:18:00 2022] PUT /identity/v3/projects/bad4aaeee3bf481aa49a73036aba3617/users/eba9240efa834724a46464813c6188ed/roles/237966442f05411191105bf336bc14f3 => generated 0 bytes in 92 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 02 10:18:00.213231 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c666eca7-aa54-4dbb-8515-94d909c3252c None None] REQUEST_METHOD: `POST` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:18:00.213766 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c666eca7-aa54-4dbb-8515-94d909c3252c None None] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:18:00.214237 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c666eca7-aa54-4dbb-8515-94d909c3252c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:18:00.291852 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.auth.core [None req-c666eca7-aa54-4dbb-8515-94d909c3252c None None] MFA Rules not processed for user `eba9240efa834724a46464813c6188ed`. Rule list: `[]` (Enabled: `True`). {{(pid=80378) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 02 10:18:00.293835 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.fernet_utils [None req-c666eca7-aa54-4dbb-8515-94d909c3252c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80378) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 10:18:00.325691 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG dbcounter [-] [80378] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80378) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 10:18:00.495242 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 1128/2270] 10.222.0.82 () {58 vars in 958 bytes} [Tue Aug 2 10:18:00 2022] POST /identity/v3/auth/tokens => generated 3315 bytes in 278 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 02 10:18:00.504675 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ad757ae-dc38-4890-82a4-d088babf2f79 admin admin] Authenticating user token {{(pid=80377) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:18:00.508015 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.fernet_utils [None req-3ad757ae-dc38-4890-82a4-d088babf2f79 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80377) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 10:18:00.573324 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8328dc35-46f7-4b98-ac47-b00af3851abd None None] REQUEST_METHOD: `POST` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:18:00.573617 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8328dc35-46f7-4b98-ac47-b00af3851abd None None] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:18:00.573932 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8328dc35-46f7-4b98-ac47-b00af3851abd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:18:00.619323 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.auth.core [None req-8328dc35-46f7-4b98-ac47-b00af3851abd None None] MFA Rules not processed for user `7b672b80961445c3a3a9b2d784f64208`. Rule list: `[]` (Enabled: `True`). {{(pid=80378) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 02 10:18:00.620024 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG dbcounter [-] [80377] Writing DB stats keystone:SELECT=94,keystone:INSERT=6 {{(pid=80377) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 10:18:00.620528 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.fernet_utils [None req-8328dc35-46f7-4b98-ac47-b00af3851abd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80378) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 10:18:00.665151 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ad757ae-dc38-4890-82a4-d088babf2f79 admin admin] Validating token access rules against request {{(pid=80377) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:18:00.677842 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cef6dc19-9b62-4fe9-a211-084f69475c3c service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1023de281ae74e9d8a6d6c312c1c97a7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f1b38ebb804c41c69772c998aae72c41', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80377) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:18:00.678687 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cef6dc19-9b62-4fe9-a211-084f69475c3c service cinder] REQUEST_METHOD: `GET` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:18:00.679080 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cef6dc19-9b62-4fe9-a211-084f69475c3c service cinder] SCRIPT_NAME: `/identity` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:18:00.679459 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cef6dc19-9b62-4fe9-a211-084f69475c3c service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:18:00.724042 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cef6dc19-9b62-4fe9-a211-084f69475c3c service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:18:00.726207 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cef6dc19-9b62-4fe9-a211-084f69475c3c service cinder] RBAC: Authorization granted {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:18:00.741818 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: [pid: 80377|app: 0|req: 1143/2271] 10.222.0.82 () {60 vars in 1365 bytes} [Tue Aug 2 10:18:00 2022] GET /identity/v3/auth/tokens => generated 3315 bytes in 237 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 02 10:18:00.830302 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 1129/2272] 10.222.0.82 () {58 vars in 958 bytes} [Tue Aug 2 10:18:00 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 258 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 02 10:18:01.469592 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cef6dc19-9b62-4fe9-a211-084f69475c3c service cinder] Authenticating user token {{(pid=80377) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:18:01.558758 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cef6dc19-9b62-4fe9-a211-084f69475c3c service cinder] Validating token access rules against request {{(pid=80377) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:18:01.567526 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b163868-e2a9-4392-bf28-715f5fa5e3c5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '891005fa5ff74041b75747d16cc66798', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7047912c5acd4adeba5d87c3bf895e39', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80377) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:18:01.568197 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b163868-e2a9-4392-bf28-715f5fa5e3c5 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:18:01.568479 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b163868-e2a9-4392-bf28-715f5fa5e3c5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:18:01.568756 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b163868-e2a9-4392-bf28-715f5fa5e3c5 admin admin] PATH_INFO: `/v3/users/b3bdb446b6054254b51687d7c1e37603` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:18:01.570081 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b163868-e2a9-4392-bf28-715f5fa5e3c5 admin admin] RBAC: Authorizing `identity:delete_user(user_id=b3bdb446b6054254b51687d7c1e37603)` {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:18:01.571915 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b163868-e2a9-4392-bf28-715f5fa5e3c5 admin admin] RBAC: Authorization granted {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:18:01.646013 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.notifications [None req-2b163868-e2a9-4392-bf28-715f5fa5e3c5 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'b3bdb446b6054254b51687d7c1e37603'} {{(pid=80377) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 10:18:01.660726 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.notifications [None req-2b163868-e2a9-4392-bf28-715f5fa5e3c5 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'b3bdb446b6054254b51687d7c1e37603'} {{(pid=80377) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 10:18:01.671155 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.notifications [None req-2b163868-e2a9-4392-bf28-715f5fa5e3c5 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'b3bdb446b6054254b51687d7c1e37603'} {{(pid=80377) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 10:18:01.671893 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.notifications [None req-2b163868-e2a9-4392-bf28-715f5fa5e3c5 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'b3bdb446b6054254b51687d7c1e37603'} {{(pid=80377) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 10:18:01.695616 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.notifications [None req-2b163868-e2a9-4392-bf28-715f5fa5e3c5 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'b3bdb446b6054254b51687d7c1e37603'} {{(pid=80377) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 10:18:01.704692 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: [pid: 80377|app: 0|req: 1144/2273] 10.222.0.82 () {58 vars in 1224 bytes} [Tue Aug 2 10:18:01 2022] DELETE /identity/v3/users/b3bdb446b6054254b51687d7c1e37603 => generated 0 bytes in 234 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 02 10:18:01.709252 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7cd4f45-d2a0-4685-af21-d0755085a54d None None] REQUEST_METHOD: `POST` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:18:01.709736 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7cd4f45-d2a0-4685-af21-d0755085a54d None None] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:18:01.709882 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7cd4f45-d2a0-4685-af21-d0755085a54d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:18:01.738437 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.auth.core [None req-a7cd4f45-d2a0-4685-af21-d0755085a54d None None] MFA Rules not processed for user `891005fa5ff74041b75747d16cc66798`. Rule list: `[]` (Enabled: `True`). {{(pid=80378) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 02 10:18:01.739835 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.fernet_utils [None req-a7cd4f45-d2a0-4685-af21-d0755085a54d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80378) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 10:18:01.898321 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 1130/2274] 10.222.0.82 () {58 vars in 958 bytes} [Tue Aug 2 10:18:01 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 190 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 02 10:18:01.916844 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b163868-e2a9-4392-bf28-715f5fa5e3c5 admin admin] Authenticating user token {{(pid=80377) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:18:01.918531 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.fernet_utils [None req-2b163868-e2a9-4392-bf28-715f5fa5e3c5 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80377) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 10:18:02.069199 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b163868-e2a9-4392-bf28-715f5fa5e3c5 admin admin] Validating token access rules against request {{(pid=80377) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:18:02.080950 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0098283-9172-4895-8c9c-635d7cc74443 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '33b87170f0634b67bce60750847b5cbe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f1b38ebb804c41c69772c998aae72c41', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80377) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:18:02.081837 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0098283-9172-4895-8c9c-635d7cc74443 service neutron] REQUEST_METHOD: `GET` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:18:02.082351 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0098283-9172-4895-8c9c-635d7cc74443 service neutron] SCRIPT_NAME: `/identity` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:18:02.082800 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0098283-9172-4895-8c9c-635d7cc74443 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:18:02.101884 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0098283-9172-4895-8c9c-635d7cc74443 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:18:02.103531 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0098283-9172-4895-8c9c-635d7cc74443 service neutron] RBAC: Authorization granted {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:18:02.110398 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG dbcounter [-] [80377] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80377) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 10:18:02.118215 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: [pid: 80377|app: 0|req: 1145/2275] 10.222.0.82 () {60 vars in 1365 bytes} [Tue Aug 2 10:18:01 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 202 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 02 10:18:02.189071 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7cd4f45-d2a0-4685-af21-d0755085a54d None None] Authenticating user token {{(pid=80378) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:18:02.190516 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.fernet_utils [None req-a7cd4f45-d2a0-4685-af21-d0755085a54d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80378) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 10:18:02.348131 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7cd4f45-d2a0-4685-af21-d0755085a54d None None] Validating token access rules against request {{(pid=80378) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:18:02.356689 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e223cef1-af59-465f-8e45-85d167ce0119 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1023de281ae74e9d8a6d6c312c1c97a7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f1b38ebb804c41c69772c998aae72c41', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80378) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:18:02.358054 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e223cef1-af59-465f-8e45-85d167ce0119 service cinder] REQUEST_METHOD: `GET` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:18:02.361333 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e223cef1-af59-465f-8e45-85d167ce0119 service cinder] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:18:02.361722 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e223cef1-af59-465f-8e45-85d167ce0119 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:18:02.372864 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.fernet_utils [None req-e223cef1-af59-465f-8e45-85d167ce0119 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80378) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 10:18:02.412378 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG dbcounter [-] [80378] Writing DB stats keystone:SELECT=100 {{(pid=80378) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 10:18:02.559045 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e223cef1-af59-465f-8e45-85d167ce0119 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:18:02.560537 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e223cef1-af59-465f-8e45-85d167ce0119 service cinder] RBAC: Authorization granted {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:18:02.585818 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 1131/2276] 10.222.0.82 () {60 vars in 1365 bytes} [Tue Aug 2 10:18:02 2022] GET /identity/v3/auth/tokens => generated 3262 bytes in 395 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 02 10:18:02.631441 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0098283-9172-4895-8c9c-635d7cc74443 service neutron] Authenticating user token {{(pid=80377) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:18:02.633607 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.fernet_utils [None req-b0098283-9172-4895-8c9c-635d7cc74443 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80377) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 10:18:02.671131 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0098283-9172-4895-8c9c-635d7cc74443 service neutron] Validating token access rules against request {{(pid=80377) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:18:02.677208 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45286ea1-dc18-43eb-8381-64734c2291ae admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '891005fa5ff74041b75747d16cc66798', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7047912c5acd4adeba5d87c3bf895e39', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80377) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:18:02.677977 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45286ea1-dc18-43eb-8381-64734c2291ae admin admin] REQUEST_METHOD: `DELETE` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:18:02.678263 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45286ea1-dc18-43eb-8381-64734c2291ae admin admin] SCRIPT_NAME: `/identity` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:18:02.678535 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45286ea1-dc18-43eb-8381-64734c2291ae admin admin] PATH_INFO: `/v3/projects/1a99db57b255411e8d1f1ec214f40610` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:18:02.680373 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45286ea1-dc18-43eb-8381-64734c2291ae admin admin] RBAC: Authorizing `identity:delete_project(project_id=1a99db57b255411e8d1f1ec214f40610)` {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:18:02.682844 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45286ea1-dc18-43eb-8381-64734c2291ae admin admin] RBAC: Authorization granted {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:18:02.759124 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.notifications [None req-45286ea1-dc18-43eb-8381-64734c2291ae admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '1a99db57b255411e8d1f1ec214f40610', 'request_id': 'req-45286ea1-dc18-43eb-8381-64734c2291ae'} {{(pid=80377) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 10:18:02.767377 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.notifications [None req-45286ea1-dc18-43eb-8381-64734c2291ae admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '1a99db57b255411e8d1f1ec214f40610', 'request_id': 'req-45286ea1-dc18-43eb-8381-64734c2291ae'} {{(pid=80377) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 10:18:02.776950 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.notifications [None req-45286ea1-dc18-43eb-8381-64734c2291ae admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '1a99db57b255411e8d1f1ec214f40610', 'request_id': 'req-45286ea1-dc18-43eb-8381-64734c2291ae'} {{(pid=80377) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 10:18:02.795755 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.notifications [None req-45286ea1-dc18-43eb-8381-64734c2291ae admin admin] The token cache is being invalidate because project 1a99db57b255411e8d1f1ec214f40610 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80377) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 02 10:18:02.796037 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.notifications [None req-45286ea1-dc18-43eb-8381-64734c2291ae admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80377) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 10:18:02.796763 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.notifications [None req-45286ea1-dc18-43eb-8381-64734c2291ae admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80377) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 10:18:02.798986 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: [pid: 80377|app: 0|req: 1146/2277] 10.222.0.82 () {58 vars in 1233 bytes} [Tue Aug 2 10:18:02 2022] DELETE /identity/v3/projects/1a99db57b255411e8d1f1ec214f40610 => generated 0 bytes in 168 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 02 10:18:02.803613 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e223cef1-af59-465f-8e45-85d167ce0119 service cinder] Authenticating user token {{(pid=80378) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:18:02.818203 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e223cef1-af59-465f-8e45-85d167ce0119 service cinder] Validating token access rules against request {{(pid=80378) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:18:02.909948 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45df9295-e2db-4081-a601-6ef1c0b63bdf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '891005fa5ff74041b75747d16cc66798', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7047912c5acd4adeba5d87c3bf895e39', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80378) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:18:02.909948 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45df9295-e2db-4081-a601-6ef1c0b63bdf admin admin] REQUEST_METHOD: `DELETE` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:18:02.910184 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45df9295-e2db-4081-a601-6ef1c0b63bdf admin admin] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:18:02.910382 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45df9295-e2db-4081-a601-6ef1c0b63bdf admin admin] PATH_INFO: `/v3/users/eba9240efa834724a46464813c6188ed` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:18:02.911562 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45df9295-e2db-4081-a601-6ef1c0b63bdf admin admin] RBAC: Authorizing `identity:delete_user(user_id=eba9240efa834724a46464813c6188ed)` {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:18:02.913831 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45df9295-e2db-4081-a601-6ef1c0b63bdf admin admin] RBAC: Authorization granted {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:18:02.986567 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.notifications [None req-45df9295-e2db-4081-a601-6ef1c0b63bdf admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'eba9240efa834724a46464813c6188ed'} {{(pid=80378) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 10:18:03.001493 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.notifications [None req-45df9295-e2db-4081-a601-6ef1c0b63bdf admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'eba9240efa834724a46464813c6188ed'} {{(pid=80378) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 10:18:03.012224 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.notifications [None req-45df9295-e2db-4081-a601-6ef1c0b63bdf admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'eba9240efa834724a46464813c6188ed'} {{(pid=80378) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 10:18:03.014600 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.notifications [None req-45df9295-e2db-4081-a601-6ef1c0b63bdf admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'eba9240efa834724a46464813c6188ed'} {{(pid=80378) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 10:18:03.030797 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.notifications [None req-45df9295-e2db-4081-a601-6ef1c0b63bdf admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'eba9240efa834724a46464813c6188ed'} {{(pid=80378) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 10:18:03.044307 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 1132/2278] 10.222.0.82 () {58 vars in 1224 bytes} [Tue Aug 2 10:18:02 2022] DELETE /identity/v3/users/eba9240efa834724a46464813c6188ed => generated 0 bytes in 237 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 02 10:18:03.801645 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45286ea1-dc18-43eb-8381-64734c2291ae admin admin] Authenticating user token {{(pid=80377) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:18:03.811228 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.fernet_utils [None req-45286ea1-dc18-43eb-8381-64734c2291ae admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80377) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 10:18:03.863373 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45286ea1-dc18-43eb-8381-64734c2291ae admin admin] Validating token access rules against request {{(pid=80377) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:18:04.030174 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45df9295-e2db-4081-a601-6ef1c0b63bdf admin admin] Authenticating user token {{(pid=80378) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:18:04.032319 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.fernet_utils [None req-45df9295-e2db-4081-a601-6ef1c0b63bdf admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80378) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 10:18:04.102561 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d18e827-949f-4fce-8fcd-482bc43eac20 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '891005fa5ff74041b75747d16cc66798', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7047912c5acd4adeba5d87c3bf895e39', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80377) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:18:04.103251 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d18e827-949f-4fce-8fcd-482bc43eac20 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:18:04.103537 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d18e827-949f-4fce-8fcd-482bc43eac20 admin admin] SCRIPT_NAME: `/identity` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:18:04.103823 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d18e827-949f-4fce-8fcd-482bc43eac20 admin admin] PATH_INFO: `/v3/projects/bad4aaeee3bf481aa49a73036aba3617` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:18:04.107051 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d18e827-949f-4fce-8fcd-482bc43eac20 admin admin] RBAC: Authorizing `identity:delete_project(project_id=bad4aaeee3bf481aa49a73036aba3617)` {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:18:04.109718 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d18e827-949f-4fce-8fcd-482bc43eac20 admin admin] RBAC: Authorization granted {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:18:04.136906 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG dbcounter [-] [80378] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80378) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 10:18:04.194618 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45df9295-e2db-4081-a601-6ef1c0b63bdf admin admin] Validating token access rules against request {{(pid=80378) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:18:04.234888 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.notifications [None req-5d18e827-949f-4fce-8fcd-482bc43eac20 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'bad4aaeee3bf481aa49a73036aba3617', 'request_id': 'req-5d18e827-949f-4fce-8fcd-482bc43eac20'} {{(pid=80377) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 10:18:04.250575 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.notifications [None req-5d18e827-949f-4fce-8fcd-482bc43eac20 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'bad4aaeee3bf481aa49a73036aba3617', 'request_id': 'req-5d18e827-949f-4fce-8fcd-482bc43eac20'} {{(pid=80377) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 10:18:04.270919 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.notifications [None req-5d18e827-949f-4fce-8fcd-482bc43eac20 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'bad4aaeee3bf481aa49a73036aba3617', 'request_id': 'req-5d18e827-949f-4fce-8fcd-482bc43eac20'} {{(pid=80377) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 10:18:04.300321 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.notifications [None req-5d18e827-949f-4fce-8fcd-482bc43eac20 admin admin] The token cache is being invalidate because project bad4aaeee3bf481aa49a73036aba3617 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80377) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 02 10:18:04.300664 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.notifications [None req-5d18e827-949f-4fce-8fcd-482bc43eac20 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80377) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 10:18:04.301394 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.notifications [None req-5d18e827-949f-4fce-8fcd-482bc43eac20 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80377) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 10:18:04.303230 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: [pid: 80377|app: 0|req: 1147/2279] 10.222.0.82 () {58 vars in 1233 bytes} [Tue Aug 2 10:18:03 2022] DELETE /identity/v3/projects/bad4aaeee3bf481aa49a73036aba3617 => generated 0 bytes in 506 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 02 10:18:04.312109 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35326114-73bd-424c-b16e-4331471b1ce4 None None] REQUEST_METHOD: `POST` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:18:04.313001 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35326114-73bd-424c-b16e-4331471b1ce4 None None] SCRIPT_NAME: `/identity` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:18:04.313566 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b18aeea-136a-4f4a-b035-e0117eb86712 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1023de281ae74e9d8a6d6c312c1c97a7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f1b38ebb804c41c69772c998aae72c41', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80378) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:18:04.313807 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35326114-73bd-424c-b16e-4331471b1ce4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:18:04.314357 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b18aeea-136a-4f4a-b035-e0117eb86712 service cinder] REQUEST_METHOD: `GET` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:18:04.314748 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b18aeea-136a-4f4a-b035-e0117eb86712 service cinder] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:18:04.315024 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b18aeea-136a-4f4a-b035-e0117eb86712 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:18:04.317432 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.fernet_utils [None req-8b18aeea-136a-4f4a-b035-e0117eb86712 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80378) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 10:18:04.356564 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.auth.core [None req-35326114-73bd-424c-b16e-4331471b1ce4 None None] MFA Rules not processed for user `891005fa5ff74041b75747d16cc66798`. Rule list: `[]` (Enabled: `True`). {{(pid=80377) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 02 10:18:04.357947 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.fernet_utils [None req-35326114-73bd-424c-b16e-4331471b1ce4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80377) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 10:18:04.366428 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG dbcounter [-] [80377] Writing DB stats keystone:SELECT=84,keystone:DELETE=14,keystone:INSERT=2 {{(pid=80377) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 10:18:04.437828 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b18aeea-136a-4f4a-b035-e0117eb86712 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:18:04.439604 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b18aeea-136a-4f4a-b035-e0117eb86712 service cinder] RBAC: Authorization granted {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:18:04.470230 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 1133/2280] 10.222.0.82 () {60 vars in 1365 bytes} [Tue Aug 2 10:18:04 2022] GET /identity/v3/auth/tokens => generated 3329 bytes in 441 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 02 10:18:04.471798 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: [pid: 80377|app: 0|req: 1148/2281] 10.222.0.82 () {58 vars in 958 bytes} [Tue Aug 2 10:18:04 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 161 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 02 10:18:04.481048 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38b8084d-bb5d-4e48-89f2-b685475e0827 None None] REQUEST_METHOD: `POST` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:18:04.481048 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38b8084d-bb5d-4e48-89f2-b685475e0827 None None] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:18:04.481048 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38b8084d-bb5d-4e48-89f2-b685475e0827 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:18:04.505269 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.auth.core [None req-38b8084d-bb5d-4e48-89f2-b685475e0827 None None] MFA Rules not processed for user `891005fa5ff74041b75747d16cc66798`. Rule list: `[]` (Enabled: `True`). {{(pid=80378) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 02 10:18:04.506665 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.fernet_utils [None req-38b8084d-bb5d-4e48-89f2-b685475e0827 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80378) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 10:18:04.520876 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 1134/2282] 10.222.0.82 () {58 vars in 958 bytes} [Tue Aug 2 10:18:04 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 42 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 02 10:18:04.525425 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35326114-73bd-424c-b16e-4331471b1ce4 None None] Authenticating user token {{(pid=80377) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:18:04.541094 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35326114-73bd-424c-b16e-4331471b1ce4 None None] Validating token access rules against request {{(pid=80377) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:18:04.550208 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03cecb90-418c-488f-b063-8c7fe2df4f52 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '891005fa5ff74041b75747d16cc66798', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7047912c5acd4adeba5d87c3bf895e39', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80377) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:18:04.551045 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03cecb90-418c-488f-b063-8c7fe2df4f52 admin admin] REQUEST_METHOD: `GET` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:18:04.551192 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03cecb90-418c-488f-b063-8c7fe2df4f52 admin admin] SCRIPT_NAME: `/identity` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:18:04.551467 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03cecb90-418c-488f-b063-8c7fe2df4f52 admin admin] PATH_INFO: `/v3/domains` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:18:04.552185 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03cecb90-418c-488f-b063-8c7fe2df4f52 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80377) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 02 10:18:04.552493 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03cecb90-418c-488f-b063-8c7fe2df4f52 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:18:04.554217 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03cecb90-418c-488f-b063-8c7fe2df4f52 admin admin] RBAC: Authorization granted {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:18:04.566016 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3af9838c-40b9-4f1c-a7f5-00e0d3f4ce06 None None] REQUEST_METHOD: `POST` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:18:04.566016 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3af9838c-40b9-4f1c-a7f5-00e0d3f4ce06 None None] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:18:04.566016 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3af9838c-40b9-4f1c-a7f5-00e0d3f4ce06 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:18:04.580168 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: [pid: 80377|app: 0|req: 1149/2283] 10.222.0.82 () {58 vars in 1153 bytes} [Tue Aug 2 10:18:04 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 55 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 02 10:18:04.586052 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03cecb90-418c-488f-b063-8c7fe2df4f52 admin admin] Authenticating user token {{(pid=80377) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:18:04.591322 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.auth.core [None req-3af9838c-40b9-4f1c-a7f5-00e0d3f4ce06 None None] MFA Rules not processed for user `54bb78d9af854abf845b47f2bb925b74`. Rule list: `[]` (Enabled: `True`). {{(pid=80378) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 02 10:18:04.593892 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.fernet_utils [None req-3af9838c-40b9-4f1c-a7f5-00e0d3f4ce06 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80378) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 10:18:04.599642 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03cecb90-418c-488f-b063-8c7fe2df4f52 admin admin] Validating token access rules against request {{(pid=80377) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:18:04.614320 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4e1c4f0-8e10-4125-9aa7-101c4173fba1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '891005fa5ff74041b75747d16cc66798', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7047912c5acd4adeba5d87c3bf895e39', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80377) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:18:04.614960 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4e1c4f0-8e10-4125-9aa7-101c4173fba1 admin admin] REQUEST_METHOD: `POST` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:18:04.615878 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4e1c4f0-8e10-4125-9aa7-101c4173fba1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:18:04.616167 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4e1c4f0-8e10-4125-9aa7-101c4173fba1 admin admin] PATH_INFO: `/v3/projects` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:18:04.616993 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4e1c4f0-8e10-4125-9aa7-101c4173fba1 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:18:04.622749 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4e1c4f0-8e10-4125-9aa7-101c4173fba1 admin admin] RBAC: Authorization granted {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:18:04.700275 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: [pid: 80377|app: 0|req: 1150/2284] 10.222.0.82 () {60 vars in 1153 bytes} [Tue Aug 2 10:18:04 2022] POST /identity/v3/projects => generated 385 bytes in 115 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 02 10:18:04.708105 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4e1c4f0-8e10-4125-9aa7-101c4173fba1 admin admin] Authenticating user token {{(pid=80377) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:18:04.708583 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 1135/2285] 10.222.0.82 () {58 vars in 958 bytes} [Tue Aug 2 10:18:04 2022] POST /identity/v3/auth/tokens => generated 3262 bytes in 142 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 02 10:18:04.789024 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4e1c4f0-8e10-4125-9aa7-101c4173fba1 admin admin] Validating token access rules against request {{(pid=80377) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:18:04.802972 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5abcf56-fa84-4260-9b64-4772bd6fdab9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '891005fa5ff74041b75747d16cc66798', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7047912c5acd4adeba5d87c3bf895e39', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80377) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:18:04.803702 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5abcf56-fa84-4260-9b64-4772bd6fdab9 admin admin] REQUEST_METHOD: `POST` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:18:04.803702 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5abcf56-fa84-4260-9b64-4772bd6fdab9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:18:04.803932 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5abcf56-fa84-4260-9b64-4772bd6fdab9 admin admin] PATH_INFO: `/v3/users` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:18:04.804727 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5abcf56-fa84-4260-9b64-4772bd6fdab9 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:18:04.807006 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5abcf56-fa84-4260-9b64-4772bd6fdab9 admin admin] RBAC: Authorization granted {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:18:04.850329 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: [pid: 80377|app: 0|req: 1151/2286] 10.222.0.82 () {60 vars in 1144 bytes} [Tue Aug 2 10:18:04 2022] POST /identity/v3/users => generated 293 bytes in 143 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 02 10:18:04.854427 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3af9838c-40b9-4f1c-a7f5-00e0d3f4ce06 None None] Authenticating user token {{(pid=80378) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:18:04.869813 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3af9838c-40b9-4f1c-a7f5-00e0d3f4ce06 None None] Validating token access rules against request {{(pid=80378) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:18:04.893454 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee0e04c2-a456-4ce2-b52a-376bbd48c722 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '891005fa5ff74041b75747d16cc66798', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7047912c5acd4adeba5d87c3bf895e39', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80378) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:18:04.902417 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee0e04c2-a456-4ce2-b52a-376bbd48c722 admin admin] REQUEST_METHOD: `GET` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:18:04.902967 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee0e04c2-a456-4ce2-b52a-376bbd48c722 admin admin] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:18:04.903300 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee0e04c2-a456-4ce2-b52a-376bbd48c722 admin admin] PATH_INFO: `/v3/roles` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:18:04.904164 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee0e04c2-a456-4ce2-b52a-376bbd48c722 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:18:04.906425 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee0e04c2-a456-4ce2-b52a-376bbd48c722 admin admin] RBAC: Authorization granted {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:18:04.920480 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 1136/2287] 10.222.0.82 () {58 vars in 1122 bytes} [Tue Aug 2 10:18:04 2022] GET /identity/v3/roles => generated 1420 bytes in 66 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 02 10:18:04.924367 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5abcf56-fa84-4260-9b64-4772bd6fdab9 admin admin] Authenticating user token {{(pid=80377) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:18:04.935549 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5abcf56-fa84-4260-9b64-4772bd6fdab9 admin admin] Validating token access rules against request {{(pid=80377) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:18:04.941950 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14767a75-5ecd-4a73-80e2-cfe07292521b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '891005fa5ff74041b75747d16cc66798', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7047912c5acd4adeba5d87c3bf895e39', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80377) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:18:04.941950 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14767a75-5ecd-4a73-80e2-cfe07292521b admin admin] REQUEST_METHOD: `GET` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:18:04.942302 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14767a75-5ecd-4a73-80e2-cfe07292521b admin admin] SCRIPT_NAME: `/identity` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:18:04.942476 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14767a75-5ecd-4a73-80e2-cfe07292521b admin admin] PATH_INFO: `/v3/roles` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:18:04.943188 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14767a75-5ecd-4a73-80e2-cfe07292521b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:18:04.944938 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14767a75-5ecd-4a73-80e2-cfe07292521b admin admin] RBAC: Authorization granted {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:18:04.968735 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: [pid: 80377|app: 0|req: 1152/2288] 10.222.0.82 () {58 vars in 1122 bytes} [Tue Aug 2 10:18:04 2022] GET /identity/v3/roles => generated 1420 bytes in 45 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 02 10:18:04.973054 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee0e04c2-a456-4ce2-b52a-376bbd48c722 admin admin] Authenticating user token {{(pid=80378) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:18:04.984603 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee0e04c2-a456-4ce2-b52a-376bbd48c722 admin admin] Validating token access rules against request {{(pid=80378) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:18:04.992249 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea4a6dd5-d691-44a4-b3fb-4016c663b4d4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '891005fa5ff74041b75747d16cc66798', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7047912c5acd4adeba5d87c3bf895e39', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80378) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:18:04.992941 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea4a6dd5-d691-44a4-b3fb-4016c663b4d4 admin admin] REQUEST_METHOD: `PUT` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:18:04.993250 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea4a6dd5-d691-44a4-b3fb-4016c663b4d4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:18:04.993551 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea4a6dd5-d691-44a4-b3fb-4016c663b4d4 admin admin] PATH_INFO: `/v3/projects/742c5fd5ee6a4dd7b068aa83f5079913/users/b1baaecbb9b44dfdaa481605b1103112/roles/e1c04b19fc4543b8b54f58bbcffbc598` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:18:05.026870 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea4a6dd5-d691-44a4-b3fb-4016c663b4d4 admin admin] RBAC: Authorizing `identity:create_grant(project_id=742c5fd5ee6a4dd7b068aa83f5079913, user_id=b1baaecbb9b44dfdaa481605b1103112, role_id=e1c04b19fc4543b8b54f58bbcffbc598)` {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:18:05.029985 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea4a6dd5-d691-44a4-b3fb-4016c663b4d4 admin admin] RBAC: Authorization granted {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:18:05.044496 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG dbcounter [-] [80378] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80378) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 10:18:05.050332 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 1137/2289] 10.222.0.82 () {60 vars in 1483 bytes} [Tue Aug 2 10:18:04 2022] PUT /identity/v3/projects/742c5fd5ee6a4dd7b068aa83f5079913/users/b1baaecbb9b44dfdaa481605b1103112/roles/e1c04b19fc4543b8b54f58bbcffbc598 => generated 0 bytes in 78 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 02 10:18:05.063365 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b7067dd-a544-4e83-bb84-0f5ca3017d1b None None] REQUEST_METHOD: `POST` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:18:05.063599 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b7067dd-a544-4e83-bb84-0f5ca3017d1b None None] SCRIPT_NAME: `/identity` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:18:05.063905 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b7067dd-a544-4e83-bb84-0f5ca3017d1b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:18:05.106024 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.auth.core [None req-2b7067dd-a544-4e83-bb84-0f5ca3017d1b None None] MFA Rules not processed for user `b1baaecbb9b44dfdaa481605b1103112`. Rule list: `[]` (Enabled: `True`). {{(pid=80377) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 02 10:18:05.107310 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.fernet_utils [None req-2b7067dd-a544-4e83-bb84-0f5ca3017d1b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80377) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 10:18:05.162549 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG dbcounter [-] [80377] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=80377) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 10:18:05.195142 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: [pid: 80377|app: 0|req: 1153/2290] 10.222.0.82 () {58 vars in 958 bytes} [Tue Aug 2 10:18:05 2022] POST /identity/v3/auth/tokens => generated 3262 bytes in 133 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 02 10:18:05.199390 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea4a6dd5-d691-44a4-b3fb-4016c663b4d4 admin admin] Authenticating user token {{(pid=80378) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:18:05.278111 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea4a6dd5-d691-44a4-b3fb-4016c663b4d4 admin admin] Validating token access rules against request {{(pid=80378) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:18:05.283717 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-917a16cf-0631-4b34-9a85-a564d61f98c1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '891005fa5ff74041b75747d16cc66798', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7047912c5acd4adeba5d87c3bf895e39', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80378) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:18:05.284351 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-917a16cf-0631-4b34-9a85-a564d61f98c1 admin admin] REQUEST_METHOD: `POST` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:18:05.284633 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-917a16cf-0631-4b34-9a85-a564d61f98c1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:18:05.284909 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-917a16cf-0631-4b34-9a85-a564d61f98c1 admin admin] PATH_INFO: `/v3/projects` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:18:05.285595 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-917a16cf-0631-4b34-9a85-a564d61f98c1 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:18:05.287575 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-917a16cf-0631-4b34-9a85-a564d61f98c1 admin admin] RBAC: Authorization granted {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:18:05.324952 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 1138/2291] 10.222.0.82 () {60 vars in 1153 bytes} [Tue Aug 2 10:18:05 2022] POST /identity/v3/projects => generated 387 bytes in 126 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 02 10:18:05.328842 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b7067dd-a544-4e83-bb84-0f5ca3017d1b None None] Authenticating user token {{(pid=80377) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:18:05.343474 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b7067dd-a544-4e83-bb84-0f5ca3017d1b None None] Validating token access rules against request {{(pid=80377) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:18:05.428071 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-761a3ca7-0fbf-42e2-b0a0-524530df1e66 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '891005fa5ff74041b75747d16cc66798', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7047912c5acd4adeba5d87c3bf895e39', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80377) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:18:05.428772 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-761a3ca7-0fbf-42e2-b0a0-524530df1e66 admin admin] REQUEST_METHOD: `POST` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:18:05.429050 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-761a3ca7-0fbf-42e2-b0a0-524530df1e66 admin admin] SCRIPT_NAME: `/identity` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:18:05.429332 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-761a3ca7-0fbf-42e2-b0a0-524530df1e66 admin admin] PATH_INFO: `/v3/users` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:18:05.434619 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-761a3ca7-0fbf-42e2-b0a0-524530df1e66 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:18:05.437624 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-761a3ca7-0fbf-42e2-b0a0-524530df1e66 admin admin] RBAC: Authorization granted {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:18:05.472833 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: [pid: 80377|app: 0|req: 1154/2292] 10.222.0.82 () {60 vars in 1144 bytes} [Tue Aug 2 10:18:05 2022] POST /identity/v3/users => generated 300 bytes in 144 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 02 10:18:05.476849 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-917a16cf-0631-4b34-9a85-a564d61f98c1 admin admin] Authenticating user token {{(pid=80378) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:18:05.487584 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-917a16cf-0631-4b34-9a85-a564d61f98c1 admin admin] Validating token access rules against request {{(pid=80378) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:18:05.492560 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bce1b90-2113-486f-b95d-71d1f4d6850f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '891005fa5ff74041b75747d16cc66798', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7047912c5acd4adeba5d87c3bf895e39', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80378) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:18:05.493292 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bce1b90-2113-486f-b95d-71d1f4d6850f admin admin] REQUEST_METHOD: `GET` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:18:05.493642 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bce1b90-2113-486f-b95d-71d1f4d6850f admin admin] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:18:05.494018 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bce1b90-2113-486f-b95d-71d1f4d6850f admin admin] PATH_INFO: `/v3/roles` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:18:05.494785 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5bce1b90-2113-486f-b95d-71d1f4d6850f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:18:05.496459 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5bce1b90-2113-486f-b95d-71d1f4d6850f admin admin] RBAC: Authorization granted {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:18:05.512875 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 1139/2293] 10.222.0.82 () {58 vars in 1122 bytes} [Tue Aug 2 10:18:05 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 02 10:18:05.514465 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-761a3ca7-0fbf-42e2-b0a0-524530df1e66 admin admin] Authenticating user token {{(pid=80377) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:18:05.525010 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-761a3ca7-0fbf-42e2-b0a0-524530df1e66 admin admin] Validating token access rules against request {{(pid=80377) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:18:05.530609 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f31290a7-418c-43de-8a64-447d259fd474 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '891005fa5ff74041b75747d16cc66798', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7047912c5acd4adeba5d87c3bf895e39', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80377) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:18:05.531281 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f31290a7-418c-43de-8a64-447d259fd474 admin admin] REQUEST_METHOD: `PUT` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:18:05.531592 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f31290a7-418c-43de-8a64-447d259fd474 admin admin] SCRIPT_NAME: `/identity` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:18:05.531868 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f31290a7-418c-43de-8a64-447d259fd474 admin admin] PATH_INFO: `/v3/projects/6138f20cbe994f3997367e03aaa2f212/users/83bda1637c8a43bcaf4071ab61df4c7e/roles/237966442f05411191105bf336bc14f3` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:18:05.547760 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f31290a7-418c-43de-8a64-447d259fd474 admin admin] RBAC: Authorizing `identity:create_grant(project_id=6138f20cbe994f3997367e03aaa2f212, user_id=83bda1637c8a43bcaf4071ab61df4c7e, role_id=237966442f05411191105bf336bc14f3)` {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:18:05.550749 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f31290a7-418c-43de-8a64-447d259fd474 admin admin] RBAC: Authorization granted {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:18:05.565672 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: [pid: 80377|app: 0|req: 1155/2294] 10.222.0.82 () {60 vars in 1483 bytes} [Tue Aug 2 10:18:05 2022] PUT /identity/v3/projects/6138f20cbe994f3997367e03aaa2f212/users/83bda1637c8a43bcaf4071ab61df4c7e/roles/237966442f05411191105bf336bc14f3 => generated 0 bytes in 53 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 02 10:18:05.570305 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bce1b90-2113-486f-b95d-71d1f4d6850f admin admin] Authenticating user token {{(pid=80378) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:18:05.582087 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bce1b90-2113-486f-b95d-71d1f4d6850f admin admin] Validating token access rules against request {{(pid=80378) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:18:05.652187 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-546130d3-61f8-49d7-b51c-af98f8a5eaff admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '891005fa5ff74041b75747d16cc66798', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7047912c5acd4adeba5d87c3bf895e39', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80378) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:18:05.653091 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-546130d3-61f8-49d7-b51c-af98f8a5eaff admin admin] REQUEST_METHOD: `GET` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:18:05.653381 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-546130d3-61f8-49d7-b51c-af98f8a5eaff admin admin] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:18:05.653684 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-546130d3-61f8-49d7-b51c-af98f8a5eaff admin admin] PATH_INFO: `/v3/roles` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:18:05.654627 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-546130d3-61f8-49d7-b51c-af98f8a5eaff admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:18:05.656265 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-546130d3-61f8-49d7-b51c-af98f8a5eaff admin admin] RBAC: Authorization granted {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:18:05.666601 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 1140/2295] 10.222.0.82 () {58 vars in 1122 bytes} [Tue Aug 2 10:18:05 2022] GET /identity/v3/roles => generated 1420 bytes in 97 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 02 10:18:05.671193 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f31290a7-418c-43de-8a64-447d259fd474 admin admin] Authenticating user token {{(pid=80377) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:18:05.682786 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f31290a7-418c-43de-8a64-447d259fd474 admin admin] Validating token access rules against request {{(pid=80377) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:18:05.689644 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c669313-6da1-469b-9a8b-aa5975b2caf7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '891005fa5ff74041b75747d16cc66798', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7047912c5acd4adeba5d87c3bf895e39', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80377) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:18:05.690457 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c669313-6da1-469b-9a8b-aa5975b2caf7 admin admin] REQUEST_METHOD: `PUT` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:18:05.690678 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c669313-6da1-469b-9a8b-aa5975b2caf7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:18:05.690950 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c669313-6da1-469b-9a8b-aa5975b2caf7 admin admin] PATH_INFO: `/v3/projects/6138f20cbe994f3997367e03aaa2f212/users/83bda1637c8a43bcaf4071ab61df4c7e/roles/237966442f05411191105bf336bc14f3` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:18:05.692973 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c669313-6da1-469b-9a8b-aa5975b2caf7 admin admin] RBAC: Authorizing `identity:create_grant(project_id=6138f20cbe994f3997367e03aaa2f212, user_id=83bda1637c8a43bcaf4071ab61df4c7e, role_id=237966442f05411191105bf336bc14f3)` {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:18:05.695819 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c669313-6da1-469b-9a8b-aa5975b2caf7 admin admin] RBAC: Authorization granted {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:18:05.707569 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: [pid: 80377|app: 0|req: 1156/2296] 10.222.0.82 () {60 vars in 1483 bytes} [Tue Aug 2 10:18:05 2022] PUT /identity/v3/projects/6138f20cbe994f3997367e03aaa2f212/users/83bda1637c8a43bcaf4071ab61df4c7e/roles/237966442f05411191105bf336bc14f3 => generated 0 bytes in 37 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 02 10:18:05.718644 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8365d998-bd97-4209-9514-80eb72b16e99 None None] REQUEST_METHOD: `POST` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:18:05.718938 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8365d998-bd97-4209-9514-80eb72b16e99 None None] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:18:05.719217 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8365d998-bd97-4209-9514-80eb72b16e99 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:18:05.752188 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.auth.core [None req-8365d998-bd97-4209-9514-80eb72b16e99 None None] MFA Rules not processed for user `83bda1637c8a43bcaf4071ab61df4c7e`. Rule list: `[]` (Enabled: `True`). {{(pid=80378) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 02 10:18:05.753521 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.fernet_utils [None req-8365d998-bd97-4209-9514-80eb72b16e99 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80378) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 10:18:05.847177 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG dbcounter [-] [80378] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80378) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 10:18:05.863604 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 1141/2297] 10.222.0.82 () {58 vars in 958 bytes} [Tue Aug 2 10:18:05 2022] POST /identity/v3/auth/tokens => generated 3331 bytes in 146 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 02 10:18:05.949874 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c669313-6da1-469b-9a8b-aa5975b2caf7 admin admin] Authenticating user token {{(pid=80377) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:18:05.951677 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.fernet_utils [None req-3c669313-6da1-469b-9a8b-aa5975b2caf7 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80377) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 10:18:06.033384 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c669313-6da1-469b-9a8b-aa5975b2caf7 admin admin] Validating token access rules against request {{(pid=80377) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:18:06.044313 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ddae8feb-5e1a-4bb6-ac11-452e398b4def service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1023de281ae74e9d8a6d6c312c1c97a7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f1b38ebb804c41c69772c998aae72c41', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80377) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:18:06.045926 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddae8feb-5e1a-4bb6-ac11-452e398b4def service cinder] REQUEST_METHOD: `GET` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:18:06.045926 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddae8feb-5e1a-4bb6-ac11-452e398b4def service cinder] SCRIPT_NAME: `/identity` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:18:06.045926 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddae8feb-5e1a-4bb6-ac11-452e398b4def service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:18:06.071588 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ddae8feb-5e1a-4bb6-ac11-452e398b4def service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:18:06.073052 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ddae8feb-5e1a-4bb6-ac11-452e398b4def service cinder] RBAC: Authorization granted {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:18:06.088355 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: [pid: 80377|app: 0|req: 1157/2298] 10.222.0.82 () {60 vars in 1365 bytes} [Tue Aug 2 10:18:05 2022] GET /identity/v3/auth/tokens => generated 3331 bytes in 139 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 02 10:18:06.341977 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8365d998-bd97-4209-9514-80eb72b16e99 None None] Authenticating user token {{(pid=80378) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:18:06.454436 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8365d998-bd97-4209-9514-80eb72b16e99 None None] Validating token access rules against request {{(pid=80378) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:18:06.459231 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7191551c-8bbb-4eeb-afad-e4ea39570f37 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '891005fa5ff74041b75747d16cc66798', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7047912c5acd4adeba5d87c3bf895e39', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80378) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:18:06.460020 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7191551c-8bbb-4eeb-afad-e4ea39570f37 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:18:06.460305 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7191551c-8bbb-4eeb-afad-e4ea39570f37 admin admin] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:18:06.460583 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7191551c-8bbb-4eeb-afad-e4ea39570f37 admin admin] PATH_INFO: `/v3/users/b1baaecbb9b44dfdaa481605b1103112` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:18:06.461826 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7191551c-8bbb-4eeb-afad-e4ea39570f37 admin admin] RBAC: Authorizing `identity:delete_user(user_id=b1baaecbb9b44dfdaa481605b1103112)` {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:18:06.463819 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7191551c-8bbb-4eeb-afad-e4ea39570f37 admin admin] RBAC: Authorization granted {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:18:06.532095 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.notifications [None req-7191551c-8bbb-4eeb-afad-e4ea39570f37 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'b1baaecbb9b44dfdaa481605b1103112'} {{(pid=80378) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 10:18:06.540858 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.notifications [None req-7191551c-8bbb-4eeb-afad-e4ea39570f37 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'b1baaecbb9b44dfdaa481605b1103112'} {{(pid=80378) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 10:18:06.548889 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.notifications [None req-7191551c-8bbb-4eeb-afad-e4ea39570f37 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'b1baaecbb9b44dfdaa481605b1103112'} {{(pid=80378) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 10:18:06.549608 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.notifications [None req-7191551c-8bbb-4eeb-afad-e4ea39570f37 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'b1baaecbb9b44dfdaa481605b1103112'} {{(pid=80378) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 10:18:06.564385 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.notifications [None req-7191551c-8bbb-4eeb-afad-e4ea39570f37 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'b1baaecbb9b44dfdaa481605b1103112'} {{(pid=80378) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 10:18:06.573813 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 1142/2299] 10.222.0.82 () {58 vars in 1224 bytes} [Tue Aug 2 10:18:06 2022] DELETE /identity/v3/users/b1baaecbb9b44dfdaa481605b1103112 => generated 0 bytes in 230 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 02 10:18:06.577300 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77bf8426-fe86-4602-b8b5-09c3563c8f54 None None] REQUEST_METHOD: `POST` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:18:06.577589 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77bf8426-fe86-4602-b8b5-09c3563c8f54 None None] SCRIPT_NAME: `/identity` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:18:06.577958 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77bf8426-fe86-4602-b8b5-09c3563c8f54 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:18:06.599927 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.auth.core [None req-77bf8426-fe86-4602-b8b5-09c3563c8f54 None None] MFA Rules not processed for user `891005fa5ff74041b75747d16cc66798`. Rule list: `[]` (Enabled: `True`). {{(pid=80377) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 02 10:18:06.601390 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.fernet_utils [None req-77bf8426-fe86-4602-b8b5-09c3563c8f54 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80377) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 10:18:06.649211 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG dbcounter [-] [80377] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80377) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 10:18:06.675455 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: [pid: 80377|app: 0|req: 1158/2300] 10.222.0.82 () {58 vars in 958 bytes} [Tue Aug 2 10:18:06 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 02 10:18:06.686401 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7191551c-8bbb-4eeb-afad-e4ea39570f37 admin admin] Authenticating user token {{(pid=80378) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:18:06.687877 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.fernet_utils [None req-7191551c-8bbb-4eeb-afad-e4ea39570f37 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80378) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 10:18:06.754870 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7191551c-8bbb-4eeb-afad-e4ea39570f37 admin admin] Validating token access rules against request {{(pid=80378) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:18:06.758615 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e47c0e63-816c-43ee-84cc-077d3c3f16dd service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '33b87170f0634b67bce60750847b5cbe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f1b38ebb804c41c69772c998aae72c41', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80378) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:18:06.759245 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e47c0e63-816c-43ee-84cc-077d3c3f16dd service neutron] REQUEST_METHOD: `GET` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:18:06.759531 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e47c0e63-816c-43ee-84cc-077d3c3f16dd service neutron] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:18:06.759855 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e47c0e63-816c-43ee-84cc-077d3c3f16dd service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:18:06.770317 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e47c0e63-816c-43ee-84cc-077d3c3f16dd service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:18:06.771482 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e47c0e63-816c-43ee-84cc-077d3c3f16dd service neutron] RBAC: Authorization granted {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:18:06.782323 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 1143/2301] 10.222.0.82 () {60 vars in 1365 bytes} [Tue Aug 2 10:18:06 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 97 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 02 10:18:07.078773 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77bf8426-fe86-4602-b8b5-09c3563c8f54 None None] Authenticating user token {{(pid=80377) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:18:07.080313 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.fernet_utils [None req-77bf8426-fe86-4602-b8b5-09c3563c8f54 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80377) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 10:18:07.098589 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77bf8426-fe86-4602-b8b5-09c3563c8f54 None None] Validating token access rules against request {{(pid=80377) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:18:07.104420 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7328d4b-298c-4097-b1ca-294ac13f439d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '891005fa5ff74041b75747d16cc66798', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7047912c5acd4adeba5d87c3bf895e39', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80377) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:18:07.105306 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7328d4b-298c-4097-b1ca-294ac13f439d admin admin] REQUEST_METHOD: `DELETE` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:18:07.105649 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7328d4b-298c-4097-b1ca-294ac13f439d admin admin] SCRIPT_NAME: `/identity` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:18:07.106087 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7328d4b-298c-4097-b1ca-294ac13f439d admin admin] PATH_INFO: `/v3/projects/742c5fd5ee6a4dd7b068aa83f5079913` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:18:07.107507 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7328d4b-298c-4097-b1ca-294ac13f439d admin admin] RBAC: Authorizing `identity:delete_project(project_id=742c5fd5ee6a4dd7b068aa83f5079913)` {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:18:07.110797 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7328d4b-298c-4097-b1ca-294ac13f439d admin admin] RBAC: Authorization granted {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:18:07.161605 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.notifications [None req-e7328d4b-298c-4097-b1ca-294ac13f439d admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '742c5fd5ee6a4dd7b068aa83f5079913', 'request_id': 'req-e7328d4b-298c-4097-b1ca-294ac13f439d'} {{(pid=80377) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 10:18:07.168198 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.notifications [None req-e7328d4b-298c-4097-b1ca-294ac13f439d admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '742c5fd5ee6a4dd7b068aa83f5079913', 'request_id': 'req-e7328d4b-298c-4097-b1ca-294ac13f439d'} {{(pid=80377) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 10:18:07.175950 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.notifications [None req-e7328d4b-298c-4097-b1ca-294ac13f439d admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '742c5fd5ee6a4dd7b068aa83f5079913', 'request_id': 'req-e7328d4b-298c-4097-b1ca-294ac13f439d'} {{(pid=80377) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 10:18:07.191291 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.notifications [None req-e7328d4b-298c-4097-b1ca-294ac13f439d admin admin] The token cache is being invalidate because project 742c5fd5ee6a4dd7b068aa83f5079913 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80377) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 02 10:18:07.191581 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.notifications [None req-e7328d4b-298c-4097-b1ca-294ac13f439d admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80377) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 10:18:07.192211 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.notifications [None req-e7328d4b-298c-4097-b1ca-294ac13f439d admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80377) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 10:18:07.193840 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: [pid: 80377|app: 0|req: 1159/2302] 10.222.0.82 () {58 vars in 1233 bytes} [Tue Aug 2 10:18:07 2022] DELETE /identity/v3/projects/742c5fd5ee6a4dd7b068aa83f5079913 => generated 0 bytes in 116 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 02 10:18:07.198171 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e47c0e63-816c-43ee-84cc-077d3c3f16dd service neutron] Authenticating user token {{(pid=80378) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:18:07.208935 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e47c0e63-816c-43ee-84cc-077d3c3f16dd service neutron] Validating token access rules against request {{(pid=80378) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:18:07.224689 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG dbcounter [-] [80378] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80378) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 10:18:07.276226 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b649b8e6-21aa-49ef-9da9-8cabf29bc780 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '891005fa5ff74041b75747d16cc66798', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7047912c5acd4adeba5d87c3bf895e39', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80378) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:18:07.276869 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b649b8e6-21aa-49ef-9da9-8cabf29bc780 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:18:07.277153 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b649b8e6-21aa-49ef-9da9-8cabf29bc780 admin admin] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:18:07.277424 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b649b8e6-21aa-49ef-9da9-8cabf29bc780 admin admin] PATH_INFO: `/v3/users/83bda1637c8a43bcaf4071ab61df4c7e` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:18:07.278669 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b649b8e6-21aa-49ef-9da9-8cabf29bc780 admin admin] RBAC: Authorizing `identity:delete_user(user_id=83bda1637c8a43bcaf4071ab61df4c7e)` {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:18:07.280766 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b649b8e6-21aa-49ef-9da9-8cabf29bc780 admin admin] RBAC: Authorization granted {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:18:07.347180 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.notifications [None req-b649b8e6-21aa-49ef-9da9-8cabf29bc780 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '83bda1637c8a43bcaf4071ab61df4c7e'} {{(pid=80378) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 10:18:07.358122 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.notifications [None req-b649b8e6-21aa-49ef-9da9-8cabf29bc780 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '83bda1637c8a43bcaf4071ab61df4c7e'} {{(pid=80378) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 10:18:07.369288 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.notifications [None req-b649b8e6-21aa-49ef-9da9-8cabf29bc780 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '83bda1637c8a43bcaf4071ab61df4c7e'} {{(pid=80378) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 10:18:07.370298 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.notifications [None req-b649b8e6-21aa-49ef-9da9-8cabf29bc780 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '83bda1637c8a43bcaf4071ab61df4c7e'} {{(pid=80378) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 10:18:07.384310 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.notifications [None req-b649b8e6-21aa-49ef-9da9-8cabf29bc780 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '83bda1637c8a43bcaf4071ab61df4c7e'} {{(pid=80378) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 10:18:07.392374 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 1144/2303] 10.222.0.82 () {58 vars in 1224 bytes} [Tue Aug 2 10:18:07 2022] DELETE /identity/v3/users/83bda1637c8a43bcaf4071ab61df4c7e => generated 0 bytes in 195 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 02 10:18:07.554582 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7328d4b-298c-4097-b1ca-294ac13f439d admin admin] Authenticating user token {{(pid=80377) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:18:07.556522 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.fernet_utils [None req-e7328d4b-298c-4097-b1ca-294ac13f439d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80377) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 10:18:07.570751 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7328d4b-298c-4097-b1ca-294ac13f439d admin admin] Validating token access rules against request {{(pid=80377) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:18:07.643774 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca38b7c5-c106-45dd-97af-cdad2a4e6466 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '891005fa5ff74041b75747d16cc66798', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7047912c5acd4adeba5d87c3bf895e39', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80377) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:18:07.644442 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca38b7c5-c106-45dd-97af-cdad2a4e6466 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:18:07.644728 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca38b7c5-c106-45dd-97af-cdad2a4e6466 admin admin] SCRIPT_NAME: `/identity` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:18:07.645008 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca38b7c5-c106-45dd-97af-cdad2a4e6466 admin admin] PATH_INFO: `/v3/users/56af49202378415f88d3310ebfe4faab` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:18:07.646298 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca38b7c5-c106-45dd-97af-cdad2a4e6466 admin admin] RBAC: Authorizing `identity:delete_user(user_id=56af49202378415f88d3310ebfe4faab)` {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:18:07.648219 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca38b7c5-c106-45dd-97af-cdad2a4e6466 admin admin] RBAC: Authorization granted {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:18:07.702532 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b649b8e6-21aa-49ef-9da9-8cabf29bc780 admin admin] Authenticating user token {{(pid=80378) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:18:07.704348 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.fernet_utils [None req-b649b8e6-21aa-49ef-9da9-8cabf29bc780 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80378) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 10:18:07.720267 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.notifications [None req-ca38b7c5-c106-45dd-97af-cdad2a4e6466 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '56af49202378415f88d3310ebfe4faab'} {{(pid=80377) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 10:18:07.733255 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.notifications [None req-ca38b7c5-c106-45dd-97af-cdad2a4e6466 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '56af49202378415f88d3310ebfe4faab'} {{(pid=80377) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 10:18:07.741516 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.notifications [None req-ca38b7c5-c106-45dd-97af-cdad2a4e6466 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '56af49202378415f88d3310ebfe4faab'} {{(pid=80377) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 10:18:07.742125 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.notifications [None req-ca38b7c5-c106-45dd-97af-cdad2a4e6466 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '56af49202378415f88d3310ebfe4faab'} {{(pid=80377) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 10:18:07.743054 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b649b8e6-21aa-49ef-9da9-8cabf29bc780 admin admin] Validating token access rules against request {{(pid=80378) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:18:07.744213 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG dbcounter [-] [80377] Writing DB stats keystone:SELECT=83,keystone:DELETE=15,keystone:INSERT=2 {{(pid=80377) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 10:18:07.752273 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15d1ca7d-10f4-4e24-b73c-f3a1f9571ef7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '891005fa5ff74041b75747d16cc66798', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7047912c5acd4adeba5d87c3bf895e39', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80378) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:18:07.753376 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15d1ca7d-10f4-4e24-b73c-f3a1f9571ef7 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:18:07.753677 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15d1ca7d-10f4-4e24-b73c-f3a1f9571ef7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:18:07.754128 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15d1ca7d-10f4-4e24-b73c-f3a1f9571ef7 admin admin] PATH_INFO: `/v3/projects/6138f20cbe994f3997367e03aaa2f212` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:18:07.755235 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15d1ca7d-10f4-4e24-b73c-f3a1f9571ef7 admin admin] RBAC: Authorizing `identity:delete_project(project_id=6138f20cbe994f3997367e03aaa2f212)` {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:18:07.757155 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15d1ca7d-10f4-4e24-b73c-f3a1f9571ef7 admin admin] RBAC: Authorization granted {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:18:07.763977 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.notifications [None req-ca38b7c5-c106-45dd-97af-cdad2a4e6466 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '56af49202378415f88d3310ebfe4faab'} {{(pid=80377) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 10:18:07.770917 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: [pid: 80377|app: 0|req: 1160/2304] 10.222.0.82 () {58 vars in 1224 bytes} [Tue Aug 2 10:18:07 2022] DELETE /identity/v3/users/56af49202378415f88d3310ebfe4faab => generated 0 bytes in 217 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 02 10:18:07.775122 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7499a15-87a2-4bbd-8ce1-7ed71e77a874 None None] REQUEST_METHOD: `POST` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:18:07.775407 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7499a15-87a2-4bbd-8ce1-7ed71e77a874 None None] SCRIPT_NAME: `/identity` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:18:07.775690 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7499a15-87a2-4bbd-8ce1-7ed71e77a874 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:18:07.796084 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.auth.core [None req-d7499a15-87a2-4bbd-8ce1-7ed71e77a874 None None] MFA Rules not processed for user `891005fa5ff74041b75747d16cc66798`. Rule list: `[]` (Enabled: `True`). {{(pid=80377) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 02 10:18:07.797304 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.fernet_utils [None req-d7499a15-87a2-4bbd-8ce1-7ed71e77a874 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80377) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 10:18:07.814603 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.notifications [None req-15d1ca7d-10f4-4e24-b73c-f3a1f9571ef7 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '6138f20cbe994f3997367e03aaa2f212', 'request_id': 'req-15d1ca7d-10f4-4e24-b73c-f3a1f9571ef7'} {{(pid=80378) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 10:18:07.821040 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.notifications [None req-15d1ca7d-10f4-4e24-b73c-f3a1f9571ef7 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '6138f20cbe994f3997367e03aaa2f212', 'request_id': 'req-15d1ca7d-10f4-4e24-b73c-f3a1f9571ef7'} {{(pid=80378) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 10:18:07.827419 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.notifications [None req-15d1ca7d-10f4-4e24-b73c-f3a1f9571ef7 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '6138f20cbe994f3997367e03aaa2f212', 'request_id': 'req-15d1ca7d-10f4-4e24-b73c-f3a1f9571ef7'} {{(pid=80378) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 10:18:07.841167 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.notifications [None req-15d1ca7d-10f4-4e24-b73c-f3a1f9571ef7 admin admin] The token cache is being invalidate because project 6138f20cbe994f3997367e03aaa2f212 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80378) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 02 10:18:07.842528 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.notifications [None req-15d1ca7d-10f4-4e24-b73c-f3a1f9571ef7 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80378) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 10:18:07.848165 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.notifications [None req-15d1ca7d-10f4-4e24-b73c-f3a1f9571ef7 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80378) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 10:18:07.848925 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 1145/2305] 10.222.0.82 () {58 vars in 1233 bytes} [Tue Aug 2 10:18:07 2022] DELETE /identity/v3/projects/6138f20cbe994f3997367e03aaa2f212 => generated 0 bytes in 147 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 02 10:18:07.854960 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12525d84-7ebe-41fc-99fa-1389ea48fc70 None None] REQUEST_METHOD: `POST` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:18:07.855246 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12525d84-7ebe-41fc-99fa-1389ea48fc70 None None] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:18:07.855886 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12525d84-7ebe-41fc-99fa-1389ea48fc70 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:18:07.869874 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: [pid: 80377|app: 0|req: 1161/2306] 10.222.0.82 () {58 vars in 958 bytes} [Tue Aug 2 10:18:07 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 02 10:18:07.879599 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.auth.core [None req-12525d84-7ebe-41fc-99fa-1389ea48fc70 None None] MFA Rules not processed for user `891005fa5ff74041b75747d16cc66798`. Rule list: `[]` (Enabled: `True`). {{(pid=80378) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 02 10:18:07.879599 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.fernet_utils [None req-12525d84-7ebe-41fc-99fa-1389ea48fc70 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80378) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 10:18:07.880927 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7499a15-87a2-4bbd-8ce1-7ed71e77a874 None None] Authenticating user token {{(pid=80377) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:18:07.882826 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.fernet_utils [None req-d7499a15-87a2-4bbd-8ce1-7ed71e77a874 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80377) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 10:18:07.886144 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 1146/2307] 10.222.0.82 () {58 vars in 958 bytes} [Tue Aug 2 10:18:07 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 02 10:18:07.895526 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f2ffe8c-643a-4aac-aac2-33165d972c0d None None] REQUEST_METHOD: `POST` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:18:07.895843 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f2ffe8c-643a-4aac-aac2-33165d972c0d None None] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:18:07.896107 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f2ffe8c-643a-4aac-aac2-33165d972c0d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:18:07.916860 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.auth.core [None req-0f2ffe8c-643a-4aac-aac2-33165d972c0d None None] MFA Rules not processed for user `891005fa5ff74041b75747d16cc66798`. Rule list: `[]` (Enabled: `True`). {{(pid=80378) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 02 10:18:07.918119 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.fernet_utils [None req-0f2ffe8c-643a-4aac-aac2-33165d972c0d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80378) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 10:18:07.928586 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 1147/2308] 10.222.0.82 () {58 vars in 958 bytes} [Tue Aug 2 10:18:07 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 35 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 02 10:18:07.941878 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f2ffe8c-643a-4aac-aac2-33165d972c0d None None] Authenticating user token {{(pid=80378) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:18:07.942259 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7499a15-87a2-4bbd-8ce1-7ed71e77a874 None None] Validating token access rules against request {{(pid=80377) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:18:07.946346 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7612ad7-394b-405c-869b-c44d054eefac service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '33b87170f0634b67bce60750847b5cbe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f1b38ebb804c41c69772c998aae72c41', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80377) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:18:07.947158 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7612ad7-394b-405c-869b-c44d054eefac service neutron] REQUEST_METHOD: `GET` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:18:07.947323 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7612ad7-394b-405c-869b-c44d054eefac service neutron] SCRIPT_NAME: `/identity` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:18:07.947510 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7612ad7-394b-405c-869b-c44d054eefac service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:18:07.954527 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f2ffe8c-643a-4aac-aac2-33165d972c0d None None] Validating token access rules against request {{(pid=80378) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:18:07.957778 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7612ad7-394b-405c-869b-c44d054eefac service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:18:07.959246 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7612ad7-394b-405c-869b-c44d054eefac service neutron] RBAC: Authorization granted {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:18:07.959358 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a407e682-bc57-40ac-9e13-e5055e84d9df admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '891005fa5ff74041b75747d16cc66798', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7047912c5acd4adeba5d87c3bf895e39', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80378) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:18:07.959972 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a407e682-bc57-40ac-9e13-e5055e84d9df admin admin] REQUEST_METHOD: `GET` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:18:07.960256 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a407e682-bc57-40ac-9e13-e5055e84d9df admin admin] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:18:07.960533 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a407e682-bc57-40ac-9e13-e5055e84d9df admin admin] PATH_INFO: `/v3/domains` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:18:07.961227 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a407e682-bc57-40ac-9e13-e5055e84d9df admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80378) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 02 10:18:07.961535 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a407e682-bc57-40ac-9e13-e5055e84d9df admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:18:07.963233 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a407e682-bc57-40ac-9e13-e5055e84d9df admin admin] RBAC: Authorization granted {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:18:07.964890 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG dbcounter [-] [80378] Writing DB stats keystone:SELECT=81,keystone:DELETE=17,keystone:INSERT=2 {{(pid=80378) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 10:18:07.970392 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: [pid: 80377|app: 0|req: 1162/2309] 10.222.0.82 () {60 vars in 1365 bytes} [Tue Aug 2 10:18:07 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 90 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 02 10:18:07.979703 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 1148/2310] 10.222.0.82 () {58 vars in 1153 bytes} [Tue Aug 2 10:18:07 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 02 10:18:07.980830 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7612ad7-394b-405c-869b-c44d054eefac service neutron] Authenticating user token {{(pid=80377) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:18:07.991100 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7612ad7-394b-405c-869b-c44d054eefac service neutron] Validating token access rules against request {{(pid=80377) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:18:07.996953 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a7fd120-fc06-4e7c-a624-8bb06d844db2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '891005fa5ff74041b75747d16cc66798', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7047912c5acd4adeba5d87c3bf895e39', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80377) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:18:07.997588 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a7fd120-fc06-4e7c-a624-8bb06d844db2 admin admin] REQUEST_METHOD: `POST` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:18:07.998424 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a7fd120-fc06-4e7c-a624-8bb06d844db2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:18:07.998839 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a7fd120-fc06-4e7c-a624-8bb06d844db2 admin admin] PATH_INFO: `/v3/projects` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:18:07.999705 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a7fd120-fc06-4e7c-a624-8bb06d844db2 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:18:08.001658 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a7fd120-fc06-4e7c-a624-8bb06d844db2 admin admin] RBAC: Authorization granted {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:18:08.031656 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: [pid: 80377|app: 0|req: 1163/2311] 10.222.0.82 () {60 vars in 1153 bytes} [Tue Aug 2 10:18:07 2022] POST /identity/v3/projects => generated 385 bytes in 51 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 02 10:18:08.035934 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a407e682-bc57-40ac-9e13-e5055e84d9df admin admin] Authenticating user token {{(pid=80378) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:18:08.046998 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a407e682-bc57-40ac-9e13-e5055e84d9df admin admin] Validating token access rules against request {{(pid=80378) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:18:08.111125 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32dfe455-6ace-4c37-b6c7-7189419236ec admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '891005fa5ff74041b75747d16cc66798', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7047912c5acd4adeba5d87c3bf895e39', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80378) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:18:08.111756 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32dfe455-6ace-4c37-b6c7-7189419236ec admin admin] REQUEST_METHOD: `POST` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:18:08.112040 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32dfe455-6ace-4c37-b6c7-7189419236ec admin admin] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:18:08.112373 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32dfe455-6ace-4c37-b6c7-7189419236ec admin admin] PATH_INFO: `/v3/users` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:18:08.113085 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32dfe455-6ace-4c37-b6c7-7189419236ec admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:18:08.115023 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32dfe455-6ace-4c37-b6c7-7189419236ec admin admin] RBAC: Authorization granted {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:18:08.139818 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 1149/2312] 10.222.0.82 () {60 vars in 1144 bytes} [Tue Aug 2 10:18:08 2022] POST /identity/v3/users => generated 293 bytes in 104 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 02 10:18:08.144391 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a7fd120-fc06-4e7c-a624-8bb06d844db2 admin admin] Authenticating user token {{(pid=80377) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:18:08.160185 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a7fd120-fc06-4e7c-a624-8bb06d844db2 admin admin] Validating token access rules against request {{(pid=80377) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:18:08.168152 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8853a046-26cc-4cbd-9d6e-d8760d2afbec admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '891005fa5ff74041b75747d16cc66798', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7047912c5acd4adeba5d87c3bf895e39', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80377) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:18:08.168810 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8853a046-26cc-4cbd-9d6e-d8760d2afbec admin admin] REQUEST_METHOD: `GET` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:18:08.169088 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8853a046-26cc-4cbd-9d6e-d8760d2afbec admin admin] SCRIPT_NAME: `/identity` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:18:08.169507 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8853a046-26cc-4cbd-9d6e-d8760d2afbec admin admin] PATH_INFO: `/v3/roles` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:18:08.170832 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8853a046-26cc-4cbd-9d6e-d8760d2afbec admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:18:08.172439 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8853a046-26cc-4cbd-9d6e-d8760d2afbec admin admin] RBAC: Authorization granted {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:18:08.187639 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: [pid: 80377|app: 0|req: 1164/2313] 10.222.0.82 () {58 vars in 1122 bytes} [Tue Aug 2 10:18:08 2022] GET /identity/v3/roles => generated 1420 bytes in 41 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 02 10:18:08.188808 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32dfe455-6ace-4c37-b6c7-7189419236ec admin admin] Authenticating user token {{(pid=80378) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:18:08.199471 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32dfe455-6ace-4c37-b6c7-7189419236ec admin admin] Validating token access rules against request {{(pid=80378) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:18:08.204249 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51424627-9681-45a1-96c1-37ddd7ba5e2e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '891005fa5ff74041b75747d16cc66798', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7047912c5acd4adeba5d87c3bf895e39', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80378) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:18:08.204877 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51424627-9681-45a1-96c1-37ddd7ba5e2e admin admin] REQUEST_METHOD: `GET` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:18:08.205169 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51424627-9681-45a1-96c1-37ddd7ba5e2e admin admin] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:18:08.205448 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51424627-9681-45a1-96c1-37ddd7ba5e2e admin admin] PATH_INFO: `/v3/roles` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:18:08.206220 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51424627-9681-45a1-96c1-37ddd7ba5e2e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:18:08.207833 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51424627-9681-45a1-96c1-37ddd7ba5e2e admin admin] RBAC: Authorization granted {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:18:08.223825 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 1150/2314] 10.222.0.82 () {58 vars in 1122 bytes} [Tue Aug 2 10:18:08 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 02 10:18:08.225536 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8853a046-26cc-4cbd-9d6e-d8760d2afbec admin admin] Authenticating user token {{(pid=80377) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:18:08.239581 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8853a046-26cc-4cbd-9d6e-d8760d2afbec admin admin] Validating token access rules against request {{(pid=80377) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:18:08.251550 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32b04a97-7cfc-4d47-8697-eba2eced6d3b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '891005fa5ff74041b75747d16cc66798', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7047912c5acd4adeba5d87c3bf895e39', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80377) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:18:08.252589 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32b04a97-7cfc-4d47-8697-eba2eced6d3b admin admin] REQUEST_METHOD: `PUT` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:18:08.253006 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32b04a97-7cfc-4d47-8697-eba2eced6d3b admin admin] SCRIPT_NAME: `/identity` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:18:08.253427 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32b04a97-7cfc-4d47-8697-eba2eced6d3b admin admin] PATH_INFO: `/v3/projects/e6bd05cc56c845e2966c00307d49cec0/users/585f44d1d85e4ef3be39358f28f70a98/roles/e1c04b19fc4543b8b54f58bbcffbc598` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:18:08.270734 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51424627-9681-45a1-96c1-37ddd7ba5e2e admin admin] Authenticating user token {{(pid=80378) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:18:08.278003 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.fernet_utils [None req-51424627-9681-45a1-96c1-37ddd7ba5e2e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80378) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 10:18:08.294644 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32b04a97-7cfc-4d47-8697-eba2eced6d3b admin admin] RBAC: Authorizing `identity:create_grant(project_id=e6bd05cc56c845e2966c00307d49cec0, user_id=585f44d1d85e4ef3be39358f28f70a98, role_id=e1c04b19fc4543b8b54f58bbcffbc598)` {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:18:08.299969 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32b04a97-7cfc-4d47-8697-eba2eced6d3b admin admin] RBAC: Authorization granted {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:18:08.303306 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51424627-9681-45a1-96c1-37ddd7ba5e2e admin admin] Validating token access rules against request {{(pid=80378) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:18:08.312162 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa039d54-f6f0-433b-a61c-338c2ec0eea6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '891005fa5ff74041b75747d16cc66798', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7047912c5acd4adeba5d87c3bf895e39', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80378) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:18:08.313108 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa039d54-f6f0-433b-a61c-338c2ec0eea6 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:18:08.313595 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa039d54-f6f0-433b-a61c-338c2ec0eea6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:18:08.314006 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa039d54-f6f0-433b-a61c-338c2ec0eea6 admin admin] PATH_INFO: `/v3/projects/92ec411d93024682ae13872739095d2a` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:18:08.322684 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa039d54-f6f0-433b-a61c-338c2ec0eea6 admin admin] RBAC: Authorizing `identity:delete_project(project_id=92ec411d93024682ae13872739095d2a)` {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:18:08.325989 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa039d54-f6f0-433b-a61c-338c2ec0eea6 admin admin] RBAC: Authorization granted {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:18:08.339206 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: [pid: 80377|app: 0|req: 1165/2315] 10.222.0.82 () {60 vars in 1483 bytes} [Tue Aug 2 10:18:08 2022] PUT /identity/v3/projects/e6bd05cc56c845e2966c00307d49cec0/users/585f44d1d85e4ef3be39358f28f70a98/roles/e1c04b19fc4543b8b54f58bbcffbc598 => generated 0 bytes in 114 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 02 10:18:08.350966 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6f7575d-ffb1-4639-a57b-de11c78ba0e1 None None] REQUEST_METHOD: `POST` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:18:08.351395 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6f7575d-ffb1-4639-a57b-de11c78ba0e1 None None] SCRIPT_NAME: `/identity` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:18:08.351908 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6f7575d-ffb1-4639-a57b-de11c78ba0e1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:18:08.416437 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.auth.core [None req-c6f7575d-ffb1-4639-a57b-de11c78ba0e1 None None] MFA Rules not processed for user `585f44d1d85e4ef3be39358f28f70a98`. Rule list: `[]` (Enabled: `True`). {{(pid=80377) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 02 10:18:08.417683 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.fernet_utils [None req-c6f7575d-ffb1-4639-a57b-de11c78ba0e1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80377) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 10:18:08.443976 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.notifications [None req-fa039d54-f6f0-433b-a61c-338c2ec0eea6 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '92ec411d93024682ae13872739095d2a', 'request_id': 'req-fa039d54-f6f0-433b-a61c-338c2ec0eea6'} {{(pid=80378) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 10:18:08.458358 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.notifications [None req-fa039d54-f6f0-433b-a61c-338c2ec0eea6 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '92ec411d93024682ae13872739095d2a', 'request_id': 'req-fa039d54-f6f0-433b-a61c-338c2ec0eea6'} {{(pid=80378) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 10:18:08.475269 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.notifications [None req-fa039d54-f6f0-433b-a61c-338c2ec0eea6 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '92ec411d93024682ae13872739095d2a', 'request_id': 'req-fa039d54-f6f0-433b-a61c-338c2ec0eea6'} {{(pid=80378) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 10:18:08.501284 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG dbcounter [-] [80377] Writing DB stats keystone:SELECT=96,keystone:DELETE=2,keystone:INSERT=2 {{(pid=80377) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 10:18:08.525350 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.notifications [None req-fa039d54-f6f0-433b-a61c-338c2ec0eea6 admin admin] The token cache is being invalidate because project 92ec411d93024682ae13872739095d2a was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80378) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 02 10:18:08.526269 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.notifications [None req-fa039d54-f6f0-433b-a61c-338c2ec0eea6 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80378) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 10:18:08.527311 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.notifications [None req-fa039d54-f6f0-433b-a61c-338c2ec0eea6 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80378) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 10:18:08.529472 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 1151/2316] 10.222.0.82 () {58 vars in 1233 bytes} [Tue Aug 2 10:18:08 2022] DELETE /identity/v3/projects/92ec411d93024682ae13872739095d2a => generated 0 bytes in 259 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 02 10:18:08.547549 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fa9433b-55fd-4bad-bdce-f5f7529c965e None None] REQUEST_METHOD: `POST` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:18:08.547890 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fa9433b-55fd-4bad-bdce-f5f7529c965e None None] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:18:08.548233 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fa9433b-55fd-4bad-bdce-f5f7529c965e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:18:08.597227 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.auth.core [None req-5fa9433b-55fd-4bad-bdce-f5f7529c965e None None] MFA Rules not processed for user `891005fa5ff74041b75747d16cc66798`. Rule list: `[]` (Enabled: `True`). {{(pid=80378) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 02 10:18:08.605396 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.fernet_utils [None req-5fa9433b-55fd-4bad-bdce-f5f7529c965e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80378) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 10:18:08.614919 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: [pid: 80377|app: 0|req: 1166/2317] 10.222.0.82 () {58 vars in 958 bytes} [Tue Aug 2 10:18:08 2022] POST /identity/v3/auth/tokens => generated 3262 bytes in 265 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 02 10:18:08.616257 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6f7575d-ffb1-4639-a57b-de11c78ba0e1 None None] Authenticating user token {{(pid=80377) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:18:08.619361 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.fernet_utils [None req-c6f7575d-ffb1-4639-a57b-de11c78ba0e1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80377) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 10:18:08.738695 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG dbcounter [-] [80378] Writing DB stats keystone:SELECT=89,keystone:INSERT=4,keystone:DELETE=7 {{(pid=80378) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 10:18:08.739337 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6f7575d-ffb1-4639-a57b-de11c78ba0e1 None None] Validating token access rules against request {{(pid=80377) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:18:08.745420 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f44a1766-4340-41ac-8d0b-95b6e2a3c06d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '891005fa5ff74041b75747d16cc66798', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7047912c5acd4adeba5d87c3bf895e39', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80377) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:18:08.753893 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f44a1766-4340-41ac-8d0b-95b6e2a3c06d admin admin] REQUEST_METHOD: `POST` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:18:08.754426 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f44a1766-4340-41ac-8d0b-95b6e2a3c06d admin admin] SCRIPT_NAME: `/identity` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:18:08.754970 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f44a1766-4340-41ac-8d0b-95b6e2a3c06d admin admin] PATH_INFO: `/v3/projects` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:18:08.755868 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f44a1766-4340-41ac-8d0b-95b6e2a3c06d admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:18:08.758333 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f44a1766-4340-41ac-8d0b-95b6e2a3c06d admin admin] RBAC: Authorization granted {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:18:08.759199 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 1152/2318] 10.222.0.82 () {58 vars in 958 bytes} [Tue Aug 2 10:18:08 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 224 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 02 10:18:08.768244 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1922978b-90bd-487f-a795-99f62c9b861c None None] REQUEST_METHOD: `POST` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:18:08.768594 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1922978b-90bd-487f-a795-99f62c9b861c None None] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:18:08.768875 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1922978b-90bd-487f-a795-99f62c9b861c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:18:08.811914 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.auth.core [None req-1922978b-90bd-487f-a795-99f62c9b861c None None] MFA Rules not processed for user `891005fa5ff74041b75747d16cc66798`. Rule list: `[]` (Enabled: `True`). {{(pid=80378) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 02 10:18:08.817360 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: [pid: 80377|app: 0|req: 1167/2319] 10.222.0.82 () {60 vars in 1153 bytes} [Tue Aug 2 10:18:08 2022] POST /identity/v3/projects => generated 385 bytes in 199 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 02 10:18:08.817518 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.fernet_utils [None req-1922978b-90bd-487f-a795-99f62c9b861c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80378) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 10:18:08.818503 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f44a1766-4340-41ac-8d0b-95b6e2a3c06d admin admin] Authenticating user token {{(pid=80377) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:18:08.956692 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f44a1766-4340-41ac-8d0b-95b6e2a3c06d admin admin] Validating token access rules against request {{(pid=80377) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:18:08.967854 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 1153/2320] 10.222.0.82 () {58 vars in 958 bytes} [Tue Aug 2 10:18:08 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 201 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 02 10:18:08.969926 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7119cd4b-0226-4862-b0e5-76069e99ac3d None None] REQUEST_METHOD: `POST` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:18:08.970436 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7119cd4b-0226-4862-b0e5-76069e99ac3d None None] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:18:08.970921 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7119cd4b-0226-4862-b0e5-76069e99ac3d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:18:08.980385 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f985ae8-8dd4-4335-8815-df81eb675eb9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '891005fa5ff74041b75747d16cc66798', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7047912c5acd4adeba5d87c3bf895e39', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80377) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:18:08.980899 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f985ae8-8dd4-4335-8815-df81eb675eb9 admin admin] REQUEST_METHOD: `POST` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:18:08.981266 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f985ae8-8dd4-4335-8815-df81eb675eb9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:18:08.981537 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f985ae8-8dd4-4335-8815-df81eb675eb9 admin admin] PATH_INFO: `/v3/users` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:18:08.982315 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f985ae8-8dd4-4335-8815-df81eb675eb9 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:18:08.984754 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f985ae8-8dd4-4335-8815-df81eb675eb9 admin admin] RBAC: Authorization granted {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:18:09.010028 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: [pid: 80377|app: 0|req: 1168/2321] 10.222.0.82 () {60 vars in 1144 bytes} [Tue Aug 2 10:18:08 2022] POST /identity/v3/users => generated 293 bytes in 192 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 02 10:18:09.011069 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f985ae8-8dd4-4335-8815-df81eb675eb9 admin admin] Authenticating user token {{(pid=80377) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:18:09.014219 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.auth.core [None req-7119cd4b-0226-4862-b0e5-76069e99ac3d None None] MFA Rules not processed for user `bd8dd72086db48b8aaac2ed0058872df`. Rule list: `[]` (Enabled: `True`). {{(pid=80378) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 02 10:18:09.015645 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.fernet_utils [None req-7119cd4b-0226-4862-b0e5-76069e99ac3d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80378) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 10:18:09.023412 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f985ae8-8dd4-4335-8815-df81eb675eb9 admin admin] Validating token access rules against request {{(pid=80377) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:18:09.031858 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b27d8a34-9ca6-44f8-8b3b-305688da62b9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '891005fa5ff74041b75747d16cc66798', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7047912c5acd4adeba5d87c3bf895e39', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80377) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:18:09.032530 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b27d8a34-9ca6-44f8-8b3b-305688da62b9 admin admin] REQUEST_METHOD: `GET` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:18:09.032818 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b27d8a34-9ca6-44f8-8b3b-305688da62b9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:18:09.033096 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b27d8a34-9ca6-44f8-8b3b-305688da62b9 admin admin] PATH_INFO: `/v3/domains` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:18:09.034059 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b27d8a34-9ca6-44f8-8b3b-305688da62b9 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80377) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 02 10:18:09.034382 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b27d8a34-9ca6-44f8-8b3b-305688da62b9 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:18:09.036018 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b27d8a34-9ca6-44f8-8b3b-305688da62b9 admin admin] RBAC: Authorization granted {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:18:09.054592 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: [pid: 80377|app: 0|req: 1169/2322] 10.222.0.82 () {58 vars in 1153 bytes} [Tue Aug 2 10:18:09 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 44 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 02 10:18:09.056397 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b27d8a34-9ca6-44f8-8b3b-305688da62b9 admin admin] Authenticating user token {{(pid=80377) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:18:09.069077 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b27d8a34-9ca6-44f8-8b3b-305688da62b9 admin admin] Validating token access rules against request {{(pid=80377) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:18:09.074428 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6eb56bb9-6e37-4440-bd2c-278d7a3b22be admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '891005fa5ff74041b75747d16cc66798', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7047912c5acd4adeba5d87c3bf895e39', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80377) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:18:09.075066 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6eb56bb9-6e37-4440-bd2c-278d7a3b22be admin admin] REQUEST_METHOD: `GET` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:18:09.075506 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6eb56bb9-6e37-4440-bd2c-278d7a3b22be admin admin] SCRIPT_NAME: `/identity` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:18:09.075794 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6eb56bb9-6e37-4440-bd2c-278d7a3b22be admin admin] PATH_INFO: `/v3/roles` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:18:09.077414 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6eb56bb9-6e37-4440-bd2c-278d7a3b22be admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:18:09.080200 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6eb56bb9-6e37-4440-bd2c-278d7a3b22be admin admin] RBAC: Authorization granted {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:18:09.090396 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: [pid: 80377|app: 0|req: 1170/2323] 10.222.0.82 () {58 vars in 1122 bytes} [Tue Aug 2 10:18:09 2022] GET /identity/v3/roles => generated 1420 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 02 10:18:09.094266 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6eb56bb9-6e37-4440-bd2c-278d7a3b22be admin admin] Authenticating user token {{(pid=80377) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:18:09.097731 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 1154/2324] 10.222.0.82 () {58 vars in 1009 bytes} [Tue Aug 2 10:18:08 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 129 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 02 10:18:09.098711 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7119cd4b-0226-4862-b0e5-76069e99ac3d None None] Authenticating user token {{(pid=80378) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:18:09.105687 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6eb56bb9-6e37-4440-bd2c-278d7a3b22be admin admin] Validating token access rules against request {{(pid=80377) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:18:09.111556 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0811b08d-b72a-418c-9f10-70a798f91029 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '891005fa5ff74041b75747d16cc66798', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7047912c5acd4adeba5d87c3bf895e39', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80377) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:18:09.112200 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0811b08d-b72a-418c-9f10-70a798f91029 admin admin] REQUEST_METHOD: `POST` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:18:09.112489 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0811b08d-b72a-418c-9f10-70a798f91029 admin admin] SCRIPT_NAME: `/identity` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:18:09.112776 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0811b08d-b72a-418c-9f10-70a798f91029 admin admin] PATH_INFO: `/v3/projects` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:18:09.113458 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0811b08d-b72a-418c-9f10-70a798f91029 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:18:09.115554 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0811b08d-b72a-418c-9f10-70a798f91029 admin admin] RBAC: Authorization granted {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:18:09.118861 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7119cd4b-0226-4862-b0e5-76069e99ac3d None None] Validating token access rules against request {{(pid=80378) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:18:09.122145 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG dbcounter [-] [80377] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=80377) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 10:18:09.126850 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-522ed8db-98eb-4325-bb7f-0dceeae75596 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '891005fa5ff74041b75747d16cc66798', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7047912c5acd4adeba5d87c3bf895e39', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80378) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:18:09.128804 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-522ed8db-98eb-4325-bb7f-0dceeae75596 admin admin] REQUEST_METHOD: `GET` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:18:09.129246 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-522ed8db-98eb-4325-bb7f-0dceeae75596 admin admin] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:18:09.129700 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-522ed8db-98eb-4325-bb7f-0dceeae75596 admin admin] PATH_INFO: `/v3/roles` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:18:09.130596 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-522ed8db-98eb-4325-bb7f-0dceeae75596 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:18:09.132945 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-522ed8db-98eb-4325-bb7f-0dceeae75596 admin admin] RBAC: Authorization granted {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:18:09.147791 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 1155/2325] 10.222.0.82 () {58 vars in 1122 bytes} [Tue Aug 2 10:18:09 2022] GET /identity/v3/roles => generated 1420 bytes in 49 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 02 10:18:09.148803 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-522ed8db-98eb-4325-bb7f-0dceeae75596 admin admin] Authenticating user token {{(pid=80378) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:18:09.150997 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.fernet_utils [None req-522ed8db-98eb-4325-bb7f-0dceeae75596 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80378) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 10:18:09.151508 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: [pid: 80377|app: 0|req: 1171/2326] 10.222.0.82 () {60 vars in 1153 bytes} [Tue Aug 2 10:18:09 2022] POST /identity/v3/projects => generated 371 bytes in 58 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 02 10:18:09.152615 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0811b08d-b72a-418c-9f10-70a798f91029 admin admin] Authenticating user token {{(pid=80377) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:18:09.227682 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0811b08d-b72a-418c-9f10-70a798f91029 admin admin] Validating token access rules against request {{(pid=80377) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:18:09.229931 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-522ed8db-98eb-4325-bb7f-0dceeae75596 admin admin] Validating token access rules against request {{(pid=80378) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:18:09.234993 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6edc6e88-2ed9-4bda-a813-6938b1740c90 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '891005fa5ff74041b75747d16cc66798', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7047912c5acd4adeba5d87c3bf895e39', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80377) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:18:09.235583 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6edc6e88-2ed9-4bda-a813-6938b1740c90 admin admin] REQUEST_METHOD: `PUT` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:18:09.236362 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6edc6e88-2ed9-4bda-a813-6938b1740c90 admin admin] SCRIPT_NAME: `/identity` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:18:09.236362 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6edc6e88-2ed9-4bda-a813-6938b1740c90 admin admin] PATH_INFO: `/v3/projects/e713ef261a4a4d74a43d5d8974e61af3/users/199d9fc41db04feea1a1e1c9bb817dc0/roles/e1c04b19fc4543b8b54f58bbcffbc598` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:18:09.253661 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6edc6e88-2ed9-4bda-a813-6938b1740c90 admin admin] RBAC: Authorizing `identity:create_grant(project_id=e713ef261a4a4d74a43d5d8974e61af3, user_id=199d9fc41db04feea1a1e1c9bb817dc0, role_id=e1c04b19fc4543b8b54f58bbcffbc598)` {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:18:09.256532 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6edc6e88-2ed9-4bda-a813-6938b1740c90 admin admin] RBAC: Authorization granted {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:18:09.271677 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG dbcounter [-] [80378] Writing DB stats keystone:SELECT=100 {{(pid=80378) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 10:18:09.272076 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: [pid: 80377|app: 0|req: 1172/2327] 10.222.0.82 () {60 vars in 1483 bytes} [Tue Aug 2 10:18:09 2022] PUT /identity/v3/projects/e713ef261a4a4d74a43d5d8974e61af3/users/199d9fc41db04feea1a1e1c9bb817dc0/roles/e1c04b19fc4543b8b54f58bbcffbc598 => generated 0 bytes in 120 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 02 10:18:09.272893 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6edc6e88-2ed9-4bda-a813-6938b1740c90 admin admin] Authenticating user token {{(pid=80377) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:18:09.317711 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9c53ee9-26aa-4aba-a463-85147a09723f service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bef1b2e714cb45c4a51eb6d3e15aad6d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f1b38ebb804c41c69772c998aae72c41', 'project_domain_id': 'default', 'roles': ['admin', 'service', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80378) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:18:09.318437 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9c53ee9-26aa-4aba-a463-85147a09723f service swift] REQUEST_METHOD: `GET` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:18:09.318724 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9c53ee9-26aa-4aba-a463-85147a09723f service swift] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:18:09.319006 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9c53ee9-26aa-4aba-a463-85147a09723f service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:18:09.358835 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6edc6e88-2ed9-4bda-a813-6938b1740c90 admin admin] Validating token access rules against request {{(pid=80377) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:18:09.363706 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42e4caa0-6e46-4044-bc54-ff38173e3e7d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '891005fa5ff74041b75747d16cc66798', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7047912c5acd4adeba5d87c3bf895e39', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80377) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:18:09.364462 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42e4caa0-6e46-4044-bc54-ff38173e3e7d admin admin] REQUEST_METHOD: `POST` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:18:09.364762 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42e4caa0-6e46-4044-bc54-ff38173e3e7d admin admin] SCRIPT_NAME: `/identity` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:18:09.365174 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42e4caa0-6e46-4044-bc54-ff38173e3e7d admin admin] PATH_INFO: `/v3/users` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:18:09.365878 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42e4caa0-6e46-4044-bc54-ff38173e3e7d admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:18:09.367831 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42e4caa0-6e46-4044-bc54-ff38173e3e7d admin admin] RBAC: Authorization granted {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:18:09.392775 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9c53ee9-26aa-4aba-a463-85147a09723f service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:18:09.394600 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9c53ee9-26aa-4aba-a463-85147a09723f service swift] RBAC: Authorization granted {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:18:09.396994 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: [pid: 80377|app: 0|req: 1173/2328] 10.222.0.82 () {60 vars in 1144 bytes} [Tue Aug 2 10:18:09 2022] POST /identity/v3/users => generated 286 bytes in 124 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 02 10:18:09.398945 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d851a5da-f597-420b-b83c-49381b76bc3c None None] REQUEST_METHOD: `POST` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:18:09.398945 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d851a5da-f597-420b-b83c-49381b76bc3c None None] SCRIPT_NAME: `/identity` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:18:09.399541 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d851a5da-f597-420b-b83c-49381b76bc3c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:18:09.414597 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 1156/2329] 10.222.0.82 () {60 vars in 1384 bytes} [Tue Aug 2 10:18:09 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 266 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Aug 02 10:18:09.415220 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9c53ee9-26aa-4aba-a463-85147a09723f service swift] Authenticating user token {{(pid=80378) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:18:09.434552 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9c53ee9-26aa-4aba-a463-85147a09723f service swift] Validating token access rules against request {{(pid=80378) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:18:09.438227 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.auth.core [None req-d851a5da-f597-420b-b83c-49381b76bc3c None None] MFA Rules not processed for user `199d9fc41db04feea1a1e1c9bb817dc0`. Rule list: `[]` (Enabled: `True`). {{(pid=80377) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 02 10:18:09.439683 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.fernet_utils [None req-d851a5da-f597-420b-b83c-49381b76bc3c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80377) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 10:18:09.442200 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19cd79ed-9901-4268-bb74-e131621cf831 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '891005fa5ff74041b75747d16cc66798', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7047912c5acd4adeba5d87c3bf895e39', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80378) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:18:09.442929 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19cd79ed-9901-4268-bb74-e131621cf831 admin admin] REQUEST_METHOD: `GET` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:18:09.443241 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19cd79ed-9901-4268-bb74-e131621cf831 admin admin] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:18:09.443569 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19cd79ed-9901-4268-bb74-e131621cf831 admin admin] PATH_INFO: `/v3/roles` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:18:09.444220 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19cd79ed-9901-4268-bb74-e131621cf831 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:18:09.446771 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19cd79ed-9901-4268-bb74-e131621cf831 admin admin] RBAC: Authorization granted {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:18:09.461237 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 1157/2330] 10.222.0.82 () {58 vars in 1122 bytes} [Tue Aug 2 10:18:09 2022] GET /identity/v3/roles => generated 1420 bytes in 46 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 02 10:18:09.483272 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19cd79ed-9901-4268-bb74-e131621cf831 admin admin] Authenticating user token {{(pid=80378) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:18:09.511264 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19cd79ed-9901-4268-bb74-e131621cf831 admin admin] Validating token access rules against request {{(pid=80378) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:18:09.516533 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG dbcounter [-] [80377] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80377) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 10:18:09.521804 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01ad4a90-10e3-49e2-b45a-a0223da4152b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '891005fa5ff74041b75747d16cc66798', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7047912c5acd4adeba5d87c3bf895e39', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80378) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:18:09.523952 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01ad4a90-10e3-49e2-b45a-a0223da4152b admin admin] REQUEST_METHOD: `GET` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:18:09.524275 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01ad4a90-10e3-49e2-b45a-a0223da4152b admin admin] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:18:09.524557 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01ad4a90-10e3-49e2-b45a-a0223da4152b admin admin] PATH_INFO: `/v3/roles` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:18:09.525308 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01ad4a90-10e3-49e2-b45a-a0223da4152b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:18:09.527706 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01ad4a90-10e3-49e2-b45a-a0223da4152b admin admin] RBAC: Authorization granted {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:18:09.563517 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 1158/2331] 10.222.0.82 () {58 vars in 1122 bytes} [Tue Aug 2 10:18:09 2022] GET /identity/v3/roles => generated 1420 bytes in 87 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 02 10:18:09.569030 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01ad4a90-10e3-49e2-b45a-a0223da4152b admin admin] Authenticating user token {{(pid=80378) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:18:09.579325 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: [pid: 80377|app: 0|req: 1174/2332] 10.222.0.82 () {58 vars in 958 bytes} [Tue Aug 2 10:18:09 2022] POST /identity/v3/auth/tokens => generated 3262 bytes in 182 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 02 10:18:09.585525 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d851a5da-f597-420b-b83c-49381b76bc3c None None] Authenticating user token {{(pid=80377) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:18:09.603342 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d851a5da-f597-420b-b83c-49381b76bc3c None None] Validating token access rules against request {{(pid=80377) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:18:09.609417 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01ad4a90-10e3-49e2-b45a-a0223da4152b admin admin] Validating token access rules against request {{(pid=80378) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:18:09.609566 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bcfd2411-c974-4923-b78f-1648b7e3aefb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '891005fa5ff74041b75747d16cc66798', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7047912c5acd4adeba5d87c3bf895e39', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80377) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:18:09.610687 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcfd2411-c974-4923-b78f-1648b7e3aefb admin admin] REQUEST_METHOD: `POST` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:18:09.610992 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcfd2411-c974-4923-b78f-1648b7e3aefb admin admin] SCRIPT_NAME: `/identity` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:18:09.611267 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcfd2411-c974-4923-b78f-1648b7e3aefb admin admin] PATH_INFO: `/v3/projects` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:18:09.611947 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bcfd2411-c974-4923-b78f-1648b7e3aefb admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:18:09.613940 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bcfd2411-c974-4923-b78f-1648b7e3aefb admin admin] RBAC: Authorization granted {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:18:09.627188 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fcedeac3-e903-4098-a365-3207abbba44a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '891005fa5ff74041b75747d16cc66798', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7047912c5acd4adeba5d87c3bf895e39', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80378) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:18:09.630667 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcedeac3-e903-4098-a365-3207abbba44a admin admin] REQUEST_METHOD: `PUT` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:18:09.630971 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcedeac3-e903-4098-a365-3207abbba44a admin admin] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:18:09.631308 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcedeac3-e903-4098-a365-3207abbba44a admin admin] PATH_INFO: `/v3/projects/b83d3a9d900c4e9383fa88fd1475c029/users/a5f2ec904f214a708a3691da24fb516d/roles/e1c04b19fc4543b8b54f58bbcffbc598` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:18:09.672472 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: [pid: 80377|app: 0|req: 1175/2333] 10.222.0.82 () {60 vars in 1153 bytes} [Tue Aug 2 10:18:09 2022] POST /identity/v3/projects => generated 383 bytes in 87 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 02 10:18:09.673535 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bcfd2411-c974-4923-b78f-1648b7e3aefb admin admin] Authenticating user token {{(pid=80377) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:18:09.683430 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fcedeac3-e903-4098-a365-3207abbba44a admin admin] RBAC: Authorizing `identity:create_grant(project_id=b83d3a9d900c4e9383fa88fd1475c029, user_id=a5f2ec904f214a708a3691da24fb516d, role_id=e1c04b19fc4543b8b54f58bbcffbc598)` {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:18:09.690892 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fcedeac3-e903-4098-a365-3207abbba44a admin admin] RBAC: Authorization granted {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:18:09.745048 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 1159/2334] 10.222.0.82 () {60 vars in 1483 bytes} [Tue Aug 2 10:18:09 2022] PUT /identity/v3/projects/b83d3a9d900c4e9383fa88fd1475c029/users/a5f2ec904f214a708a3691da24fb516d/roles/e1c04b19fc4543b8b54f58bbcffbc598 => generated 0 bytes in 172 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 02 10:18:09.745773 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d822ded3-f474-4582-b1a4-7a04b2b64d09 None None] REQUEST_METHOD: `POST` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:18:09.746065 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d822ded3-f474-4582-b1a4-7a04b2b64d09 None None] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:18:09.746401 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d822ded3-f474-4582-b1a4-7a04b2b64d09 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:18:09.800026 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.auth.core [None req-d822ded3-f474-4582-b1a4-7a04b2b64d09 None None] MFA Rules not processed for user `891005fa5ff74041b75747d16cc66798`. Rule list: `[]` (Enabled: `True`). {{(pid=80378) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 02 10:18:09.803138 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.fernet_utils [None req-d822ded3-f474-4582-b1a4-7a04b2b64d09 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80378) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 10:18:09.867799 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bcfd2411-c974-4923-b78f-1648b7e3aefb admin admin] Validating token access rules against request {{(pid=80377) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:18:09.984560 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 1160/2335] 10.222.0.82 () {58 vars in 958 bytes} [Tue Aug 2 10:18:09 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 240 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 02 10:18:09.996190 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d822ded3-f474-4582-b1a4-7a04b2b64d09 None None] Authenticating user token {{(pid=80378) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:18:09.999434 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.fernet_utils [None req-d822ded3-f474-4582-b1a4-7a04b2b64d09 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80378) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 10:18:10.053040 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d075c35e-d416-4458-acc4-eb97ebb64edf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '891005fa5ff74041b75747d16cc66798', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7047912c5acd4adeba5d87c3bf895e39', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80377) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:18:10.054230 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d075c35e-d416-4458-acc4-eb97ebb64edf admin admin] REQUEST_METHOD: `POST` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:18:10.066980 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d075c35e-d416-4458-acc4-eb97ebb64edf admin admin] SCRIPT_NAME: `/identity` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:18:10.073877 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d075c35e-d416-4458-acc4-eb97ebb64edf admin admin] PATH_INFO: `/v3/users` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:18:10.077693 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d075c35e-d416-4458-acc4-eb97ebb64edf admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:18:10.078549 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG dbcounter [-] [80378] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80378) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 10:18:10.088039 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d075c35e-d416-4458-acc4-eb97ebb64edf admin admin] RBAC: Authorization granted {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:18:10.149904 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d822ded3-f474-4582-b1a4-7a04b2b64d09 None None] Validating token access rules against request {{(pid=80378) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:18:10.151458 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: [pid: 80377|app: 0|req: 1176/2336] 10.222.0.82 () {60 vars in 1144 bytes} [Tue Aug 2 10:18:09 2022] POST /identity/v3/users => generated 298 bytes in 478 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 02 10:18:10.152355 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d075c35e-d416-4458-acc4-eb97ebb64edf admin admin] Authenticating user token {{(pid=80377) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:18:10.165642 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d075c35e-d416-4458-acc4-eb97ebb64edf admin admin] Validating token access rules against request {{(pid=80377) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:18:10.185037 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-078cf787-80f1-4832-ae05-4e1c06817d40 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '33b87170f0634b67bce60750847b5cbe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f1b38ebb804c41c69772c998aae72c41', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80378) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:18:10.185750 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-078cf787-80f1-4832-ae05-4e1c06817d40 service neutron] REQUEST_METHOD: `GET` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:18:10.186802 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-078cf787-80f1-4832-ae05-4e1c06817d40 service neutron] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:18:10.187134 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-078cf787-80f1-4832-ae05-4e1c06817d40 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:18:10.187866 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7f684f4-0252-452b-99ee-c5dbe2e7d22b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '891005fa5ff74041b75747d16cc66798', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7047912c5acd4adeba5d87c3bf895e39', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80377) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:18:10.188514 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7f684f4-0252-452b-99ee-c5dbe2e7d22b admin admin] REQUEST_METHOD: `GET` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:18:10.188799 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7f684f4-0252-452b-99ee-c5dbe2e7d22b admin admin] SCRIPT_NAME: `/identity` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:18:10.189078 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7f684f4-0252-452b-99ee-c5dbe2e7d22b admin admin] PATH_INFO: `/v3/roles` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:18:10.189827 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7f684f4-0252-452b-99ee-c5dbe2e7d22b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:18:10.196247 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7f684f4-0252-452b-99ee-c5dbe2e7d22b admin admin] RBAC: Authorization granted {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:18:10.199930 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-078cf787-80f1-4832-ae05-4e1c06817d40 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:18:10.201442 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-078cf787-80f1-4832-ae05-4e1c06817d40 service neutron] RBAC: Authorization granted {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:18:10.224416 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: [pid: 80377|app: 0|req: 1177/2337] 10.222.0.82 () {58 vars in 1122 bytes} [Tue Aug 2 10:18:10 2022] GET /identity/v3/roles => generated 1420 bytes in 72 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 02 10:18:10.237399 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7f684f4-0252-452b-99ee-c5dbe2e7d22b admin admin] Authenticating user token {{(pid=80377) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:18:10.247157 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.fernet_utils [None req-b7f684f4-0252-452b-99ee-c5dbe2e7d22b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80377) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 10:18:10.264798 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 1161/2338] 10.222.0.82 () {60 vars in 1365 bytes} [Tue Aug 2 10:18:09 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 269 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 02 10:18:10.265672 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-078cf787-80f1-4832-ae05-4e1c06817d40 service neutron] Authenticating user token {{(pid=80378) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:18:10.274109 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7f684f4-0252-452b-99ee-c5dbe2e7d22b admin admin] Validating token access rules against request {{(pid=80377) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:18:10.282003 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-078cf787-80f1-4832-ae05-4e1c06817d40 service neutron] Validating token access rules against request {{(pid=80378) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:18:10.287333 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e9c3b78-ef52-4830-9d00-33b360662918 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '891005fa5ff74041b75747d16cc66798', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7047912c5acd4adeba5d87c3bf895e39', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80378) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:18:10.288037 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e9c3b78-ef52-4830-9d00-33b360662918 admin admin] REQUEST_METHOD: `PUT` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:18:10.288328 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e9c3b78-ef52-4830-9d00-33b360662918 admin admin] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:18:10.288608 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e9c3b78-ef52-4830-9d00-33b360662918 admin admin] PATH_INFO: `/v3/projects/0be9d5f87bf3453493d7b9625a85bbda/users/5ae3e3e0ec7646e187a45d1b0045c016/roles/237966442f05411191105bf336bc14f3` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:18:10.290759 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3068be4b-4390-4f8b-8244-066d23a6fda5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '891005fa5ff74041b75747d16cc66798', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7047912c5acd4adeba5d87c3bf895e39', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80377) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:18:10.291429 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3068be4b-4390-4f8b-8244-066d23a6fda5 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:18:10.318262 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3068be4b-4390-4f8b-8244-066d23a6fda5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:18:10.318262 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3068be4b-4390-4f8b-8244-066d23a6fda5 admin admin] PATH_INFO: `/v3/users/ffe0cbc6a5f34d779966d4cee544d428` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:18:10.318893 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3068be4b-4390-4f8b-8244-066d23a6fda5 admin admin] RBAC: Authorizing `identity:delete_user(user_id=ffe0cbc6a5f34d779966d4cee544d428)` {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:18:10.321570 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3068be4b-4390-4f8b-8244-066d23a6fda5 admin admin] RBAC: Authorization granted {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:18:10.365931 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e9c3b78-ef52-4830-9d00-33b360662918 admin admin] RBAC: Authorizing `identity:create_grant(project_id=0be9d5f87bf3453493d7b9625a85bbda, user_id=5ae3e3e0ec7646e187a45d1b0045c016, role_id=237966442f05411191105bf336bc14f3)` {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:18:10.368845 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e9c3b78-ef52-4830-9d00-33b360662918 admin admin] RBAC: Authorization granted {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:18:10.429599 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 1162/2339] 10.222.0.82 () {60 vars in 1483 bytes} [Tue Aug 2 10:18:10 2022] PUT /identity/v3/projects/0be9d5f87bf3453493d7b9625a85bbda/users/5ae3e3e0ec7646e187a45d1b0045c016/roles/237966442f05411191105bf336bc14f3 => generated 0 bytes in 164 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 02 10:18:10.433961 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e9c3b78-ef52-4830-9d00-33b360662918 admin admin] Authenticating user token {{(pid=80378) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:18:10.468674 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.notifications [None req-3068be4b-4390-4f8b-8244-066d23a6fda5 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'ffe0cbc6a5f34d779966d4cee544d428'} {{(pid=80377) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 10:18:10.480744 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG dbcounter [-] [80377] Writing DB stats keystone:SELECT=89,keystone:INSERT=4,keystone:DELETE=7 {{(pid=80377) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 10:18:10.496272 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.notifications [None req-3068be4b-4390-4f8b-8244-066d23a6fda5 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'ffe0cbc6a5f34d779966d4cee544d428'} {{(pid=80377) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 10:18:10.517492 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.notifications [None req-3068be4b-4390-4f8b-8244-066d23a6fda5 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'ffe0cbc6a5f34d779966d4cee544d428'} {{(pid=80377) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 10:18:10.518863 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.notifications [None req-3068be4b-4390-4f8b-8244-066d23a6fda5 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'ffe0cbc6a5f34d779966d4cee544d428'} {{(pid=80377) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 10:18:10.564448 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.notifications [None req-3068be4b-4390-4f8b-8244-066d23a6fda5 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'ffe0cbc6a5f34d779966d4cee544d428'} {{(pid=80377) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 10:18:10.598313 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: [pid: 80377|app: 0|req: 1178/2340] 10.222.0.82 () {58 vars in 1224 bytes} [Tue Aug 2 10:18:10 2022] DELETE /identity/v3/users/ffe0cbc6a5f34d779966d4cee544d428 => generated 0 bytes in 371 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 02 10:18:10.603954 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de60ac4c-3b9a-4d73-b4b9-1f42d8727aa8 None None] REQUEST_METHOD: `POST` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:18:10.603954 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de60ac4c-3b9a-4d73-b4b9-1f42d8727aa8 None None] SCRIPT_NAME: `/identity` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:18:10.603954 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de60ac4c-3b9a-4d73-b4b9-1f42d8727aa8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:18:10.626031 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e9c3b78-ef52-4830-9d00-33b360662918 admin admin] Validating token access rules against request {{(pid=80378) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:18:10.649191 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.auth.core [None req-de60ac4c-3b9a-4d73-b4b9-1f42d8727aa8 None None] MFA Rules not processed for user `891005fa5ff74041b75747d16cc66798`. Rule list: `[]` (Enabled: `True`). {{(pid=80377) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 02 10:18:10.650512 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.fernet_utils [None req-de60ac4c-3b9a-4d73-b4b9-1f42d8727aa8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80377) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 10:18:10.731605 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-233eb136-aa24-47ab-ab40-b3d50fbf4751 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '891005fa5ff74041b75747d16cc66798', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7047912c5acd4adeba5d87c3bf895e39', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80378) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:18:10.733382 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-233eb136-aa24-47ab-ab40-b3d50fbf4751 admin admin] REQUEST_METHOD: `GET` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:18:10.733879 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-233eb136-aa24-47ab-ab40-b3d50fbf4751 admin admin] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:18:10.734486 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-233eb136-aa24-47ab-ab40-b3d50fbf4751 admin admin] PATH_INFO: `/v3/roles` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:18:10.735529 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-233eb136-aa24-47ab-ab40-b3d50fbf4751 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:18:10.738068 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-233eb136-aa24-47ab-ab40-b3d50fbf4751 admin admin] RBAC: Authorization granted {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:18:10.754324 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 1163/2341] 10.222.0.82 () {58 vars in 1122 bytes} [Tue Aug 2 10:18:10 2022] GET /identity/v3/roles => generated 1420 bytes in 321 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 02 10:18:10.757972 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-233eb136-aa24-47ab-ab40-b3d50fbf4751 admin admin] Authenticating user token {{(pid=80378) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:18:10.760106 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.fernet_utils [None req-233eb136-aa24-47ab-ab40-b3d50fbf4751 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80378) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 10:18:10.776766 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: [pid: 80377|app: 0|req: 1179/2342] 10.222.0.82 () {58 vars in 958 bytes} [Tue Aug 2 10:18:10 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 177 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 02 10:18:10.780061 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-233eb136-aa24-47ab-ab40-b3d50fbf4751 admin admin] Validating token access rules against request {{(pid=80378) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:18:10.787203 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de60ac4c-3b9a-4d73-b4b9-1f42d8727aa8 None None] Authenticating user token {{(pid=80377) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:18:10.788862 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.fernet_utils [None req-de60ac4c-3b9a-4d73-b4b9-1f42d8727aa8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80377) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 10:18:10.793290 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b8c9b7a-0d90-48b4-90f1-e048e4d0debe admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '891005fa5ff74041b75747d16cc66798', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7047912c5acd4adeba5d87c3bf895e39', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80378) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:18:10.796322 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b8c9b7a-0d90-48b4-90f1-e048e4d0debe admin admin] REQUEST_METHOD: `PUT` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:18:10.797063 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b8c9b7a-0d90-48b4-90f1-e048e4d0debe admin admin] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:18:10.799384 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b8c9b7a-0d90-48b4-90f1-e048e4d0debe admin admin] PATH_INFO: `/v3/projects/0be9d5f87bf3453493d7b9625a85bbda/users/5ae3e3e0ec7646e187a45d1b0045c016/roles/237966442f05411191105bf336bc14f3` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:18:10.803992 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b8c9b7a-0d90-48b4-90f1-e048e4d0debe admin admin] RBAC: Authorizing `identity:create_grant(project_id=0be9d5f87bf3453493d7b9625a85bbda, user_id=5ae3e3e0ec7646e187a45d1b0045c016, role_id=237966442f05411191105bf336bc14f3)` {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:18:10.807300 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b8c9b7a-0d90-48b4-90f1-e048e4d0debe admin admin] RBAC: Authorization granted {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:18:10.851385 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 1164/2343] 10.222.0.82 () {60 vars in 1483 bytes} [Tue Aug 2 10:18:10 2022] PUT /identity/v3/projects/0be9d5f87bf3453493d7b9625a85bbda/users/5ae3e3e0ec7646e187a45d1b0045c016/roles/237966442f05411191105bf336bc14f3 => generated 0 bytes in 87 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 02 10:18:10.869061 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f58e278e-2830-4561-8eea-1dfd9347850b None None] REQUEST_METHOD: `POST` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:18:10.869061 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f58e278e-2830-4561-8eea-1dfd9347850b None None] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:18:10.869061 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f58e278e-2830-4561-8eea-1dfd9347850b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:18:10.932414 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de60ac4c-3b9a-4d73-b4b9-1f42d8727aa8 None None] Validating token access rules against request {{(pid=80377) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:18:10.999543 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.auth.core [None req-f58e278e-2830-4561-8eea-1dfd9347850b None None] MFA Rules not processed for user `5ae3e3e0ec7646e187a45d1b0045c016`. Rule list: `[]` (Enabled: `True`). {{(pid=80378) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 02 10:18:11.004081 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.fernet_utils [None req-f58e278e-2830-4561-8eea-1dfd9347850b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80378) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 10:18:11.044912 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1892220d-05e6-48eb-8c16-b80f815155be service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '33b87170f0634b67bce60750847b5cbe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f1b38ebb804c41c69772c998aae72c41', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80377) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:18:11.045555 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1892220d-05e6-48eb-8c16-b80f815155be service neutron] REQUEST_METHOD: `GET` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:18:11.045859 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1892220d-05e6-48eb-8c16-b80f815155be service neutron] SCRIPT_NAME: `/identity` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:18:11.046140 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1892220d-05e6-48eb-8c16-b80f815155be service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:18:11.062566 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG dbcounter [-] [80378] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80378) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 10:18:11.144641 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG dbcounter [-] [80377] Writing DB stats keystone:INSERT=1,keystone:DELETE=3,keystone:SELECT=96 {{(pid=80377) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 10:18:11.149545 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 1165/2344] 10.222.0.82 () {58 vars in 958 bytes} [Tue Aug 2 10:18:10 2022] POST /identity/v3/auth/tokens => generated 3327 bytes in 287 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 02 10:18:11.165664 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f58e278e-2830-4561-8eea-1dfd9347850b None None] Authenticating user token {{(pid=80378) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:18:11.168456 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.fernet_utils [None req-f58e278e-2830-4561-8eea-1dfd9347850b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80378) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 10:18:11.197188 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1892220d-05e6-48eb-8c16-b80f815155be service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:18:11.199068 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1892220d-05e6-48eb-8c16-b80f815155be service neutron] RBAC: Authorization granted {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:18:11.210781 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: [pid: 80377|app: 0|req: 1180/2345] 10.222.0.82 () {60 vars in 1365 bytes} [Tue Aug 2 10:18:10 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 424 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 02 10:18:11.246988 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f58e278e-2830-4561-8eea-1dfd9347850b None None] Validating token access rules against request {{(pid=80378) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:18:11.252445 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbff9fea-7c64-485a-926c-52f2a70e0ef2 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1023de281ae74e9d8a6d6c312c1c97a7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f1b38ebb804c41c69772c998aae72c41', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80378) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:18:11.253087 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbff9fea-7c64-485a-926c-52f2a70e0ef2 service cinder] REQUEST_METHOD: `GET` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:18:11.253371 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbff9fea-7c64-485a-926c-52f2a70e0ef2 service cinder] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:18:11.253949 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbff9fea-7c64-485a-926c-52f2a70e0ef2 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:18:11.273726 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cbff9fea-7c64-485a-926c-52f2a70e0ef2 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:18:11.276429 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cbff9fea-7c64-485a-926c-52f2a70e0ef2 service cinder] RBAC: Authorization granted {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:18:11.292861 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 1166/2346] 10.222.0.82 () {60 vars in 1365 bytes} [Tue Aug 2 10:18:11 2022] GET /identity/v3/auth/tokens => generated 3327 bytes in 128 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 02 10:18:11.565523 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31695716-70a7-4ae6-84c2-cc8a67422afb None None] REQUEST_METHOD: `POST` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:18:11.565886 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31695716-70a7-4ae6-84c2-cc8a67422afb None None] SCRIPT_NAME: `/identity` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:18:11.566180 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31695716-70a7-4ae6-84c2-cc8a67422afb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:18:11.643919 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.auth.core [None req-31695716-70a7-4ae6-84c2-cc8a67422afb None None] MFA Rules not processed for user `a5f2ec904f214a708a3691da24fb516d`. Rule list: `[]` (Enabled: `True`). {{(pid=80377) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 02 10:18:11.645362 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.fernet_utils [None req-31695716-70a7-4ae6-84c2-cc8a67422afb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80377) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 10:18:11.837529 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: [pid: 80377|app: 0|req: 1181/2347] 10.222.0.82 () {58 vars in 958 bytes} [Tue Aug 2 10:18:11 2022] POST /identity/v3/auth/tokens => generated 3248 bytes in 273 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 02 10:18:11.870811 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbff9fea-7c64-485a-926c-52f2a70e0ef2 service cinder] Authenticating user token {{(pid=80378) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:18:11.875864 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.fernet_utils [None req-cbff9fea-7c64-485a-926c-52f2a70e0ef2 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80378) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 10:18:11.904047 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbff9fea-7c64-485a-926c-52f2a70e0ef2 service cinder] Validating token access rules against request {{(pid=80378) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:18:11.912264 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9f66b2b-322a-402c-a126-3192cd56bf77 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1023de281ae74e9d8a6d6c312c1c97a7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f1b38ebb804c41c69772c998aae72c41', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80378) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:18:11.912905 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9f66b2b-322a-402c-a126-3192cd56bf77 service cinder] REQUEST_METHOD: `GET` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:18:11.913193 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9f66b2b-322a-402c-a126-3192cd56bf77 service cinder] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:18:11.913471 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9f66b2b-322a-402c-a126-3192cd56bf77 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:18:11.941536 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9f66b2b-322a-402c-a126-3192cd56bf77 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:18:11.943084 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9f66b2b-322a-402c-a126-3192cd56bf77 service cinder] RBAC: Authorization granted {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:18:11.959674 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 1167/2348] 10.222.0.82 () {60 vars in 1365 bytes} [Tue Aug 2 10:18:11 2022] GET /identity/v3/auth/tokens => generated 3248 bytes in 90 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 02 10:18:12.145389 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31695716-70a7-4ae6-84c2-cc8a67422afb None None] Authenticating user token {{(pid=80377) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:18:12.147501 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.fernet_utils [None req-31695716-70a7-4ae6-84c2-cc8a67422afb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80377) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 10:18:12.187874 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31695716-70a7-4ae6-84c2-cc8a67422afb None None] Validating token access rules against request {{(pid=80377) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:18:12.207497 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a0da9ab-deb2-4b14-8e15-ccd5f8349cb3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '891005fa5ff74041b75747d16cc66798', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7047912c5acd4adeba5d87c3bf895e39', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80377) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:18:12.209574 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a0da9ab-deb2-4b14-8e15-ccd5f8349cb3 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:18:12.210078 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a0da9ab-deb2-4b14-8e15-ccd5f8349cb3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:18:12.210365 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a0da9ab-deb2-4b14-8e15-ccd5f8349cb3 admin admin] PATH_INFO: `/v3/projects/8f69565b334545f2903d55518686abae` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:18:12.211690 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a0da9ab-deb2-4b14-8e15-ccd5f8349cb3 admin admin] RBAC: Authorizing `identity:delete_project(project_id=8f69565b334545f2903d55518686abae)` {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:18:12.213637 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a0da9ab-deb2-4b14-8e15-ccd5f8349cb3 admin admin] RBAC: Authorization granted {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:18:12.273724 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.notifications [None req-8a0da9ab-deb2-4b14-8e15-ccd5f8349cb3 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '8f69565b334545f2903d55518686abae', 'request_id': 'req-8a0da9ab-deb2-4b14-8e15-ccd5f8349cb3'} {{(pid=80377) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 10:18:12.280638 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.notifications [None req-8a0da9ab-deb2-4b14-8e15-ccd5f8349cb3 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '8f69565b334545f2903d55518686abae', 'request_id': 'req-8a0da9ab-deb2-4b14-8e15-ccd5f8349cb3'} {{(pid=80377) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 10:18:12.289975 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.notifications [None req-8a0da9ab-deb2-4b14-8e15-ccd5f8349cb3 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '8f69565b334545f2903d55518686abae', 'request_id': 'req-8a0da9ab-deb2-4b14-8e15-ccd5f8349cb3'} {{(pid=80377) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 10:18:12.314424 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.notifications [None req-8a0da9ab-deb2-4b14-8e15-ccd5f8349cb3 admin admin] The token cache is being invalidate because project 8f69565b334545f2903d55518686abae was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80377) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 02 10:18:12.314732 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.notifications [None req-8a0da9ab-deb2-4b14-8e15-ccd5f8349cb3 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80377) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 10:18:12.315511 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.notifications [None req-8a0da9ab-deb2-4b14-8e15-ccd5f8349cb3 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80377) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 10:18:12.317487 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: [pid: 80377|app: 0|req: 1182/2349] 10.222.0.82 () {58 vars in 1233 bytes} [Tue Aug 2 10:18:12 2022] DELETE /identity/v3/projects/8f69565b334545f2903d55518686abae => generated 0 bytes in 172 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 02 10:18:12.321556 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9f66b2b-322a-402c-a126-3192cd56bf77 service cinder] Authenticating user token {{(pid=80378) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:18:12.343169 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9f66b2b-322a-402c-a126-3192cd56bf77 service cinder] Validating token access rules against request {{(pid=80378) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:18:12.444411 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-252af228-75b8-4b4b-89a9-1273ee83847a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '891005fa5ff74041b75747d16cc66798', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7047912c5acd4adeba5d87c3bf895e39', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80378) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:18:12.445641 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-252af228-75b8-4b4b-89a9-1273ee83847a admin admin] REQUEST_METHOD: `DELETE` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:18:12.446259 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-252af228-75b8-4b4b-89a9-1273ee83847a admin admin] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:18:12.446762 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-252af228-75b8-4b4b-89a9-1273ee83847a admin admin] PATH_INFO: `/v3/users/2ac6c9dcc5cb4be981026eb113525afb` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:18:12.449229 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-252af228-75b8-4b4b-89a9-1273ee83847a admin admin] RBAC: Authorizing `identity:delete_user(user_id=2ac6c9dcc5cb4be981026eb113525afb)` {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:18:12.450910 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a0da9ab-deb2-4b14-8e15-ccd5f8349cb3 admin admin] Authenticating user token {{(pid=80377) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:18:12.453163 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.fernet_utils [None req-8a0da9ab-deb2-4b14-8e15-ccd5f8349cb3 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80377) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 10:18:12.456175 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-252af228-75b8-4b4b-89a9-1273ee83847a admin admin] RBAC: Authorization granted {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:18:12.559294 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG dbcounter [-] [80377] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80377) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 10:18:12.571279 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.notifications [None req-252af228-75b8-4b4b-89a9-1273ee83847a admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '2ac6c9dcc5cb4be981026eb113525afb'} {{(pid=80378) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 10:18:12.580093 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.notifications [None req-252af228-75b8-4b4b-89a9-1273ee83847a admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '2ac6c9dcc5cb4be981026eb113525afb'} {{(pid=80378) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 10:18:12.582310 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG dbcounter [-] [80378] Writing DB stats keystone:SELECT=91,keystone:DELETE=8,keystone:INSERT=1 {{(pid=80378) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 10:18:12.590452 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a0da9ab-deb2-4b14-8e15-ccd5f8349cb3 admin admin] Validating token access rules against request {{(pid=80377) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:18:12.592500 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.notifications [None req-252af228-75b8-4b4b-89a9-1273ee83847a admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '2ac6c9dcc5cb4be981026eb113525afb'} {{(pid=80378) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 10:18:12.593421 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.notifications [None req-252af228-75b8-4b4b-89a9-1273ee83847a admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '2ac6c9dcc5cb4be981026eb113525afb'} {{(pid=80378) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 10:18:12.598439 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9a7bdd7-4962-4974-9af9-a94b06413950 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1023de281ae74e9d8a6d6c312c1c97a7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f1b38ebb804c41c69772c998aae72c41', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80377) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:18:12.599086 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9a7bdd7-4962-4974-9af9-a94b06413950 service cinder] REQUEST_METHOD: `GET` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:18:12.599513 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9a7bdd7-4962-4974-9af9-a94b06413950 service cinder] SCRIPT_NAME: `/identity` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:18:12.599804 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9a7bdd7-4962-4974-9af9-a94b06413950 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:18:12.606847 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.fernet_utils [None req-c9a7bdd7-4962-4974-9af9-a94b06413950 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80377) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 10:18:12.610906 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.notifications [None req-252af228-75b8-4b4b-89a9-1273ee83847a admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '2ac6c9dcc5cb4be981026eb113525afb'} {{(pid=80378) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 10:18:12.617538 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 1168/2350] 10.222.0.82 () {58 vars in 1224 bytes} [Tue Aug 2 10:18:12 2022] DELETE /identity/v3/users/2ac6c9dcc5cb4be981026eb113525afb => generated 0 bytes in 297 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 02 10:18:12.688118 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9a7bdd7-4962-4974-9af9-a94b06413950 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:18:12.690179 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9a7bdd7-4962-4974-9af9-a94b06413950 service cinder] RBAC: Authorization granted {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:18:12.705844 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: [pid: 80377|app: 0|req: 1183/2351] 10.222.0.82 () {60 vars in 1365 bytes} [Tue Aug 2 10:18:12 2022] GET /identity/v3/auth/tokens => generated 3262 bytes in 255 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 02 10:18:12.948224 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-252af228-75b8-4b4b-89a9-1273ee83847a admin admin] Authenticating user token {{(pid=80378) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:18:12.949738 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.fernet_utils [None req-252af228-75b8-4b4b-89a9-1273ee83847a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80378) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 10:18:13.059045 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-252af228-75b8-4b4b-89a9-1273ee83847a admin admin] Validating token access rules against request {{(pid=80378) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:18:13.064221 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-728d111b-d769-48dd-94fb-2baa0832b4fd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '891005fa5ff74041b75747d16cc66798', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7047912c5acd4adeba5d87c3bf895e39', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80378) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:18:13.065144 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-728d111b-d769-48dd-94fb-2baa0832b4fd admin admin] REQUEST_METHOD: `DELETE` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:18:13.065531 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-728d111b-d769-48dd-94fb-2baa0832b4fd admin admin] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:18:13.065951 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-728d111b-d769-48dd-94fb-2baa0832b4fd admin admin] PATH_INFO: `/v3/projects/38d139e43bae4406b94f11047bc998a4` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:18:13.067271 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-728d111b-d769-48dd-94fb-2baa0832b4fd admin admin] RBAC: Authorizing `identity:delete_project(project_id=38d139e43bae4406b94f11047bc998a4)` {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:18:13.069379 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-728d111b-d769-48dd-94fb-2baa0832b4fd admin admin] RBAC: Authorization granted {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:18:13.208672 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.notifications [None req-728d111b-d769-48dd-94fb-2baa0832b4fd admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '38d139e43bae4406b94f11047bc998a4', 'request_id': 'req-728d111b-d769-48dd-94fb-2baa0832b4fd'} {{(pid=80378) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 10:18:13.220614 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.notifications [None req-728d111b-d769-48dd-94fb-2baa0832b4fd admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '38d139e43bae4406b94f11047bc998a4', 'request_id': 'req-728d111b-d769-48dd-94fb-2baa0832b4fd'} {{(pid=80378) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 10:18:13.228602 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.notifications [None req-728d111b-d769-48dd-94fb-2baa0832b4fd admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '38d139e43bae4406b94f11047bc998a4', 'request_id': 'req-728d111b-d769-48dd-94fb-2baa0832b4fd'} {{(pid=80378) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 10:18:13.245116 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.notifications [None req-728d111b-d769-48dd-94fb-2baa0832b4fd admin admin] The token cache is being invalidate because project 38d139e43bae4406b94f11047bc998a4 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80378) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 02 10:18:13.246296 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.notifications [None req-728d111b-d769-48dd-94fb-2baa0832b4fd admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80378) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 10:18:13.247079 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.notifications [None req-728d111b-d769-48dd-94fb-2baa0832b4fd admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80378) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 10:18:13.249562 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 1169/2352] 10.222.0.82 () {58 vars in 1233 bytes} [Tue Aug 2 10:18:12 2022] DELETE /identity/v3/projects/38d139e43bae4406b94f11047bc998a4 => generated 0 bytes in 302 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 02 10:18:13.253502 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9a7bdd7-4962-4974-9af9-a94b06413950 service cinder] Authenticating user token {{(pid=80377) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:18:13.269350 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9a7bdd7-4962-4974-9af9-a94b06413950 service cinder] Validating token access rules against request {{(pid=80377) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:18:13.360569 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80864cd7-2afd-403a-908d-e9777ebb41f0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '891005fa5ff74041b75747d16cc66798', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7047912c5acd4adeba5d87c3bf895e39', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80377) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:18:13.362512 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80864cd7-2afd-403a-908d-e9777ebb41f0 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:18:13.363040 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80864cd7-2afd-403a-908d-e9777ebb41f0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:18:13.363518 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80864cd7-2afd-403a-908d-e9777ebb41f0 admin admin] PATH_INFO: `/v3/users/870771ad4e3f4151a1ba07b305968320` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:18:13.365024 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80864cd7-2afd-403a-908d-e9777ebb41f0 admin admin] RBAC: Authorizing `identity:delete_user(user_id=870771ad4e3f4151a1ba07b305968320)` {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:18:13.367366 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80864cd7-2afd-403a-908d-e9777ebb41f0 admin admin] RBAC: Authorization granted {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:18:13.429253 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.notifications [None req-80864cd7-2afd-403a-908d-e9777ebb41f0 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '870771ad4e3f4151a1ba07b305968320'} {{(pid=80377) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 10:18:13.438276 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.notifications [None req-80864cd7-2afd-403a-908d-e9777ebb41f0 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '870771ad4e3f4151a1ba07b305968320'} {{(pid=80377) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 10:18:13.446347 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.notifications [None req-80864cd7-2afd-403a-908d-e9777ebb41f0 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '870771ad4e3f4151a1ba07b305968320'} {{(pid=80377) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 10:18:13.446984 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.notifications [None req-80864cd7-2afd-403a-908d-e9777ebb41f0 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '870771ad4e3f4151a1ba07b305968320'} {{(pid=80377) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 10:18:13.462318 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.notifications [None req-80864cd7-2afd-403a-908d-e9777ebb41f0 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '870771ad4e3f4151a1ba07b305968320'} {{(pid=80377) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 10:18:13.468552 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: [pid: 80377|app: 0|req: 1184/2353] 10.222.0.82 () {58 vars in 1224 bytes} [Tue Aug 2 10:18:13 2022] DELETE /identity/v3/users/870771ad4e3f4151a1ba07b305968320 => generated 0 bytes in 216 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 02 10:18:13.627701 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d8837f3-5506-40a7-8e65-b26f9573e7ce None None] REQUEST_METHOD: `POST` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:18:13.628276 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d8837f3-5506-40a7-8e65-b26f9573e7ce None None] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:18:13.628700 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d8837f3-5506-40a7-8e65-b26f9573e7ce None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:18:13.652538 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.auth.core [None req-7d8837f3-5506-40a7-8e65-b26f9573e7ce None None] MFA Rules not processed for user `a5f2ec904f214a708a3691da24fb516d`. Rule list: `[]` (Enabled: `True`). {{(pid=80378) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 02 10:18:13.654497 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.fernet_utils [None req-7d8837f3-5506-40a7-8e65-b26f9573e7ce None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80378) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 10:18:13.738596 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 1170/2354] 10.222.0.82 () {58 vars in 958 bytes} [Tue Aug 2 10:18:13 2022] POST /identity/v3/auth/tokens => generated 3248 bytes in 112 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 02 10:18:13.809739 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80864cd7-2afd-403a-908d-e9777ebb41f0 admin admin] Authenticating user token {{(pid=80377) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:18:13.811314 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.fernet_utils [None req-80864cd7-2afd-403a-908d-e9777ebb41f0 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80377) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 10:18:13.841720 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG dbcounter [-] [80377] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80377) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 10:18:13.905941 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80864cd7-2afd-403a-908d-e9777ebb41f0 admin admin] Validating token access rules against request {{(pid=80377) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:18:13.910971 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8bd5a31-2399-4fc5-852a-5afeb3402eea admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '891005fa5ff74041b75747d16cc66798', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7047912c5acd4adeba5d87c3bf895e39', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80377) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:18:13.911648 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8bd5a31-2399-4fc5-852a-5afeb3402eea admin admin] REQUEST_METHOD: `DELETE` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:18:13.911931 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8bd5a31-2399-4fc5-852a-5afeb3402eea admin admin] SCRIPT_NAME: `/identity` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:18:13.912203 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8bd5a31-2399-4fc5-852a-5afeb3402eea admin admin] PATH_INFO: `/v3/projects/8c47a826f47544a5977034bf3067629f` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:18:13.913359 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8bd5a31-2399-4fc5-852a-5afeb3402eea admin admin] RBAC: Authorizing `identity:delete_project(project_id=8c47a826f47544a5977034bf3067629f)` {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:18:13.915295 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8bd5a31-2399-4fc5-852a-5afeb3402eea admin admin] RBAC: Authorization granted {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:18:13.963024 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.notifications [None req-e8bd5a31-2399-4fc5-852a-5afeb3402eea admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '8c47a826f47544a5977034bf3067629f', 'request_id': 'req-e8bd5a31-2399-4fc5-852a-5afeb3402eea'} {{(pid=80377) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 10:18:13.969349 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.notifications [None req-e8bd5a31-2399-4fc5-852a-5afeb3402eea admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '8c47a826f47544a5977034bf3067629f', 'request_id': 'req-e8bd5a31-2399-4fc5-852a-5afeb3402eea'} {{(pid=80377) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 10:18:13.976186 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.notifications [None req-e8bd5a31-2399-4fc5-852a-5afeb3402eea admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '8c47a826f47544a5977034bf3067629f', 'request_id': 'req-e8bd5a31-2399-4fc5-852a-5afeb3402eea'} {{(pid=80377) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 10:18:13.990197 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.notifications [None req-e8bd5a31-2399-4fc5-852a-5afeb3402eea admin admin] The token cache is being invalidate because project 8c47a826f47544a5977034bf3067629f was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80377) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 02 10:18:13.990879 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.notifications [None req-e8bd5a31-2399-4fc5-852a-5afeb3402eea admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80377) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 10:18:13.991692 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.notifications [None req-e8bd5a31-2399-4fc5-852a-5afeb3402eea admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80377) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 10:18:13.993399 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: [pid: 80377|app: 0|req: 1185/2355] 10.222.0.82 () {58 vars in 1233 bytes} [Tue Aug 2 10:18:13 2022] DELETE /identity/v3/projects/8c47a826f47544a5977034bf3067629f => generated 0 bytes in 184 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 02 10:18:13.998969 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e05fae9-4f9a-4a68-914f-6b871f4f8683 None None] REQUEST_METHOD: `POST` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:18:13.999541 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e05fae9-4f9a-4a68-914f-6b871f4f8683 None None] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:18:13.999541 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e05fae9-4f9a-4a68-914f-6b871f4f8683 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:18:14.014706 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG dbcounter [-] [80378] Writing DB stats keystone:SELECT=90,keystone:DELETE=9,keystone:INSERT=1 {{(pid=80378) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 10:18:14.021334 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.auth.core [None req-6e05fae9-4f9a-4a68-914f-6b871f4f8683 None None] MFA Rules not processed for user `891005fa5ff74041b75747d16cc66798`. Rule list: `[]` (Enabled: `True`). {{(pid=80378) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 02 10:18:14.023594 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.fernet_utils [None req-6e05fae9-4f9a-4a68-914f-6b871f4f8683 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80378) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 10:18:14.111031 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 1171/2356] 10.222.0.82 () {58 vars in 958 bytes} [Tue Aug 2 10:18:13 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 114 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 02 10:18:14.119122 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ff5a8ec-ce29-4c62-a977-882d07d91c43 None None] REQUEST_METHOD: `POST` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:18:14.119519 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ff5a8ec-ce29-4c62-a977-882d07d91c43 None None] SCRIPT_NAME: `/identity` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:18:14.119903 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ff5a8ec-ce29-4c62-a977-882d07d91c43 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:18:14.143901 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.auth.core [None req-4ff5a8ec-ce29-4c62-a977-882d07d91c43 None None] MFA Rules not processed for user `891005fa5ff74041b75747d16cc66798`. Rule list: `[]` (Enabled: `True`). {{(pid=80377) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 02 10:18:14.145343 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.fernet_utils [None req-4ff5a8ec-ce29-4c62-a977-882d07d91c43 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80377) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 10:18:14.154040 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: [pid: 80377|app: 0|req: 1186/2357] 10.222.0.82 () {58 vars in 958 bytes} [Tue Aug 2 10:18:14 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 36 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 02 10:18:14.159348 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e05fae9-4f9a-4a68-914f-6b871f4f8683 None None] Authenticating user token {{(pid=80378) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:18:14.174586 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e05fae9-4f9a-4a68-914f-6b871f4f8683 None None] Validating token access rules against request {{(pid=80378) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:18:14.189832 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dab48543-66bf-46a8-bf1e-1871490dea83 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '891005fa5ff74041b75747d16cc66798', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7047912c5acd4adeba5d87c3bf895e39', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80378) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:18:14.191234 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dab48543-66bf-46a8-bf1e-1871490dea83 admin admin] REQUEST_METHOD: `GET` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:18:14.192213 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dab48543-66bf-46a8-bf1e-1871490dea83 admin admin] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:18:14.193223 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dab48543-66bf-46a8-bf1e-1871490dea83 admin admin] PATH_INFO: `/v3/domains` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:18:14.194568 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dab48543-66bf-46a8-bf1e-1871490dea83 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80378) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 02 10:18:14.197828 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dab48543-66bf-46a8-bf1e-1871490dea83 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:18:14.198543 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dab48543-66bf-46a8-bf1e-1871490dea83 admin admin] RBAC: Authorization granted {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:18:14.216427 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 1172/2358] 10.222.0.82 () {58 vars in 1153 bytes} [Tue Aug 2 10:18:14 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 57 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 02 10:18:14.221072 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ff5a8ec-ce29-4c62-a977-882d07d91c43 None None] Authenticating user token {{(pid=80377) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:18:14.233119 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ff5a8ec-ce29-4c62-a977-882d07d91c43 None None] Validating token access rules against request {{(pid=80377) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:18:14.239608 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c696e110-79f2-4638-9b1a-6403eab0df42 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '891005fa5ff74041b75747d16cc66798', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7047912c5acd4adeba5d87c3bf895e39', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80377) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:18:14.240191 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c696e110-79f2-4638-9b1a-6403eab0df42 admin admin] REQUEST_METHOD: `POST` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:18:14.240393 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c696e110-79f2-4638-9b1a-6403eab0df42 admin admin] SCRIPT_NAME: `/identity` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:18:14.240764 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c696e110-79f2-4638-9b1a-6403eab0df42 admin admin] PATH_INFO: `/v3/projects` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:18:14.242034 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c696e110-79f2-4638-9b1a-6403eab0df42 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:18:14.245426 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c696e110-79f2-4638-9b1a-6403eab0df42 admin admin] RBAC: Authorization granted {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:18:14.291908 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: [pid: 80377|app: 0|req: 1187/2359] 10.222.0.82 () {60 vars in 1153 bytes} [Tue Aug 2 10:18:14 2022] POST /identity/v3/projects => generated 363 bytes in 71 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 02 10:18:14.296772 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dab48543-66bf-46a8-bf1e-1871490dea83 admin admin] Authenticating user token {{(pid=80378) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:18:14.313691 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dab48543-66bf-46a8-bf1e-1871490dea83 admin admin] Validating token access rules against request {{(pid=80378) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:18:14.377734 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-189b2150-ab15-4aa1-a145-e9ca23610a13 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '891005fa5ff74041b75747d16cc66798', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7047912c5acd4adeba5d87c3bf895e39', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80378) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:18:14.378400 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-189b2150-ab15-4aa1-a145-e9ca23610a13 admin admin] REQUEST_METHOD: `POST` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:18:14.378684 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-189b2150-ab15-4aa1-a145-e9ca23610a13 admin admin] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:18:14.379124 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-189b2150-ab15-4aa1-a145-e9ca23610a13 admin admin] PATH_INFO: `/v3/users` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:18:14.379801 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-189b2150-ab15-4aa1-a145-e9ca23610a13 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:18:14.381727 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-189b2150-ab15-4aa1-a145-e9ca23610a13 admin admin] RBAC: Authorization granted {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:18:14.410628 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 1173/2360] 10.222.0.82 () {60 vars in 1144 bytes} [Tue Aug 2 10:18:14 2022] POST /identity/v3/users => generated 282 bytes in 114 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 02 10:18:14.415024 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c696e110-79f2-4638-9b1a-6403eab0df42 admin admin] Authenticating user token {{(pid=80377) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:18:14.427103 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c696e110-79f2-4638-9b1a-6403eab0df42 admin admin] Validating token access rules against request {{(pid=80377) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:18:14.433528 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c96da38c-0b7a-4f99-961f-d44eedb87e01 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '891005fa5ff74041b75747d16cc66798', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7047912c5acd4adeba5d87c3bf895e39', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80377) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:18:14.434365 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c96da38c-0b7a-4f99-961f-d44eedb87e01 admin admin] REQUEST_METHOD: `GET` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:18:14.436026 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c96da38c-0b7a-4f99-961f-d44eedb87e01 admin admin] SCRIPT_NAME: `/identity` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:18:14.436026 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c96da38c-0b7a-4f99-961f-d44eedb87e01 admin admin] PATH_INFO: `/v3/roles` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:18:14.436843 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c96da38c-0b7a-4f99-961f-d44eedb87e01 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:18:14.438685 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c96da38c-0b7a-4f99-961f-d44eedb87e01 admin admin] RBAC: Authorization granted {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:18:14.453974 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: [pid: 80377|app: 0|req: 1188/2361] 10.222.0.82 () {58 vars in 1122 bytes} [Tue Aug 2 10:18:14 2022] GET /identity/v3/roles => generated 1420 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 02 10:18:14.455526 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-189b2150-ab15-4aa1-a145-e9ca23610a13 admin admin] Authenticating user token {{(pid=80378) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:18:14.475408 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-189b2150-ab15-4aa1-a145-e9ca23610a13 admin admin] Validating token access rules against request {{(pid=80378) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:18:14.480448 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bea3129-22e1-4846-ae8d-f5e0a737cd69 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '891005fa5ff74041b75747d16cc66798', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7047912c5acd4adeba5d87c3bf895e39', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80378) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:18:14.481183 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bea3129-22e1-4846-ae8d-f5e0a737cd69 admin admin] REQUEST_METHOD: `GET` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:18:14.481575 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bea3129-22e1-4846-ae8d-f5e0a737cd69 admin admin] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:18:14.482011 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bea3129-22e1-4846-ae8d-f5e0a737cd69 admin admin] PATH_INFO: `/v3/roles` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:18:14.482905 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7bea3129-22e1-4846-ae8d-f5e0a737cd69 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:18:14.484745 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7bea3129-22e1-4846-ae8d-f5e0a737cd69 admin admin] RBAC: Authorization granted {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:18:14.496361 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 1174/2362] 10.222.0.82 () {58 vars in 1122 bytes} [Tue Aug 2 10:18:14 2022] GET /identity/v3/roles => generated 1420 bytes in 41 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 02 10:18:14.500785 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c96da38c-0b7a-4f99-961f-d44eedb87e01 admin admin] Authenticating user token {{(pid=80377) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:18:14.510969 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c96da38c-0b7a-4f99-961f-d44eedb87e01 admin admin] Validating token access rules against request {{(pid=80377) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:18:14.522429 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e31bd83-e520-4485-9ce5-08e6b6c318ec admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '891005fa5ff74041b75747d16cc66798', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7047912c5acd4adeba5d87c3bf895e39', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80377) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:18:14.522429 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e31bd83-e520-4485-9ce5-08e6b6c318ec admin admin] REQUEST_METHOD: `PUT` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:18:14.522429 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e31bd83-e520-4485-9ce5-08e6b6c318ec admin admin] SCRIPT_NAME: `/identity` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:18:14.522429 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e31bd83-e520-4485-9ce5-08e6b6c318ec admin admin] PATH_INFO: `/v3/projects/ccf59d67afaf4235bcc5eb1b7abb71fe/users/1e7b3d7f3da64f90b6ca4f19b7ca9a02/roles/e1c04b19fc4543b8b54f58bbcffbc598` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:18:14.543638 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e31bd83-e520-4485-9ce5-08e6b6c318ec admin admin] RBAC: Authorizing `identity:create_grant(project_id=ccf59d67afaf4235bcc5eb1b7abb71fe, user_id=1e7b3d7f3da64f90b6ca4f19b7ca9a02, role_id=e1c04b19fc4543b8b54f58bbcffbc598)` {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:18:14.562567 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e31bd83-e520-4485-9ce5-08e6b6c318ec admin admin] RBAC: Authorization granted {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:18:14.586542 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: [pid: 80377|app: 0|req: 1189/2363] 10.222.0.82 () {60 vars in 1483 bytes} [Tue Aug 2 10:18:14 2022] PUT /identity/v3/projects/ccf59d67afaf4235bcc5eb1b7abb71fe/users/1e7b3d7f3da64f90b6ca4f19b7ca9a02/roles/e1c04b19fc4543b8b54f58bbcffbc598 => generated 0 bytes in 86 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 02 10:18:14.599038 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-beb3068a-8f41-4ab9-aebd-3cf6dc20d5fd None None] REQUEST_METHOD: `POST` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:18:14.599328 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-beb3068a-8f41-4ab9-aebd-3cf6dc20d5fd None None] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:18:14.599622 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-beb3068a-8f41-4ab9-aebd-3cf6dc20d5fd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:18:14.644555 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.auth.core [None req-beb3068a-8f41-4ab9-aebd-3cf6dc20d5fd None None] MFA Rules not processed for user `1e7b3d7f3da64f90b6ca4f19b7ca9a02`. Rule list: `[]` (Enabled: `True`). {{(pid=80378) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 02 10:18:14.645952 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.fernet_utils [None req-beb3068a-8f41-4ab9-aebd-3cf6dc20d5fd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80378) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 10:18:14.812274 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 1175/2364] 10.222.0.82 () {58 vars in 958 bytes} [Tue Aug 2 10:18:14 2022] POST /identity/v3/auth/tokens => generated 3240 bytes in 214 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 02 10:18:14.824689 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e31bd83-e520-4485-9ce5-08e6b6c318ec admin admin] Authenticating user token {{(pid=80377) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:18:14.827151 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.fernet_utils [None req-1e31bd83-e520-4485-9ce5-08e6b6c318ec admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80377) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 10:18:14.923003 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG dbcounter [-] [80377] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=80377) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 10:18:14.962025 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e31bd83-e520-4485-9ce5-08e6b6c318ec admin admin] Validating token access rules against request {{(pid=80377) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:18:14.974790 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2356eb0e-a68d-4c58-8d7a-4e371effc13f service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1023de281ae74e9d8a6d6c312c1c97a7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f1b38ebb804c41c69772c998aae72c41', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80377) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:18:14.975447 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2356eb0e-a68d-4c58-8d7a-4e371effc13f service cinder] REQUEST_METHOD: `GET` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:18:14.975726 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2356eb0e-a68d-4c58-8d7a-4e371effc13f service cinder] SCRIPT_NAME: `/identity` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:18:14.975998 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2356eb0e-a68d-4c58-8d7a-4e371effc13f service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:18:14.988940 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2356eb0e-a68d-4c58-8d7a-4e371effc13f service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:18:14.990641 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2356eb0e-a68d-4c58-8d7a-4e371effc13f service cinder] RBAC: Authorization granted {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:18:15.013205 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: [pid: 80377|app: 0|req: 1190/2365] 10.222.0.82 () {60 vars in 1365 bytes} [Tue Aug 2 10:18:14 2022] GET /identity/v3/auth/tokens => generated 3240 bytes in 189 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 02 10:18:21.362638 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-beb3068a-8f41-4ab9-aebd-3cf6dc20d5fd None None] Authenticating user token {{(pid=80378) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:18:21.364108 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.fernet_utils [None req-beb3068a-8f41-4ab9-aebd-3cf6dc20d5fd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80378) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 10:18:21.371092 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG dbcounter [-] [80378] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80378) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 10:18:21.378337 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-beb3068a-8f41-4ab9-aebd-3cf6dc20d5fd None None] Validating token access rules against request {{(pid=80378) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:18:21.382156 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-158e0b5b-0cfc-4d3a-986c-d43657096128 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1023de281ae74e9d8a6d6c312c1c97a7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f1b38ebb804c41c69772c998aae72c41', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80378) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:18:21.382770 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-158e0b5b-0cfc-4d3a-986c-d43657096128 service cinder] REQUEST_METHOD: `GET` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:18:21.383081 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-158e0b5b-0cfc-4d3a-986c-d43657096128 service cinder] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:18:21.383356 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-158e0b5b-0cfc-4d3a-986c-d43657096128 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:18:21.385085 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.fernet_utils [None req-158e0b5b-0cfc-4d3a-986c-d43657096128 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80378) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 10:18:21.444772 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-158e0b5b-0cfc-4d3a-986c-d43657096128 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:18:21.446265 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-158e0b5b-0cfc-4d3a-986c-d43657096128 service cinder] RBAC: Authorization granted {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:18:21.455068 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 1176/2366] 10.222.0.82 () {60 vars in 1365 bytes} [Tue Aug 2 10:18:21 2022] GET /identity/v3/auth/tokens => generated 3262 bytes in 93 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 02 10:18:23.570249 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2356eb0e-a68d-4c58-8d7a-4e371effc13f service cinder] Authenticating user token {{(pid=80377) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:18:23.571690 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.fernet_utils [None req-2356eb0e-a68d-4c58-8d7a-4e371effc13f service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80377) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 10:18:23.645629 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2356eb0e-a68d-4c58-8d7a-4e371effc13f service cinder] Validating token access rules against request {{(pid=80377) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:18:23.650753 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33e52939-44de-401b-b1db-5c2a5692d485 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '891005fa5ff74041b75747d16cc66798', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7047912c5acd4adeba5d87c3bf895e39', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80377) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:18:23.651406 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33e52939-44de-401b-b1db-5c2a5692d485 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:18:23.651689 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33e52939-44de-401b-b1db-5c2a5692d485 admin admin] SCRIPT_NAME: `/identity` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:18:23.651969 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33e52939-44de-401b-b1db-5c2a5692d485 admin admin] PATH_INFO: `/v3/users/585f44d1d85e4ef3be39358f28f70a98` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:18:23.653173 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33e52939-44de-401b-b1db-5c2a5692d485 admin admin] RBAC: Authorizing `identity:delete_user(user_id=585f44d1d85e4ef3be39358f28f70a98)` {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:18:23.655699 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33e52939-44de-401b-b1db-5c2a5692d485 admin admin] RBAC: Authorization granted {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:18:23.703041 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.notifications [None req-33e52939-44de-401b-b1db-5c2a5692d485 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '585f44d1d85e4ef3be39358f28f70a98'} {{(pid=80377) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 10:18:23.710214 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.notifications [None req-33e52939-44de-401b-b1db-5c2a5692d485 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '585f44d1d85e4ef3be39358f28f70a98'} {{(pid=80377) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 10:18:23.717876 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.notifications [None req-33e52939-44de-401b-b1db-5c2a5692d485 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '585f44d1d85e4ef3be39358f28f70a98'} {{(pid=80377) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 10:18:23.718531 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.notifications [None req-33e52939-44de-401b-b1db-5c2a5692d485 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '585f44d1d85e4ef3be39358f28f70a98'} {{(pid=80377) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 10:18:23.730980 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.notifications [None req-33e52939-44de-401b-b1db-5c2a5692d485 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '585f44d1d85e4ef3be39358f28f70a98'} {{(pid=80377) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 10:18:23.736895 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: [pid: 80377|app: 0|req: 1191/2367] 10.222.0.82 () {58 vars in 1224 bytes} [Tue Aug 2 10:18:23 2022] DELETE /identity/v3/users/585f44d1d85e4ef3be39358f28f70a98 => generated 0 bytes in 167 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 02 10:18:23.741547 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc7e476f-62ed-4ea0-9ba6-d3d03da955df None None] REQUEST_METHOD: `POST` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:18:23.742055 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc7e476f-62ed-4ea0-9ba6-d3d03da955df None None] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:18:23.742170 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc7e476f-62ed-4ea0-9ba6-d3d03da955df None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:18:23.761720 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.auth.core [None req-dc7e476f-62ed-4ea0-9ba6-d3d03da955df None None] MFA Rules not processed for user `891005fa5ff74041b75747d16cc66798`. Rule list: `[]` (Enabled: `True`). {{(pid=80378) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 02 10:18:23.762960 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.fernet_utils [None req-dc7e476f-62ed-4ea0-9ba6-d3d03da955df None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80378) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 10:18:23.824737 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 1177/2368] 10.222.0.82 () {58 vars in 958 bytes} [Tue Aug 2 10:18:23 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 02 10:18:23.834797 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33e52939-44de-401b-b1db-5c2a5692d485 admin admin] Authenticating user token {{(pid=80377) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:18:23.836212 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.fernet_utils [None req-33e52939-44de-401b-b1db-5c2a5692d485 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80377) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 10:18:23.893471 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33e52939-44de-401b-b1db-5c2a5692d485 admin admin] Validating token access rules against request {{(pid=80377) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:18:23.897045 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2cad103-0c43-41e2-a08b-7773c5e00f89 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '33b87170f0634b67bce60750847b5cbe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f1b38ebb804c41c69772c998aae72c41', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80377) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:18:23.897672 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2cad103-0c43-41e2-a08b-7773c5e00f89 service neutron] REQUEST_METHOD: `GET` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:18:23.897996 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2cad103-0c43-41e2-a08b-7773c5e00f89 service neutron] SCRIPT_NAME: `/identity` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:18:23.898276 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2cad103-0c43-41e2-a08b-7773c5e00f89 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:18:23.907232 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2cad103-0c43-41e2-a08b-7773c5e00f89 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:18:23.908666 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2cad103-0c43-41e2-a08b-7773c5e00f89 service neutron] RBAC: Authorization granted {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:18:23.918149 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: [pid: 80377|app: 0|req: 1192/2369] 10.222.0.82 () {60 vars in 1365 bytes} [Tue Aug 2 10:18:23 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 84 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 02 10:18:24.195919 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc7e476f-62ed-4ea0-9ba6-d3d03da955df None None] Authenticating user token {{(pid=80378) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:18:24.197373 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.fernet_utils [None req-dc7e476f-62ed-4ea0-9ba6-d3d03da955df None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80378) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 10:18:24.209891 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc7e476f-62ed-4ea0-9ba6-d3d03da955df None None] Validating token access rules against request {{(pid=80378) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:18:24.214520 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee400258-b94b-46f8-83ba-fc44f17f6098 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '891005fa5ff74041b75747d16cc66798', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7047912c5acd4adeba5d87c3bf895e39', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80378) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:18:24.215233 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee400258-b94b-46f8-83ba-fc44f17f6098 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:18:24.215447 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee400258-b94b-46f8-83ba-fc44f17f6098 admin admin] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:18:24.215725 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee400258-b94b-46f8-83ba-fc44f17f6098 admin admin] PATH_INFO: `/v3/projects/e6bd05cc56c845e2966c00307d49cec0` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:18:24.217407 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee400258-b94b-46f8-83ba-fc44f17f6098 admin admin] RBAC: Authorizing `identity:delete_project(project_id=e6bd05cc56c845e2966c00307d49cec0)` {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:18:24.218872 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee400258-b94b-46f8-83ba-fc44f17f6098 admin admin] RBAC: Authorization granted {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:18:24.266339 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.notifications [None req-ee400258-b94b-46f8-83ba-fc44f17f6098 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'e6bd05cc56c845e2966c00307d49cec0', 'request_id': 'req-ee400258-b94b-46f8-83ba-fc44f17f6098'} {{(pid=80378) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 10:18:24.272589 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.notifications [None req-ee400258-b94b-46f8-83ba-fc44f17f6098 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'e6bd05cc56c845e2966c00307d49cec0', 'request_id': 'req-ee400258-b94b-46f8-83ba-fc44f17f6098'} {{(pid=80378) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 10:18:24.278279 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.notifications [None req-ee400258-b94b-46f8-83ba-fc44f17f6098 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'e6bd05cc56c845e2966c00307d49cec0', 'request_id': 'req-ee400258-b94b-46f8-83ba-fc44f17f6098'} {{(pid=80378) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 10:18:24.291028 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.notifications [None req-ee400258-b94b-46f8-83ba-fc44f17f6098 admin admin] The token cache is being invalidate because project e6bd05cc56c845e2966c00307d49cec0 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80378) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 02 10:18:24.291322 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.notifications [None req-ee400258-b94b-46f8-83ba-fc44f17f6098 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80378) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 10:18:24.291979 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.notifications [None req-ee400258-b94b-46f8-83ba-fc44f17f6098 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80378) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 10:18:24.293482 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 1178/2370] 10.222.0.82 () {58 vars in 1233 bytes} [Tue Aug 2 10:18:24 2022] DELETE /identity/v3/projects/e6bd05cc56c845e2966c00307d49cec0 => generated 0 bytes in 98 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 02 10:18:24.297972 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2cad103-0c43-41e2-a08b-7773c5e00f89 service neutron] Authenticating user token {{(pid=80377) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:18:24.307886 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2cad103-0c43-41e2-a08b-7773c5e00f89 service neutron] Validating token access rules against request {{(pid=80377) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:18:24.313129 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG dbcounter [-] [80377] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80377) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 10:18:24.369873 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a351705-3756-46b8-bd46-ba03ae95b64e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '891005fa5ff74041b75747d16cc66798', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7047912c5acd4adeba5d87c3bf895e39', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80377) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:18:24.370533 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a351705-3756-46b8-bd46-ba03ae95b64e admin admin] REQUEST_METHOD: `DELETE` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:18:24.370819 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a351705-3756-46b8-bd46-ba03ae95b64e admin admin] SCRIPT_NAME: `/identity` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:18:24.371099 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a351705-3756-46b8-bd46-ba03ae95b64e admin admin] PATH_INFO: `/v3/users/199d9fc41db04feea1a1e1c9bb817dc0` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:18:24.372285 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a351705-3756-46b8-bd46-ba03ae95b64e admin admin] RBAC: Authorizing `identity:delete_user(user_id=199d9fc41db04feea1a1e1c9bb817dc0)` {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:18:24.374905 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a351705-3756-46b8-bd46-ba03ae95b64e admin admin] RBAC: Authorization granted {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:18:24.422278 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.notifications [None req-3a351705-3756-46b8-bd46-ba03ae95b64e admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '199d9fc41db04feea1a1e1c9bb817dc0'} {{(pid=80377) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 10:18:24.429669 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.notifications [None req-3a351705-3756-46b8-bd46-ba03ae95b64e admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '199d9fc41db04feea1a1e1c9bb817dc0'} {{(pid=80377) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 10:18:24.437417 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.notifications [None req-3a351705-3756-46b8-bd46-ba03ae95b64e admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '199d9fc41db04feea1a1e1c9bb817dc0'} {{(pid=80377) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 10:18:24.438074 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.notifications [None req-3a351705-3756-46b8-bd46-ba03ae95b64e admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '199d9fc41db04feea1a1e1c9bb817dc0'} {{(pid=80377) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 10:18:24.449698 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.notifications [None req-3a351705-3756-46b8-bd46-ba03ae95b64e admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '199d9fc41db04feea1a1e1c9bb817dc0'} {{(pid=80377) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 10:18:24.455719 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: [pid: 80377|app: 0|req: 1193/2371] 10.222.0.82 () {58 vars in 1224 bytes} [Tue Aug 2 10:18:24 2022] DELETE /identity/v3/users/199d9fc41db04feea1a1e1c9bb817dc0 => generated 0 bytes in 158 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 02 10:18:24.756177 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee400258-b94b-46f8-83ba-fc44f17f6098 admin admin] Authenticating user token {{(pid=80378) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:18:24.757975 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.fernet_utils [None req-ee400258-b94b-46f8-83ba-fc44f17f6098 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80378) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 10:18:24.777175 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee400258-b94b-46f8-83ba-fc44f17f6098 admin admin] Validating token access rules against request {{(pid=80378) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:18:24.801109 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG dbcounter [-] [80378] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80378) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 10:18:24.840974 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d1752d6-847b-4fe9-a7a8-62f37a573b5b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '891005fa5ff74041b75747d16cc66798', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7047912c5acd4adeba5d87c3bf895e39', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80378) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:18:24.841626 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d1752d6-847b-4fe9-a7a8-62f37a573b5b admin admin] REQUEST_METHOD: `DELETE` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:18:24.842471 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d1752d6-847b-4fe9-a7a8-62f37a573b5b admin admin] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:18:24.842748 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d1752d6-847b-4fe9-a7a8-62f37a573b5b admin admin] PATH_INFO: `/v3/projects/e713ef261a4a4d74a43d5d8974e61af3` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:18:24.843911 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d1752d6-847b-4fe9-a7a8-62f37a573b5b admin admin] RBAC: Authorizing `identity:delete_project(project_id=e713ef261a4a4d74a43d5d8974e61af3)` {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:18:24.846433 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d1752d6-847b-4fe9-a7a8-62f37a573b5b admin admin] RBAC: Authorization granted {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:18:24.892310 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.notifications [None req-2d1752d6-847b-4fe9-a7a8-62f37a573b5b admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'e713ef261a4a4d74a43d5d8974e61af3', 'request_id': 'req-2d1752d6-847b-4fe9-a7a8-62f37a573b5b'} {{(pid=80378) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 10:18:24.898770 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.notifications [None req-2d1752d6-847b-4fe9-a7a8-62f37a573b5b admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'e713ef261a4a4d74a43d5d8974e61af3', 'request_id': 'req-2d1752d6-847b-4fe9-a7a8-62f37a573b5b'} {{(pid=80378) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 10:18:24.905418 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.notifications [None req-2d1752d6-847b-4fe9-a7a8-62f37a573b5b admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'e713ef261a4a4d74a43d5d8974e61af3', 'request_id': 'req-2d1752d6-847b-4fe9-a7a8-62f37a573b5b'} {{(pid=80378) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 10:18:24.917688 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.notifications [None req-2d1752d6-847b-4fe9-a7a8-62f37a573b5b admin admin] The token cache is being invalidate because project e713ef261a4a4d74a43d5d8974e61af3 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80378) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 02 10:18:24.918014 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.notifications [None req-2d1752d6-847b-4fe9-a7a8-62f37a573b5b admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80378) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 10:18:24.918642 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.notifications [None req-2d1752d6-847b-4fe9-a7a8-62f37a573b5b admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80378) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 10:18:24.920188 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 1179/2372] 10.222.0.82 () {58 vars in 1233 bytes} [Tue Aug 2 10:18:24 2022] DELETE /identity/v3/projects/e713ef261a4a4d74a43d5d8974e61af3 => generated 0 bytes in 165 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 02 10:18:24.924193 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a351705-3756-46b8-bd46-ba03ae95b64e admin admin] Authenticating user token {{(pid=80377) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:18:24.925674 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.fernet_utils [None req-3a351705-3756-46b8-bd46-ba03ae95b64e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80377) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 10:18:24.938846 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a351705-3756-46b8-bd46-ba03ae95b64e admin admin] Validating token access rules against request {{(pid=80377) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:18:24.995162 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-127a3fd6-c155-4433-9efe-0fbe0abc9cb6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '891005fa5ff74041b75747d16cc66798', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7047912c5acd4adeba5d87c3bf895e39', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80377) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:18:24.995816 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-127a3fd6-c155-4433-9efe-0fbe0abc9cb6 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:18:24.996139 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-127a3fd6-c155-4433-9efe-0fbe0abc9cb6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:18:24.996418 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-127a3fd6-c155-4433-9efe-0fbe0abc9cb6 admin admin] PATH_INFO: `/v3/users/5ae3e3e0ec7646e187a45d1b0045c016` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:18:24.997606 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-127a3fd6-c155-4433-9efe-0fbe0abc9cb6 admin admin] RBAC: Authorizing `identity:delete_user(user_id=5ae3e3e0ec7646e187a45d1b0045c016)` {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:18:24.999635 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-127a3fd6-c155-4433-9efe-0fbe0abc9cb6 admin admin] RBAC: Authorization granted {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:18:25.036184 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG dbcounter [-] [80377] Writing DB stats keystone:SELECT=84,keystone:DELETE=15,keystone:INSERT=1 {{(pid=80377) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 10:18:25.052528 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.notifications [None req-127a3fd6-c155-4433-9efe-0fbe0abc9cb6 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '5ae3e3e0ec7646e187a45d1b0045c016'} {{(pid=80377) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 10:18:25.061082 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.notifications [None req-127a3fd6-c155-4433-9efe-0fbe0abc9cb6 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '5ae3e3e0ec7646e187a45d1b0045c016'} {{(pid=80377) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 10:18:25.068151 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.notifications [None req-127a3fd6-c155-4433-9efe-0fbe0abc9cb6 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '5ae3e3e0ec7646e187a45d1b0045c016'} {{(pid=80377) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 10:18:25.068808 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.notifications [None req-127a3fd6-c155-4433-9efe-0fbe0abc9cb6 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '5ae3e3e0ec7646e187a45d1b0045c016'} {{(pid=80377) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 10:18:25.083250 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.notifications [None req-127a3fd6-c155-4433-9efe-0fbe0abc9cb6 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '5ae3e3e0ec7646e187a45d1b0045c016'} {{(pid=80377) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 10:18:25.089702 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: [pid: 80377|app: 0|req: 1194/2373] 10.222.0.82 () {58 vars in 1224 bytes} [Tue Aug 2 10:18:24 2022] DELETE /identity/v3/users/5ae3e3e0ec7646e187a45d1b0045c016 => generated 0 bytes in 166 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 02 10:18:25.382213 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d1752d6-847b-4fe9-a7a8-62f37a573b5b admin admin] Authenticating user token {{(pid=80378) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:18:25.383774 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.fernet_utils [None req-2d1752d6-847b-4fe9-a7a8-62f37a573b5b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80378) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 10:18:25.406353 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d1752d6-847b-4fe9-a7a8-62f37a573b5b admin admin] Validating token access rules against request {{(pid=80378) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:18:25.474550 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a977b34-db16-41dd-9cfe-4d9965cb716d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '891005fa5ff74041b75747d16cc66798', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7047912c5acd4adeba5d87c3bf895e39', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80378) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:18:25.475461 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a977b34-db16-41dd-9cfe-4d9965cb716d admin admin] REQUEST_METHOD: `DELETE` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:18:25.475873 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a977b34-db16-41dd-9cfe-4d9965cb716d admin admin] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:18:25.476260 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a977b34-db16-41dd-9cfe-4d9965cb716d admin admin] PATH_INFO: `/v3/projects/0be9d5f87bf3453493d7b9625a85bbda` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:18:25.477671 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a977b34-db16-41dd-9cfe-4d9965cb716d admin admin] RBAC: Authorizing `identity:delete_project(project_id=0be9d5f87bf3453493d7b9625a85bbda)` {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:18:25.480029 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a977b34-db16-41dd-9cfe-4d9965cb716d admin admin] RBAC: Authorization granted {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:18:25.532128 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.notifications [None req-0a977b34-db16-41dd-9cfe-4d9965cb716d admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '0be9d5f87bf3453493d7b9625a85bbda', 'request_id': 'req-0a977b34-db16-41dd-9cfe-4d9965cb716d'} {{(pid=80378) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 10:18:25.539646 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.notifications [None req-0a977b34-db16-41dd-9cfe-4d9965cb716d admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '0be9d5f87bf3453493d7b9625a85bbda', 'request_id': 'req-0a977b34-db16-41dd-9cfe-4d9965cb716d'} {{(pid=80378) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 10:18:25.543003 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG dbcounter [-] [80378] Writing DB stats keystone:SELECT=87,keystone:DELETE=12,keystone:INSERT=1 {{(pid=80378) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 10:18:25.548070 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.notifications [None req-0a977b34-db16-41dd-9cfe-4d9965cb716d admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '0be9d5f87bf3453493d7b9625a85bbda', 'request_id': 'req-0a977b34-db16-41dd-9cfe-4d9965cb716d'} {{(pid=80378) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 10:18:25.575549 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.notifications [None req-0a977b34-db16-41dd-9cfe-4d9965cb716d admin admin] The token cache is being invalidate because project 0be9d5f87bf3453493d7b9625a85bbda was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80378) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 02 10:18:25.576288 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.notifications [None req-0a977b34-db16-41dd-9cfe-4d9965cb716d admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80378) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 10:18:25.577579 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.notifications [None req-0a977b34-db16-41dd-9cfe-4d9965cb716d admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80378) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 10:18:25.579336 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 1180/2374] 10.222.0.82 () {58 vars in 1233 bytes} [Tue Aug 2 10:18:25 2022] DELETE /identity/v3/projects/0be9d5f87bf3453493d7b9625a85bbda => generated 0 bytes in 197 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 02 10:18:25.587296 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00de2a01-1a46-4ee6-83fc-c2f6e73392be None None] REQUEST_METHOD: `POST` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:18:25.587588 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00de2a01-1a46-4ee6-83fc-c2f6e73392be None None] SCRIPT_NAME: `/identity` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:18:25.587870 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00de2a01-1a46-4ee6-83fc-c2f6e73392be None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:18:25.610099 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.auth.core [None req-00de2a01-1a46-4ee6-83fc-c2f6e73392be None None] MFA Rules not processed for user `891005fa5ff74041b75747d16cc66798`. Rule list: `[]` (Enabled: `True`). {{(pid=80377) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 02 10:18:25.611390 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.fernet_utils [None req-00de2a01-1a46-4ee6-83fc-c2f6e73392be None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80377) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 10:18:25.677348 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: [pid: 80377|app: 0|req: 1195/2375] 10.222.0.82 () {58 vars in 958 bytes} [Tue Aug 2 10:18:25 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 91 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 02 10:18:25.686911 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5abafb52-b3cc-49d8-a307-3ffaddf0159f None None] REQUEST_METHOD: `POST` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:18:25.687122 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5abafb52-b3cc-49d8-a307-3ffaddf0159f None None] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:18:25.687550 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5abafb52-b3cc-49d8-a307-3ffaddf0159f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:18:25.707496 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.auth.core [None req-5abafb52-b3cc-49d8-a307-3ffaddf0159f None None] MFA Rules not processed for user `891005fa5ff74041b75747d16cc66798`. Rule list: `[]` (Enabled: `True`). {{(pid=80378) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 02 10:18:25.708713 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.fernet_utils [None req-5abafb52-b3cc-49d8-a307-3ffaddf0159f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80378) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 10:18:25.716714 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 1181/2376] 10.222.0.82 () {58 vars in 958 bytes} [Tue Aug 2 10:18:25 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 02 10:18:25.721082 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00de2a01-1a46-4ee6-83fc-c2f6e73392be None None] Authenticating user token {{(pid=80377) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:18:25.730822 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00de2a01-1a46-4ee6-83fc-c2f6e73392be None None] Validating token access rules against request {{(pid=80377) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:18:25.736022 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-616bf6ef-8a75-4daa-b530-cf0b3f4bb750 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '891005fa5ff74041b75747d16cc66798', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7047912c5acd4adeba5d87c3bf895e39', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80377) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:18:25.736639 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-616bf6ef-8a75-4daa-b530-cf0b3f4bb750 admin admin] REQUEST_METHOD: `GET` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:18:25.736925 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-616bf6ef-8a75-4daa-b530-cf0b3f4bb750 admin admin] SCRIPT_NAME: `/identity` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:18:25.737204 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-616bf6ef-8a75-4daa-b530-cf0b3f4bb750 admin admin] PATH_INFO: `/v3/domains` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:18:25.737948 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-616bf6ef-8a75-4daa-b530-cf0b3f4bb750 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80377) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 02 10:18:25.738264 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-616bf6ef-8a75-4daa-b530-cf0b3f4bb750 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:18:25.739859 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-616bf6ef-8a75-4daa-b530-cf0b3f4bb750 admin admin] RBAC: Authorization granted {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:18:25.752921 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: [pid: 80377|app: 0|req: 1196/2377] 10.222.0.82 () {58 vars in 1153 bytes} [Tue Aug 2 10:18:25 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 02 10:18:25.757509 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5abafb52-b3cc-49d8-a307-3ffaddf0159f None None] Authenticating user token {{(pid=80378) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:18:25.767198 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5abafb52-b3cc-49d8-a307-3ffaddf0159f None None] Validating token access rules against request {{(pid=80378) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:18:25.772177 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c8861cd-0d51-4f38-b317-8e1203e865d2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '891005fa5ff74041b75747d16cc66798', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7047912c5acd4adeba5d87c3bf895e39', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80378) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:18:25.772790 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c8861cd-0d51-4f38-b317-8e1203e865d2 admin admin] REQUEST_METHOD: `POST` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:18:25.773071 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c8861cd-0d51-4f38-b317-8e1203e865d2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:18:25.773345 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c8861cd-0d51-4f38-b317-8e1203e865d2 admin admin] PATH_INFO: `/v3/projects` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:18:25.774058 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c8861cd-0d51-4f38-b317-8e1203e865d2 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:18:25.775959 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c8861cd-0d51-4f38-b317-8e1203e865d2 admin admin] RBAC: Authorization granted {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:18:25.805150 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 1182/2378] 10.222.0.82 () {60 vars in 1153 bytes} [Tue Aug 2 10:18:25 2022] POST /identity/v3/projects => generated 379 bytes in 48 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 02 10:18:25.809644 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-616bf6ef-8a75-4daa-b530-cf0b3f4bb750 admin admin] Authenticating user token {{(pid=80377) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:18:25.823649 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-616bf6ef-8a75-4daa-b530-cf0b3f4bb750 admin admin] Validating token access rules against request {{(pid=80377) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:18:25.907897 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-091fe1f7-5410-4e34-85c8-00da2a8f2359 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '891005fa5ff74041b75747d16cc66798', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7047912c5acd4adeba5d87c3bf895e39', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80377) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:18:25.908645 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-091fe1f7-5410-4e34-85c8-00da2a8f2359 admin admin] REQUEST_METHOD: `POST` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:18:25.908930 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-091fe1f7-5410-4e34-85c8-00da2a8f2359 admin admin] SCRIPT_NAME: `/identity` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:18:25.909204 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-091fe1f7-5410-4e34-85c8-00da2a8f2359 admin admin] PATH_INFO: `/v3/users` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:18:25.909902 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-091fe1f7-5410-4e34-85c8-00da2a8f2359 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:18:25.911854 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-091fe1f7-5410-4e34-85c8-00da2a8f2359 admin admin] RBAC: Authorization granted {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:18:25.938329 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: [pid: 80377|app: 0|req: 1197/2379] 10.222.0.82 () {60 vars in 1144 bytes} [Tue Aug 2 10:18:25 2022] POST /identity/v3/users => generated 290 bytes in 126 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 02 10:18:25.939593 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c8861cd-0d51-4f38-b317-8e1203e865d2 admin admin] Authenticating user token {{(pid=80378) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:18:25.950798 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c8861cd-0d51-4f38-b317-8e1203e865d2 admin admin] Validating token access rules against request {{(pid=80378) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:18:25.955732 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa4810b1-a872-4022-84ce-8b506bc3c3e4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '891005fa5ff74041b75747d16cc66798', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7047912c5acd4adeba5d87c3bf895e39', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80378) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:18:25.956357 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa4810b1-a872-4022-84ce-8b506bc3c3e4 admin admin] REQUEST_METHOD: `GET` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:18:25.956644 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa4810b1-a872-4022-84ce-8b506bc3c3e4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:18:25.956921 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa4810b1-a872-4022-84ce-8b506bc3c3e4 admin admin] PATH_INFO: `/v3/roles` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:18:25.957630 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa4810b1-a872-4022-84ce-8b506bc3c3e4 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:18:25.959305 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa4810b1-a872-4022-84ce-8b506bc3c3e4 admin admin] RBAC: Authorization granted {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:18:25.973061 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 1183/2380] 10.222.0.82 () {58 vars in 1122 bytes} [Tue Aug 2 10:18:25 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 02 10:18:25.974294 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-091fe1f7-5410-4e34-85c8-00da2a8f2359 admin admin] Authenticating user token {{(pid=80377) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:18:25.985014 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-091fe1f7-5410-4e34-85c8-00da2a8f2359 admin admin] Validating token access rules against request {{(pid=80377) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:18:25.989800 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce83381b-d26a-4b09-a7b3-d43057347e23 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '891005fa5ff74041b75747d16cc66798', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7047912c5acd4adeba5d87c3bf895e39', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80377) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:18:25.990434 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce83381b-d26a-4b09-a7b3-d43057347e23 admin admin] REQUEST_METHOD: `GET` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:18:25.990721 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce83381b-d26a-4b09-a7b3-d43057347e23 admin admin] SCRIPT_NAME: `/identity` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:18:25.991000 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce83381b-d26a-4b09-a7b3-d43057347e23 admin admin] PATH_INFO: `/v3/roles` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:18:25.991704 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce83381b-d26a-4b09-a7b3-d43057347e23 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:18:25.993305 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce83381b-d26a-4b09-a7b3-d43057347e23 admin admin] RBAC: Authorization granted {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:18:26.005056 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: [pid: 80377|app: 0|req: 1198/2381] 10.222.0.82 () {58 vars in 1122 bytes} [Tue Aug 2 10:18:25 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 02 10:18:26.009206 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa4810b1-a872-4022-84ce-8b506bc3c3e4 admin admin] Authenticating user token {{(pid=80378) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:18:26.021985 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa4810b1-a872-4022-84ce-8b506bc3c3e4 admin admin] Validating token access rules against request {{(pid=80378) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:18:26.026877 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b86bf703-f7ae-4380-9261-bed5bec896b0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '891005fa5ff74041b75747d16cc66798', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7047912c5acd4adeba5d87c3bf895e39', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80378) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:18:26.027650 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b86bf703-f7ae-4380-9261-bed5bec896b0 admin admin] REQUEST_METHOD: `PUT` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:18:26.027810 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b86bf703-f7ae-4380-9261-bed5bec896b0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:18:26.028081 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b86bf703-f7ae-4380-9261-bed5bec896b0 admin admin] PATH_INFO: `/v3/projects/49b834a4a22b44729fdf5d7b2589e17e/users/e139fa460d7e426490fb8d69286a3b9c/roles/e1c04b19fc4543b8b54f58bbcffbc598` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:18:26.043546 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b86bf703-f7ae-4380-9261-bed5bec896b0 admin admin] RBAC: Authorizing `identity:create_grant(project_id=49b834a4a22b44729fdf5d7b2589e17e, user_id=e139fa460d7e426490fb8d69286a3b9c, role_id=e1c04b19fc4543b8b54f58bbcffbc598)` {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:18:26.046777 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b86bf703-f7ae-4380-9261-bed5bec896b0 admin admin] RBAC: Authorization granted {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:18:26.059924 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 1184/2382] 10.222.0.82 () {60 vars in 1483 bytes} [Tue Aug 2 10:18:26 2022] PUT /identity/v3/projects/49b834a4a22b44729fdf5d7b2589e17e/users/e139fa460d7e426490fb8d69286a3b9c/roles/e1c04b19fc4543b8b54f58bbcffbc598 => generated 0 bytes in 51 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 02 10:18:26.071244 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-536c6103-dfa2-4c63-bcfb-865b07702bc5 None None] REQUEST_METHOD: `POST` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:18:26.071532 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-536c6103-dfa2-4c63-bcfb-865b07702bc5 None None] SCRIPT_NAME: `/identity` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:18:26.071811 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-536c6103-dfa2-4c63-bcfb-865b07702bc5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:18:26.097793 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG dbcounter [-] [80377] Writing DB stats keystone:SELECT=91,keystone:DELETE=5,keystone:INSERT=4 {{(pid=80377) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 10:18:26.100902 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.auth.core [None req-536c6103-dfa2-4c63-bcfb-865b07702bc5 None None] MFA Rules not processed for user `e139fa460d7e426490fb8d69286a3b9c`. Rule list: `[]` (Enabled: `True`). {{(pid=80377) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 02 10:18:26.102852 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.fernet_utils [None req-536c6103-dfa2-4c63-bcfb-865b07702bc5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80377) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 10:18:26.171893 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: [pid: 80377|app: 0|req: 1199/2383] 10.222.0.82 () {58 vars in 958 bytes} [Tue Aug 2 10:18:26 2022] POST /identity/v3/auth/tokens => generated 3256 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 02 10:18:26.176550 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b86bf703-f7ae-4380-9261-bed5bec896b0 admin admin] Authenticating user token {{(pid=80378) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:18:26.240861 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b86bf703-f7ae-4380-9261-bed5bec896b0 admin admin] Validating token access rules against request {{(pid=80378) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:18:26.245446 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-942baf0c-d604-4041-a054-2ed046ad6b58 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '891005fa5ff74041b75747d16cc66798', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7047912c5acd4adeba5d87c3bf895e39', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80378) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:18:26.246107 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-942baf0c-d604-4041-a054-2ed046ad6b58 admin admin] REQUEST_METHOD: `POST` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:18:26.246390 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-942baf0c-d604-4041-a054-2ed046ad6b58 admin admin] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:18:26.246663 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-942baf0c-d604-4041-a054-2ed046ad6b58 admin admin] PATH_INFO: `/v3/projects` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:18:26.247322 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-942baf0c-d604-4041-a054-2ed046ad6b58 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:18:26.249269 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-942baf0c-d604-4041-a054-2ed046ad6b58 admin admin] RBAC: Authorization granted {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:18:26.275018 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 1185/2384] 10.222.0.82 () {60 vars in 1153 bytes} [Tue Aug 2 10:18:26 2022] POST /identity/v3/projects => generated 377 bytes in 99 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 02 10:18:26.279089 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-536c6103-dfa2-4c63-bcfb-865b07702bc5 None None] Authenticating user token {{(pid=80377) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:18:26.290232 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-536c6103-dfa2-4c63-bcfb-865b07702bc5 None None] Validating token access rules against request {{(pid=80377) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:18:26.346182 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90767c60-8ced-421e-8ff7-d390f22de99c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '891005fa5ff74041b75747d16cc66798', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7047912c5acd4adeba5d87c3bf895e39', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80377) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:18:26.346792 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90767c60-8ced-421e-8ff7-d390f22de99c admin admin] REQUEST_METHOD: `POST` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:18:26.347077 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90767c60-8ced-421e-8ff7-d390f22de99c admin admin] SCRIPT_NAME: `/identity` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:18:26.347350 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90767c60-8ced-421e-8ff7-d390f22de99c admin admin] PATH_INFO: `/v3/users` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:18:26.348001 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90767c60-8ced-421e-8ff7-d390f22de99c admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:18:26.349926 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90767c60-8ced-421e-8ff7-d390f22de99c admin admin] RBAC: Authorization granted {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:18:26.370756 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: [pid: 80377|app: 0|req: 1200/2385] 10.222.0.82 () {60 vars in 1144 bytes} [Tue Aug 2 10:18:26 2022] POST /identity/v3/users => generated 295 bytes in 92 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 02 10:18:26.374779 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-942baf0c-d604-4041-a054-2ed046ad6b58 admin admin] Authenticating user token {{(pid=80378) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:18:26.385076 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-942baf0c-d604-4041-a054-2ed046ad6b58 admin admin] Validating token access rules against request {{(pid=80378) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:18:26.390288 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e8ac53a-60b2-43e7-9814-adc21541f94d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '891005fa5ff74041b75747d16cc66798', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7047912c5acd4adeba5d87c3bf895e39', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80378) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:18:26.390898 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e8ac53a-60b2-43e7-9814-adc21541f94d admin admin] REQUEST_METHOD: `GET` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:18:26.391184 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e8ac53a-60b2-43e7-9814-adc21541f94d admin admin] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:18:26.391458 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e8ac53a-60b2-43e7-9814-adc21541f94d admin admin] PATH_INFO: `/v3/roles` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:18:26.392160 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e8ac53a-60b2-43e7-9814-adc21541f94d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:18:26.393762 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e8ac53a-60b2-43e7-9814-adc21541f94d admin admin] RBAC: Authorization granted {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:18:26.404601 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 1186/2386] 10.222.0.82 () {58 vars in 1122 bytes} [Tue Aug 2 10:18:26 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 02 10:18:26.410055 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90767c60-8ced-421e-8ff7-d390f22de99c admin admin] Authenticating user token {{(pid=80377) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:18:26.421263 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90767c60-8ced-421e-8ff7-d390f22de99c admin admin] Validating token access rules against request {{(pid=80377) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:18:26.426114 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61af9e87-f74b-4554-8729-ab8d1f8f6262 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '891005fa5ff74041b75747d16cc66798', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7047912c5acd4adeba5d87c3bf895e39', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80377) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:18:26.426919 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61af9e87-f74b-4554-8729-ab8d1f8f6262 admin admin] REQUEST_METHOD: `PUT` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:18:26.427202 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61af9e87-f74b-4554-8729-ab8d1f8f6262 admin admin] SCRIPT_NAME: `/identity` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:18:26.427481 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61af9e87-f74b-4554-8729-ab8d1f8f6262 admin admin] PATH_INFO: `/v3/projects/11b0f59fb265423dbc9bff6c38f90629/users/893211d73f834e3ca6960d15d8006eb7/roles/237966442f05411191105bf336bc14f3` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:18:26.442483 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61af9e87-f74b-4554-8729-ab8d1f8f6262 admin admin] RBAC: Authorizing `identity:create_grant(project_id=11b0f59fb265423dbc9bff6c38f90629, user_id=893211d73f834e3ca6960d15d8006eb7, role_id=237966442f05411191105bf336bc14f3)` {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:18:26.445147 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61af9e87-f74b-4554-8729-ab8d1f8f6262 admin admin] RBAC: Authorization granted {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:18:26.459896 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: [pid: 80377|app: 0|req: 1201/2387] 10.222.0.82 () {60 vars in 1483 bytes} [Tue Aug 2 10:18:26 2022] PUT /identity/v3/projects/11b0f59fb265423dbc9bff6c38f90629/users/893211d73f834e3ca6960d15d8006eb7/roles/237966442f05411191105bf336bc14f3 => generated 0 bytes in 51 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 02 10:18:26.463626 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e8ac53a-60b2-43e7-9814-adc21541f94d admin admin] Authenticating user token {{(pid=80378) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:18:26.474259 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e8ac53a-60b2-43e7-9814-adc21541f94d admin admin] Validating token access rules against request {{(pid=80378) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:18:26.523016 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG dbcounter [-] [80378] Writing DB stats keystone:SELECT=94,keystone:DELETE=2,keystone:INSERT=4 {{(pid=80378) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 10:18:26.531962 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4edd7ff8-761e-4d85-af97-9892bc691a99 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '891005fa5ff74041b75747d16cc66798', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7047912c5acd4adeba5d87c3bf895e39', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80378) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:18:26.532830 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4edd7ff8-761e-4d85-af97-9892bc691a99 admin admin] REQUEST_METHOD: `GET` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:18:26.533162 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4edd7ff8-761e-4d85-af97-9892bc691a99 admin admin] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:18:26.533606 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4edd7ff8-761e-4d85-af97-9892bc691a99 admin admin] PATH_INFO: `/v3/roles` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:18:26.534481 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4edd7ff8-761e-4d85-af97-9892bc691a99 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:18:26.536301 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4edd7ff8-761e-4d85-af97-9892bc691a99 admin admin] RBAC: Authorization granted {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:18:26.546155 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 1187/2388] 10.222.0.82 () {58 vars in 1122 bytes} [Tue Aug 2 10:18:26 2022] GET /identity/v3/roles => generated 1420 bytes in 83 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 02 10:18:26.550473 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61af9e87-f74b-4554-8729-ab8d1f8f6262 admin admin] Authenticating user token {{(pid=80377) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:18:26.561336 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61af9e87-f74b-4554-8729-ab8d1f8f6262 admin admin] Validating token access rules against request {{(pid=80377) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:18:26.565904 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2103c07-ba41-4eb5-8f02-d186d8f4cae3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '891005fa5ff74041b75747d16cc66798', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7047912c5acd4adeba5d87c3bf895e39', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80377) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:18:26.566567 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2103c07-ba41-4eb5-8f02-d186d8f4cae3 admin admin] REQUEST_METHOD: `PUT` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:18:26.566875 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2103c07-ba41-4eb5-8f02-d186d8f4cae3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:18:26.567153 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2103c07-ba41-4eb5-8f02-d186d8f4cae3 admin admin] PATH_INFO: `/v3/projects/11b0f59fb265423dbc9bff6c38f90629/users/893211d73f834e3ca6960d15d8006eb7/roles/237966442f05411191105bf336bc14f3` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:18:26.569398 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2103c07-ba41-4eb5-8f02-d186d8f4cae3 admin admin] RBAC: Authorizing `identity:create_grant(project_id=11b0f59fb265423dbc9bff6c38f90629, user_id=893211d73f834e3ca6960d15d8006eb7, role_id=237966442f05411191105bf336bc14f3)` {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:18:26.571818 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2103c07-ba41-4eb5-8f02-d186d8f4cae3 admin admin] RBAC: Authorization granted {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:18:26.582847 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: [pid: 80377|app: 0|req: 1202/2389] 10.222.0.82 () {60 vars in 1483 bytes} [Tue Aug 2 10:18:26 2022] PUT /identity/v3/projects/11b0f59fb265423dbc9bff6c38f90629/users/893211d73f834e3ca6960d15d8006eb7/roles/237966442f05411191105bf336bc14f3 => generated 0 bytes in 33 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 02 10:18:26.594719 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63a4e3b3-117f-473b-b09c-f729319f123a None None] REQUEST_METHOD: `POST` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:18:26.595010 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63a4e3b3-117f-473b-b09c-f729319f123a None None] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:18:26.595286 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63a4e3b3-117f-473b-b09c-f729319f123a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:18:26.624343 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.auth.core [None req-63a4e3b3-117f-473b-b09c-f729319f123a None None] MFA Rules not processed for user `893211d73f834e3ca6960d15d8006eb7`. Rule list: `[]` (Enabled: `True`). {{(pid=80378) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 02 10:18:26.625591 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.fernet_utils [None req-63a4e3b3-117f-473b-b09c-f729319f123a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80378) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 10:18:26.695647 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 1188/2390] 10.222.0.82 () {58 vars in 958 bytes} [Tue Aug 2 10:18:26 2022] POST /identity/v3/auth/tokens => generated 3321 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 02 10:18:26.706899 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2103c07-ba41-4eb5-8f02-d186d8f4cae3 admin admin] Authenticating user token {{(pid=80377) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:18:26.708624 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.fernet_utils [None req-a2103c07-ba41-4eb5-8f02-d186d8f4cae3 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80377) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 10:18:26.765319 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2103c07-ba41-4eb5-8f02-d186d8f4cae3 admin admin] Validating token access rules against request {{(pid=80377) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:18:26.769325 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a1fef41-b0fd-49c4-b426-448aa683b2b6 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1023de281ae74e9d8a6d6c312c1c97a7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f1b38ebb804c41c69772c998aae72c41', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80377) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:18:26.769975 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a1fef41-b0fd-49c4-b426-448aa683b2b6 service cinder] REQUEST_METHOD: `GET` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:18:26.770263 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a1fef41-b0fd-49c4-b426-448aa683b2b6 service cinder] SCRIPT_NAME: `/identity` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:18:26.770541 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a1fef41-b0fd-49c4-b426-448aa683b2b6 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:18:26.779850 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a1fef41-b0fd-49c4-b426-448aa683b2b6 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:18:26.781292 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a1fef41-b0fd-49c4-b426-448aa683b2b6 service cinder] RBAC: Authorization granted {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:18:26.785201 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG dbcounter [-] [80377] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80377) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 10:18:26.791519 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: [pid: 80377|app: 0|req: 1203/2391] 10.222.0.82 () {60 vars in 1365 bytes} [Tue Aug 2 10:18:26 2022] GET /identity/v3/auth/tokens => generated 3321 bytes in 85 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 02 10:18:27.178608 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63a4e3b3-117f-473b-b09c-f729319f123a None None] Authenticating user token {{(pid=80378) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:18:27.241724 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63a4e3b3-117f-473b-b09c-f729319f123a None None] Validating token access rules against request {{(pid=80378) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:18:27.246403 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e9d1866-9fdc-428e-81be-c1f98d679111 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '891005fa5ff74041b75747d16cc66798', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7047912c5acd4adeba5d87c3bf895e39', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80378) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:18:27.247049 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e9d1866-9fdc-428e-81be-c1f98d679111 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:18:27.247340 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e9d1866-9fdc-428e-81be-c1f98d679111 admin admin] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:18:27.247620 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e9d1866-9fdc-428e-81be-c1f98d679111 admin admin] PATH_INFO: `/v3/users/e139fa460d7e426490fb8d69286a3b9c` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:18:27.249390 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e9d1866-9fdc-428e-81be-c1f98d679111 admin admin] RBAC: Authorizing `identity:delete_user(user_id=e139fa460d7e426490fb8d69286a3b9c)` {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:18:27.251341 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e9d1866-9fdc-428e-81be-c1f98d679111 admin admin] RBAC: Authorization granted {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:18:27.299510 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.notifications [None req-3e9d1866-9fdc-428e-81be-c1f98d679111 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'e139fa460d7e426490fb8d69286a3b9c'} {{(pid=80378) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 10:18:27.308027 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.notifications [None req-3e9d1866-9fdc-428e-81be-c1f98d679111 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'e139fa460d7e426490fb8d69286a3b9c'} {{(pid=80378) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 10:18:27.316629 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.notifications [None req-3e9d1866-9fdc-428e-81be-c1f98d679111 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'e139fa460d7e426490fb8d69286a3b9c'} {{(pid=80378) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 10:18:27.317464 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.notifications [None req-3e9d1866-9fdc-428e-81be-c1f98d679111 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'e139fa460d7e426490fb8d69286a3b9c'} {{(pid=80378) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 10:18:27.328781 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG dbcounter [-] [80378] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80378) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 10:18:27.330086 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.notifications [None req-3e9d1866-9fdc-428e-81be-c1f98d679111 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'e139fa460d7e426490fb8d69286a3b9c'} {{(pid=80378) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 10:18:27.336936 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 1189/2392] 10.222.0.82 () {58 vars in 1224 bytes} [Tue Aug 2 10:18:27 2022] DELETE /identity/v3/users/e139fa460d7e426490fb8d69286a3b9c => generated 0 bytes in 159 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 02 10:18:27.341391 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c8bb9d2-b262-46af-8be6-76eab0a97075 None None] REQUEST_METHOD: `POST` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:18:27.341910 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c8bb9d2-b262-46af-8be6-76eab0a97075 None None] SCRIPT_NAME: `/identity` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:18:27.342058 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c8bb9d2-b262-46af-8be6-76eab0a97075 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:18:27.360798 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.auth.core [None req-1c8bb9d2-b262-46af-8be6-76eab0a97075 None None] MFA Rules not processed for user `891005fa5ff74041b75747d16cc66798`. Rule list: `[]` (Enabled: `True`). {{(pid=80377) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 02 10:18:27.362028 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.fernet_utils [None req-1c8bb9d2-b262-46af-8be6-76eab0a97075 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80377) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 10:18:27.418892 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: [pid: 80377|app: 0|req: 1204/2393] 10.222.0.82 () {58 vars in 958 bytes} [Tue Aug 2 10:18:27 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 78 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 02 10:18:27.429297 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e9d1866-9fdc-428e-81be-c1f98d679111 admin admin] Authenticating user token {{(pid=80378) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:18:27.430722 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.fernet_utils [None req-3e9d1866-9fdc-428e-81be-c1f98d679111 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80378) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 10:18:27.488418 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e9d1866-9fdc-428e-81be-c1f98d679111 admin admin] Validating token access rules against request {{(pid=80378) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:18:27.492082 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-986f775c-ac18-438d-b4ba-f377c5a8ebc2 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '33b87170f0634b67bce60750847b5cbe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f1b38ebb804c41c69772c998aae72c41', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80378) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:18:27.492705 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-986f775c-ac18-438d-b4ba-f377c5a8ebc2 service neutron] REQUEST_METHOD: `GET` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:18:27.492995 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-986f775c-ac18-438d-b4ba-f377c5a8ebc2 service neutron] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:18:27.493271 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-986f775c-ac18-438d-b4ba-f377c5a8ebc2 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:18:27.502485 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-986f775c-ac18-438d-b4ba-f377c5a8ebc2 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:18:27.504003 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-986f775c-ac18-438d-b4ba-f377c5a8ebc2 service neutron] RBAC: Authorization granted {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:18:27.513394 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 1190/2394] 10.222.0.82 () {60 vars in 1365 bytes} [Tue Aug 2 10:18:27 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 85 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 02 10:18:27.786245 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c8bb9d2-b262-46af-8be6-76eab0a97075 None None] Authenticating user token {{(pid=80377) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:18:27.787650 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.fernet_utils [None req-1c8bb9d2-b262-46af-8be6-76eab0a97075 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80377) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 10:18:27.800513 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c8bb9d2-b262-46af-8be6-76eab0a97075 None None] Validating token access rules against request {{(pid=80377) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:18:27.805650 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb58a0d5-3140-47d6-92f7-b8c7456fc4cc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '891005fa5ff74041b75747d16cc66798', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7047912c5acd4adeba5d87c3bf895e39', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80377) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:18:27.806670 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb58a0d5-3140-47d6-92f7-b8c7456fc4cc admin admin] REQUEST_METHOD: `DELETE` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:18:27.807009 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb58a0d5-3140-47d6-92f7-b8c7456fc4cc admin admin] SCRIPT_NAME: `/identity` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:18:27.808106 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb58a0d5-3140-47d6-92f7-b8c7456fc4cc admin admin] PATH_INFO: `/v3/projects/49b834a4a22b44729fdf5d7b2589e17e` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:18:27.808934 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb58a0d5-3140-47d6-92f7-b8c7456fc4cc admin admin] RBAC: Authorizing `identity:delete_project(project_id=49b834a4a22b44729fdf5d7b2589e17e)` {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:18:27.811025 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb58a0d5-3140-47d6-92f7-b8c7456fc4cc admin admin] RBAC: Authorization granted {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:18:27.860213 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.notifications [None req-cb58a0d5-3140-47d6-92f7-b8c7456fc4cc admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '49b834a4a22b44729fdf5d7b2589e17e', 'request_id': 'req-cb58a0d5-3140-47d6-92f7-b8c7456fc4cc'} {{(pid=80377) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 10:18:27.865910 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.notifications [None req-cb58a0d5-3140-47d6-92f7-b8c7456fc4cc admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '49b834a4a22b44729fdf5d7b2589e17e', 'request_id': 'req-cb58a0d5-3140-47d6-92f7-b8c7456fc4cc'} {{(pid=80377) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 10:18:27.872157 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.notifications [None req-cb58a0d5-3140-47d6-92f7-b8c7456fc4cc admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '49b834a4a22b44729fdf5d7b2589e17e', 'request_id': 'req-cb58a0d5-3140-47d6-92f7-b8c7456fc4cc'} {{(pid=80377) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 10:18:27.885217 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.notifications [None req-cb58a0d5-3140-47d6-92f7-b8c7456fc4cc admin admin] The token cache is being invalidate because project 49b834a4a22b44729fdf5d7b2589e17e was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80377) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 02 10:18:27.885509 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.notifications [None req-cb58a0d5-3140-47d6-92f7-b8c7456fc4cc admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80377) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 10:18:27.886174 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.notifications [None req-cb58a0d5-3140-47d6-92f7-b8c7456fc4cc admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80377) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 10:18:27.887731 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: [pid: 80377|app: 0|req: 1205/2395] 10.222.0.82 () {58 vars in 1233 bytes} [Tue Aug 2 10:18:27 2022] DELETE /identity/v3/projects/49b834a4a22b44729fdf5d7b2589e17e => generated 0 bytes in 102 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 02 10:18:27.892135 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-986f775c-ac18-438d-b4ba-f377c5a8ebc2 service neutron] Authenticating user token {{(pid=80378) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:18:27.901646 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-986f775c-ac18-438d-b4ba-f377c5a8ebc2 service neutron] Validating token access rules against request {{(pid=80378) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:18:27.959816 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b1a1524-8234-426f-be66-d93ce5c910f1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '891005fa5ff74041b75747d16cc66798', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7047912c5acd4adeba5d87c3bf895e39', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80378) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:18:27.960459 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b1a1524-8234-426f-be66-d93ce5c910f1 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:18:27.960741 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b1a1524-8234-426f-be66-d93ce5c910f1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:18:27.961044 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b1a1524-8234-426f-be66-d93ce5c910f1 admin admin] PATH_INFO: `/v3/users/893211d73f834e3ca6960d15d8006eb7` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:18:27.962248 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b1a1524-8234-426f-be66-d93ce5c910f1 admin admin] RBAC: Authorizing `identity:delete_user(user_id=893211d73f834e3ca6960d15d8006eb7)` {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:18:27.964184 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b1a1524-8234-426f-be66-d93ce5c910f1 admin admin] RBAC: Authorization granted {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:18:28.008768 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.notifications [None req-3b1a1524-8234-426f-be66-d93ce5c910f1 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '893211d73f834e3ca6960d15d8006eb7'} {{(pid=80378) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 10:18:28.016683 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.notifications [None req-3b1a1524-8234-426f-be66-d93ce5c910f1 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '893211d73f834e3ca6960d15d8006eb7'} {{(pid=80378) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 10:18:28.023675 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.notifications [None req-3b1a1524-8234-426f-be66-d93ce5c910f1 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '893211d73f834e3ca6960d15d8006eb7'} {{(pid=80378) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 10:18:28.024297 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.notifications [None req-3b1a1524-8234-426f-be66-d93ce5c910f1 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '893211d73f834e3ca6960d15d8006eb7'} {{(pid=80378) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 10:18:28.037745 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.notifications [None req-3b1a1524-8234-426f-be66-d93ce5c910f1 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '893211d73f834e3ca6960d15d8006eb7'} {{(pid=80378) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 10:18:28.043984 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 1191/2396] 10.222.0.82 () {58 vars in 1224 bytes} [Tue Aug 2 10:18:27 2022] DELETE /identity/v3/users/893211d73f834e3ca6960d15d8006eb7 => generated 0 bytes in 152 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 02 10:18:28.369594 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb58a0d5-3140-47d6-92f7-b8c7456fc4cc admin admin] Authenticating user token {{(pid=80377) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:18:28.371084 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.fernet_utils [None req-cb58a0d5-3140-47d6-92f7-b8c7456fc4cc admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80377) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 10:18:28.398754 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb58a0d5-3140-47d6-92f7-b8c7456fc4cc admin admin] Validating token access rules against request {{(pid=80377) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:18:28.471165 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5bde5b6-43e0-4b61-820e-c9778817f3e5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '891005fa5ff74041b75747d16cc66798', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7047912c5acd4adeba5d87c3bf895e39', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80377) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:18:28.471821 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5bde5b6-43e0-4b61-820e-c9778817f3e5 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:18:28.472106 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5bde5b6-43e0-4b61-820e-c9778817f3e5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:18:28.472383 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5bde5b6-43e0-4b61-820e-c9778817f3e5 admin admin] PATH_INFO: `/v3/projects/11b0f59fb265423dbc9bff6c38f90629` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:18:28.473570 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5bde5b6-43e0-4b61-820e-c9778817f3e5 admin admin] RBAC: Authorizing `identity:delete_project(project_id=11b0f59fb265423dbc9bff6c38f90629)` {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:18:28.475510 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5bde5b6-43e0-4b61-820e-c9778817f3e5 admin admin] RBAC: Authorization granted {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:18:28.499937 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG dbcounter [-] [80377] Writing DB stats keystone:SELECT=91,keystone:DELETE=8,keystone:INSERT=1 {{(pid=80377) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 10:18:28.529949 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.notifications [None req-d5bde5b6-43e0-4b61-820e-c9778817f3e5 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '11b0f59fb265423dbc9bff6c38f90629', 'request_id': 'req-d5bde5b6-43e0-4b61-820e-c9778817f3e5'} {{(pid=80377) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 10:18:28.535999 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.notifications [None req-d5bde5b6-43e0-4b61-820e-c9778817f3e5 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '11b0f59fb265423dbc9bff6c38f90629', 'request_id': 'req-d5bde5b6-43e0-4b61-820e-c9778817f3e5'} {{(pid=80377) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 10:18:28.542300 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.notifications [None req-d5bde5b6-43e0-4b61-820e-c9778817f3e5 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '11b0f59fb265423dbc9bff6c38f90629', 'request_id': 'req-d5bde5b6-43e0-4b61-820e-c9778817f3e5'} {{(pid=80377) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 10:18:28.554777 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.notifications [None req-d5bde5b6-43e0-4b61-820e-c9778817f3e5 admin admin] The token cache is being invalidate because project 11b0f59fb265423dbc9bff6c38f90629 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80377) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 02 10:18:28.555069 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.notifications [None req-d5bde5b6-43e0-4b61-820e-c9778817f3e5 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80377) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 10:18:28.555704 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.notifications [None req-d5bde5b6-43e0-4b61-820e-c9778817f3e5 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80377) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 10:18:28.557469 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: [pid: 80377|app: 0|req: 1206/2397] 10.222.0.82 () {58 vars in 1233 bytes} [Tue Aug 2 10:18:28 2022] DELETE /identity/v3/projects/11b0f59fb265423dbc9bff6c38f90629 => generated 0 bytes in 190 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 02 10:18:28.563761 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5fe4b37-98d6-4bea-934d-afadf9efa998 None None] REQUEST_METHOD: `POST` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:18:28.564217 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5fe4b37-98d6-4bea-934d-afadf9efa998 None None] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:18:28.564323 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5fe4b37-98d6-4bea-934d-afadf9efa998 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:18:28.584377 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.auth.core [None req-b5fe4b37-98d6-4bea-934d-afadf9efa998 None None] MFA Rules not processed for user `891005fa5ff74041b75747d16cc66798`. Rule list: `[]` (Enabled: `True`). {{(pid=80378) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 02 10:18:28.585569 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.fernet_utils [None req-b5fe4b37-98d6-4bea-934d-afadf9efa998 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80378) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 10:18:28.597190 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG dbcounter [-] [80378] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80378) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 10:18:28.646060 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 1192/2398] 10.222.0.82 () {58 vars in 958 bytes} [Tue Aug 2 10:18:28 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 83 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 02 10:18:28.654206 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-398584a4-c6ea-4e06-bc2e-83a77dfd6142 None None] REQUEST_METHOD: `POST` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:18:28.654499 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-398584a4-c6ea-4e06-bc2e-83a77dfd6142 None None] SCRIPT_NAME: `/identity` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:18:28.654784 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-398584a4-c6ea-4e06-bc2e-83a77dfd6142 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:18:28.675223 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.auth.core [None req-398584a4-c6ea-4e06-bc2e-83a77dfd6142 None None] MFA Rules not processed for user `891005fa5ff74041b75747d16cc66798`. Rule list: `[]` (Enabled: `True`). {{(pid=80377) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 02 10:18:28.676443 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.fernet_utils [None req-398584a4-c6ea-4e06-bc2e-83a77dfd6142 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80377) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 10:18:28.684685 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: [pid: 80377|app: 0|req: 1207/2399] 10.222.0.82 () {58 vars in 958 bytes} [Tue Aug 2 10:18:28 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 02 10:18:28.689290 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5fe4b37-98d6-4bea-934d-afadf9efa998 None None] Authenticating user token {{(pid=80378) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:18:28.699135 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5fe4b37-98d6-4bea-934d-afadf9efa998 None None] Validating token access rules against request {{(pid=80378) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:18:28.703801 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ee0e713-5b52-4e9b-a337-de7442ca87e1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '891005fa5ff74041b75747d16cc66798', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7047912c5acd4adeba5d87c3bf895e39', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80378) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:18:28.704420 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ee0e713-5b52-4e9b-a337-de7442ca87e1 admin admin] REQUEST_METHOD: `GET` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:18:28.704705 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ee0e713-5b52-4e9b-a337-de7442ca87e1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:18:28.704983 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ee0e713-5b52-4e9b-a337-de7442ca87e1 admin admin] PATH_INFO: `/v3/domains` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:18:28.705686 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ee0e713-5b52-4e9b-a337-de7442ca87e1 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80378) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 02 10:18:28.706028 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ee0e713-5b52-4e9b-a337-de7442ca87e1 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:18:28.707706 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ee0e713-5b52-4e9b-a337-de7442ca87e1 admin admin] RBAC: Authorization granted {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:18:28.723694 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 1193/2400] 10.222.0.82 () {58 vars in 1153 bytes} [Tue Aug 2 10:18:28 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 02 10:18:28.725346 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-398584a4-c6ea-4e06-bc2e-83a77dfd6142 None None] Authenticating user token {{(pid=80377) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:18:28.735749 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-398584a4-c6ea-4e06-bc2e-83a77dfd6142 None None] Validating token access rules against request {{(pid=80377) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:18:28.740436 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-099ffc06-ca53-430a-8dc0-514ba5ff7ccd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '891005fa5ff74041b75747d16cc66798', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7047912c5acd4adeba5d87c3bf895e39', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80377) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:18:28.741055 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-099ffc06-ca53-430a-8dc0-514ba5ff7ccd admin admin] REQUEST_METHOD: `POST` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:18:28.741339 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-099ffc06-ca53-430a-8dc0-514ba5ff7ccd admin admin] SCRIPT_NAME: `/identity` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:18:28.741617 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-099ffc06-ca53-430a-8dc0-514ba5ff7ccd admin admin] PATH_INFO: `/v3/projects` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:18:28.742348 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-099ffc06-ca53-430a-8dc0-514ba5ff7ccd admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:18:28.744259 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-099ffc06-ca53-430a-8dc0-514ba5ff7ccd admin admin] RBAC: Authorization granted {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:18:28.774394 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: [pid: 80377|app: 0|req: 1208/2401] 10.222.0.82 () {60 vars in 1153 bytes} [Tue Aug 2 10:18:28 2022] POST /identity/v3/projects => generated 385 bytes in 50 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 02 10:18:28.778526 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ee0e713-5b52-4e9b-a337-de7442ca87e1 admin admin] Authenticating user token {{(pid=80378) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:18:28.788826 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ee0e713-5b52-4e9b-a337-de7442ca87e1 admin admin] Validating token access rules against request {{(pid=80378) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:18:28.845963 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d0ae10a-7754-47f1-af7a-a50e314dbda0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '891005fa5ff74041b75747d16cc66798', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7047912c5acd4adeba5d87c3bf895e39', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80378) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:18:28.846593 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d0ae10a-7754-47f1-af7a-a50e314dbda0 admin admin] REQUEST_METHOD: `POST` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:18:28.846883 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d0ae10a-7754-47f1-af7a-a50e314dbda0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:18:28.847159 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d0ae10a-7754-47f1-af7a-a50e314dbda0 admin admin] PATH_INFO: `/v3/users` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:18:28.847834 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d0ae10a-7754-47f1-af7a-a50e314dbda0 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:18:28.849812 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d0ae10a-7754-47f1-af7a-a50e314dbda0 admin admin] RBAC: Authorization granted {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:18:28.871637 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 1194/2402] 10.222.0.82 () {60 vars in 1144 bytes} [Tue Aug 2 10:18:28 2022] POST /identity/v3/users => generated 293 bytes in 94 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 02 10:18:28.876836 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-099ffc06-ca53-430a-8dc0-514ba5ff7ccd admin admin] Authenticating user token {{(pid=80377) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:18:28.887266 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-099ffc06-ca53-430a-8dc0-514ba5ff7ccd admin admin] Validating token access rules against request {{(pid=80377) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:18:28.892148 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df43e62f-ba5b-47f4-94cd-804d43d1a8d3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '891005fa5ff74041b75747d16cc66798', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7047912c5acd4adeba5d87c3bf895e39', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80377) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:18:28.892882 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df43e62f-ba5b-47f4-94cd-804d43d1a8d3 admin admin] REQUEST_METHOD: `GET` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:18:28.893171 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df43e62f-ba5b-47f4-94cd-804d43d1a8d3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:18:28.893450 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df43e62f-ba5b-47f4-94cd-804d43d1a8d3 admin admin] PATH_INFO: `/v3/roles` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:18:28.894195 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df43e62f-ba5b-47f4-94cd-804d43d1a8d3 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:18:28.895800 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df43e62f-ba5b-47f4-94cd-804d43d1a8d3 admin admin] RBAC: Authorization granted {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:18:28.905883 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: [pid: 80377|app: 0|req: 1209/2403] 10.222.0.82 () {58 vars in 1122 bytes} [Tue Aug 2 10:18:28 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 02 10:18:28.909867 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d0ae10a-7754-47f1-af7a-a50e314dbda0 admin admin] Authenticating user token {{(pid=80378) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:18:28.919908 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d0ae10a-7754-47f1-af7a-a50e314dbda0 admin admin] Validating token access rules against request {{(pid=80378) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:18:28.924982 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-224f191f-d5e3-454e-9153-cb62e2a2ca48 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '891005fa5ff74041b75747d16cc66798', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7047912c5acd4adeba5d87c3bf895e39', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80378) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:18:28.925587 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-224f191f-d5e3-454e-9153-cb62e2a2ca48 admin admin] REQUEST_METHOD: `GET` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:18:28.925904 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-224f191f-d5e3-454e-9153-cb62e2a2ca48 admin admin] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:18:28.926179 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-224f191f-d5e3-454e-9153-cb62e2a2ca48 admin admin] PATH_INFO: `/v3/roles` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:18:28.926903 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-224f191f-d5e3-454e-9153-cb62e2a2ca48 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:18:28.928498 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-224f191f-d5e3-454e-9153-cb62e2a2ca48 admin admin] RBAC: Authorization granted {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:18:28.939918 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 1195/2404] 10.222.0.82 () {58 vars in 1122 bytes} [Tue Aug 2 10:18:28 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 02 10:18:28.944177 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df43e62f-ba5b-47f4-94cd-804d43d1a8d3 admin admin] Authenticating user token {{(pid=80377) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:18:28.954429 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df43e62f-ba5b-47f4-94cd-804d43d1a8d3 admin admin] Validating token access rules against request {{(pid=80377) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:18:28.959102 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6234eb3f-d6b2-4872-b6ee-0a35b90b9191 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '891005fa5ff74041b75747d16cc66798', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7047912c5acd4adeba5d87c3bf895e39', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80377) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:18:28.959763 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6234eb3f-d6b2-4872-b6ee-0a35b90b9191 admin admin] REQUEST_METHOD: `PUT` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:18:28.960075 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6234eb3f-d6b2-4872-b6ee-0a35b90b9191 admin admin] SCRIPT_NAME: `/identity` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:18:28.960354 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6234eb3f-d6b2-4872-b6ee-0a35b90b9191 admin admin] PATH_INFO: `/v3/projects/3b0a4e80b761485ba4bf70c78ebaf599/users/d600f95cdb8f46cfb3d244cd3ca89886/roles/e1c04b19fc4543b8b54f58bbcffbc598` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:18:28.975446 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6234eb3f-d6b2-4872-b6ee-0a35b90b9191 admin admin] RBAC: Authorizing `identity:create_grant(project_id=3b0a4e80b761485ba4bf70c78ebaf599, user_id=d600f95cdb8f46cfb3d244cd3ca89886, role_id=e1c04b19fc4543b8b54f58bbcffbc598)` {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:18:28.978109 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6234eb3f-d6b2-4872-b6ee-0a35b90b9191 admin admin] RBAC: Authorization granted {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:18:28.991224 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: [pid: 80377|app: 0|req: 1210/2405] 10.222.0.82 () {60 vars in 1483 bytes} [Tue Aug 2 10:18:28 2022] PUT /identity/v3/projects/3b0a4e80b761485ba4bf70c78ebaf599/users/d600f95cdb8f46cfb3d244cd3ca89886/roles/e1c04b19fc4543b8b54f58bbcffbc598 => generated 0 bytes in 47 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 02 10:18:29.001381 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27f04895-9a69-49bd-b4e7-0662abe6185a None None] REQUEST_METHOD: `POST` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:18:29.001670 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27f04895-9a69-49bd-b4e7-0662abe6185a None None] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:18:29.001979 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27f04895-9a69-49bd-b4e7-0662abe6185a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:18:29.030379 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.auth.core [None req-27f04895-9a69-49bd-b4e7-0662abe6185a None None] MFA Rules not processed for user `d600f95cdb8f46cfb3d244cd3ca89886`. Rule list: `[]` (Enabled: `True`). {{(pid=80378) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 02 10:18:29.031585 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.fernet_utils [None req-27f04895-9a69-49bd-b4e7-0662abe6185a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80378) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 10:18:29.094880 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 1196/2406] 10.222.0.82 () {58 vars in 958 bytes} [Tue Aug 2 10:18:28 2022] POST /identity/v3/auth/tokens => generated 3262 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 02 10:18:29.106258 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6234eb3f-d6b2-4872-b6ee-0a35b90b9191 admin admin] Authenticating user token {{(pid=80377) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:18:29.108494 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.fernet_utils [None req-6234eb3f-d6b2-4872-b6ee-0a35b90b9191 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80377) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 10:18:29.163571 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6234eb3f-d6b2-4872-b6ee-0a35b90b9191 admin admin] Validating token access rules against request {{(pid=80377) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:18:29.167152 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf89557d-edd5-4de3-849d-04473355ec5f service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1023de281ae74e9d8a6d6c312c1c97a7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f1b38ebb804c41c69772c998aae72c41', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80377) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:18:29.167789 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf89557d-edd5-4de3-849d-04473355ec5f service cinder] REQUEST_METHOD: `GET` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:18:29.168072 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf89557d-edd5-4de3-849d-04473355ec5f service cinder] SCRIPT_NAME: `/identity` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:18:29.169638 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf89557d-edd5-4de3-849d-04473355ec5f service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:18:29.177077 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf89557d-edd5-4de3-849d-04473355ec5f service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:18:29.178596 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf89557d-edd5-4de3-849d-04473355ec5f service cinder] RBAC: Authorization granted {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:18:29.187428 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: [pid: 80377|app: 0|req: 1211/2407] 10.222.0.82 () {60 vars in 1365 bytes} [Tue Aug 2 10:18:29 2022] GET /identity/v3/auth/tokens => generated 3262 bytes in 82 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 02 10:18:37.064170 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27f04895-9a69-49bd-b4e7-0662abe6185a None None] Authenticating user token {{(pid=80378) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:18:37.080566 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG dbcounter [-] [80378] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80378) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 10:18:37.149449 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27f04895-9a69-49bd-b4e7-0662abe6185a None None] Validating token access rules against request {{(pid=80378) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:18:37.156432 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67c1e998-94c6-4ec3-acbb-ff6944be721f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '891005fa5ff74041b75747d16cc66798', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7047912c5acd4adeba5d87c3bf895e39', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80378) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:18:37.157186 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67c1e998-94c6-4ec3-acbb-ff6944be721f admin admin] REQUEST_METHOD: `DELETE` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:18:37.157847 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67c1e998-94c6-4ec3-acbb-ff6944be721f admin admin] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:18:37.157847 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67c1e998-94c6-4ec3-acbb-ff6944be721f admin admin] PATH_INFO: `/v3/users/d600f95cdb8f46cfb3d244cd3ca89886` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:18:37.159460 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67c1e998-94c6-4ec3-acbb-ff6944be721f admin admin] RBAC: Authorizing `identity:delete_user(user_id=d600f95cdb8f46cfb3d244cd3ca89886)` {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:18:37.161575 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67c1e998-94c6-4ec3-acbb-ff6944be721f admin admin] RBAC: Authorization granted {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:18:37.221436 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.notifications [None req-67c1e998-94c6-4ec3-acbb-ff6944be721f admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'd600f95cdb8f46cfb3d244cd3ca89886'} {{(pid=80378) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 10:18:37.229971 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.notifications [None req-67c1e998-94c6-4ec3-acbb-ff6944be721f admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'd600f95cdb8f46cfb3d244cd3ca89886'} {{(pid=80378) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 10:18:37.237989 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.notifications [None req-67c1e998-94c6-4ec3-acbb-ff6944be721f admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'd600f95cdb8f46cfb3d244cd3ca89886'} {{(pid=80378) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 10:18:37.238900 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.notifications [None req-67c1e998-94c6-4ec3-acbb-ff6944be721f admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'd600f95cdb8f46cfb3d244cd3ca89886'} {{(pid=80378) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 10:18:37.255457 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.notifications [None req-67c1e998-94c6-4ec3-acbb-ff6944be721f admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'd600f95cdb8f46cfb3d244cd3ca89886'} {{(pid=80378) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 10:18:37.262743 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 1197/2408] 10.222.0.82 () {58 vars in 1224 bytes} [Tue Aug 2 10:18:37 2022] DELETE /identity/v3/users/d600f95cdb8f46cfb3d244cd3ca89886 => generated 0 bytes in 199 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 02 10:18:37.267648 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-367a03fa-77b7-4259-82aa-41eca31b53f7 None None] REQUEST_METHOD: `POST` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:18:37.268144 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-367a03fa-77b7-4259-82aa-41eca31b53f7 None None] SCRIPT_NAME: `/identity` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:18:37.268250 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-367a03fa-77b7-4259-82aa-41eca31b53f7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:18:37.298064 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.auth.core [None req-367a03fa-77b7-4259-82aa-41eca31b53f7 None None] MFA Rules not processed for user `891005fa5ff74041b75747d16cc66798`. Rule list: `[]` (Enabled: `True`). {{(pid=80377) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 02 10:18:37.299377 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.fernet_utils [None req-367a03fa-77b7-4259-82aa-41eca31b53f7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80377) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 10:18:37.347730 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG dbcounter [-] [80377] Writing DB stats keystone:SELECT=91,keystone:DELETE=6,keystone:INSERT=3 {{(pid=80377) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 10:18:37.365601 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: [pid: 80377|app: 0|req: 1212/2409] 10.222.0.82 () {58 vars in 958 bytes} [Tue Aug 2 10:18:37 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 02 10:18:37.375491 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67c1e998-94c6-4ec3-acbb-ff6944be721f admin admin] Authenticating user token {{(pid=80378) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:18:37.376901 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.fernet_utils [None req-67c1e998-94c6-4ec3-acbb-ff6944be721f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80378) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 10:18:37.437405 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67c1e998-94c6-4ec3-acbb-ff6944be721f admin admin] Validating token access rules against request {{(pid=80378) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:18:37.441161 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6623da69-0cf6-4f0e-ae3d-7449e8f50242 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '33b87170f0634b67bce60750847b5cbe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f1b38ebb804c41c69772c998aae72c41', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80378) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:18:37.441956 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6623da69-0cf6-4f0e-ae3d-7449e8f50242 service neutron] REQUEST_METHOD: `GET` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:18:37.442243 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6623da69-0cf6-4f0e-ae3d-7449e8f50242 service neutron] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:18:37.442548 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6623da69-0cf6-4f0e-ae3d-7449e8f50242 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:18:37.451840 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6623da69-0cf6-4f0e-ae3d-7449e8f50242 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:18:37.453291 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6623da69-0cf6-4f0e-ae3d-7449e8f50242 service neutron] RBAC: Authorization granted {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:18:37.463473 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 1198/2410] 10.222.0.82 () {60 vars in 1365 bytes} [Tue Aug 2 10:18:37 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 88 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 02 10:18:37.969395 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-367a03fa-77b7-4259-82aa-41eca31b53f7 None None] Authenticating user token {{(pid=80377) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:18:37.971013 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.fernet_utils [None req-367a03fa-77b7-4259-82aa-41eca31b53f7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80377) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 10:18:37.985174 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-367a03fa-77b7-4259-82aa-41eca31b53f7 None None] Validating token access rules against request {{(pid=80377) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:18:38.009715 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f42768c-4f49-4ceb-a287-846b64b1a572 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '891005fa5ff74041b75747d16cc66798', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7047912c5acd4adeba5d87c3bf895e39', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80377) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:18:38.010599 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f42768c-4f49-4ceb-a287-846b64b1a572 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:18:38.011539 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f42768c-4f49-4ceb-a287-846b64b1a572 admin admin] SCRIPT_NAME: `/identity` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:18:38.011841 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f42768c-4f49-4ceb-a287-846b64b1a572 admin admin] PATH_INFO: `/v3/projects/3b0a4e80b761485ba4bf70c78ebaf599` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:18:38.013213 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f42768c-4f49-4ceb-a287-846b64b1a572 admin admin] RBAC: Authorizing `identity:delete_project(project_id=3b0a4e80b761485ba4bf70c78ebaf599)` {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:18:38.015702 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f42768c-4f49-4ceb-a287-846b64b1a572 admin admin] RBAC: Authorization granted {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:18:38.074094 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.notifications [None req-9f42768c-4f49-4ceb-a287-846b64b1a572 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '3b0a4e80b761485ba4bf70c78ebaf599', 'request_id': 'req-9f42768c-4f49-4ceb-a287-846b64b1a572'} {{(pid=80377) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 10:18:38.080822 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.notifications [None req-9f42768c-4f49-4ceb-a287-846b64b1a572 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '3b0a4e80b761485ba4bf70c78ebaf599', 'request_id': 'req-9f42768c-4f49-4ceb-a287-846b64b1a572'} {{(pid=80377) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 10:18:38.089212 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.notifications [None req-9f42768c-4f49-4ceb-a287-846b64b1a572 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '3b0a4e80b761485ba4bf70c78ebaf599', 'request_id': 'req-9f42768c-4f49-4ceb-a287-846b64b1a572'} {{(pid=80377) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 10:18:38.106065 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.notifications [None req-9f42768c-4f49-4ceb-a287-846b64b1a572 admin admin] The token cache is being invalidate because project 3b0a4e80b761485ba4bf70c78ebaf599 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80377) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 02 10:18:38.106361 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.notifications [None req-9f42768c-4f49-4ceb-a287-846b64b1a572 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80377) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 10:18:38.107042 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.notifications [None req-9f42768c-4f49-4ceb-a287-846b64b1a572 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80377) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 10:18:38.109520 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: [pid: 80377|app: 0|req: 1213/2411] 10.222.0.82 () {58 vars in 1233 bytes} [Tue Aug 2 10:18:37 2022] DELETE /identity/v3/projects/3b0a4e80b761485ba4bf70c78ebaf599 => generated 0 bytes in 140 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 02 10:18:38.114469 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63b5d848-0b5c-4930-822f-c9b73a6360bb None None] REQUEST_METHOD: `POST` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:18:38.116913 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63b5d848-0b5c-4930-822f-c9b73a6360bb None None] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:18:38.119824 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63b5d848-0b5c-4930-822f-c9b73a6360bb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:18:38.147107 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.auth.core [None req-63b5d848-0b5c-4930-822f-c9b73a6360bb None None] MFA Rules not processed for user `891005fa5ff74041b75747d16cc66798`. Rule list: `[]` (Enabled: `True`). {{(pid=80378) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 02 10:18:38.147483 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.fernet_utils [None req-63b5d848-0b5c-4930-822f-c9b73a6360bb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80378) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 10:18:38.191854 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG dbcounter [-] [80378] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80378) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 10:18:38.231275 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 1199/2412] 10.222.0.82 () {58 vars in 958 bytes} [Tue Aug 2 10:18:38 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 118 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 02 10:18:38.239773 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ed80736-f017-4ced-afbd-a245b66accb7 None None] REQUEST_METHOD: `POST` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:18:38.240130 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ed80736-f017-4ced-afbd-a245b66accb7 None None] SCRIPT_NAME: `/identity` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:18:38.240665 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ed80736-f017-4ced-afbd-a245b66accb7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:18:38.271062 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.auth.core [None req-7ed80736-f017-4ced-afbd-a245b66accb7 None None] MFA Rules not processed for user `891005fa5ff74041b75747d16cc66798`. Rule list: `[]` (Enabled: `True`). {{(pid=80377) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 02 10:18:38.272373 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.fernet_utils [None req-7ed80736-f017-4ced-afbd-a245b66accb7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80377) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 10:18:38.282974 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: [pid: 80377|app: 0|req: 1214/2413] 10.222.0.82 () {58 vars in 958 bytes} [Tue Aug 2 10:18:38 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 44 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 02 10:18:38.287963 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63b5d848-0b5c-4930-822f-c9b73a6360bb None None] Authenticating user token {{(pid=80378) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:18:38.299825 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63b5d848-0b5c-4930-822f-c9b73a6360bb None None] Validating token access rules against request {{(pid=80378) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:18:38.305741 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-907d42c6-dd80-425b-a775-f8e6d7fe4f24 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '891005fa5ff74041b75747d16cc66798', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7047912c5acd4adeba5d87c3bf895e39', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80378) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:18:38.306961 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-907d42c6-dd80-425b-a775-f8e6d7fe4f24 admin admin] REQUEST_METHOD: `GET` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:18:38.307487 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-907d42c6-dd80-425b-a775-f8e6d7fe4f24 admin admin] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:18:38.307941 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-907d42c6-dd80-425b-a775-f8e6d7fe4f24 admin admin] PATH_INFO: `/v3/domains` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:18:38.308811 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-907d42c6-dd80-425b-a775-f8e6d7fe4f24 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80378) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 02 10:18:38.309281 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-907d42c6-dd80-425b-a775-f8e6d7fe4f24 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:18:38.311695 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-907d42c6-dd80-425b-a775-f8e6d7fe4f24 admin admin] RBAC: Authorization granted {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:18:38.325598 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 1200/2414] 10.222.0.82 () {58 vars in 1153 bytes} [Tue Aug 2 10:18:38 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 38 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 02 10:18:38.331015 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ed80736-f017-4ced-afbd-a245b66accb7 None None] Authenticating user token {{(pid=80377) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:18:38.344317 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ed80736-f017-4ced-afbd-a245b66accb7 None None] Validating token access rules against request {{(pid=80377) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:18:38.349175 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5f48645-258a-4f87-adc2-ab3955dc95db admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '891005fa5ff74041b75747d16cc66798', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7047912c5acd4adeba5d87c3bf895e39', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80377) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:18:38.349852 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5f48645-258a-4f87-adc2-ab3955dc95db admin admin] REQUEST_METHOD: `POST` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:18:38.350141 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5f48645-258a-4f87-adc2-ab3955dc95db admin admin] SCRIPT_NAME: `/identity` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:18:38.350415 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5f48645-258a-4f87-adc2-ab3955dc95db admin admin] PATH_INFO: `/v3/projects` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:18:38.351096 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5f48645-258a-4f87-adc2-ab3955dc95db admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:18:38.352998 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5f48645-258a-4f87-adc2-ab3955dc95db admin admin] RBAC: Authorization granted {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:18:38.400940 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: [pid: 80377|app: 0|req: 1215/2415] 10.222.0.82 () {60 vars in 1153 bytes} [Tue Aug 2 10:18:38 2022] POST /identity/v3/projects => generated 389 bytes in 71 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 02 10:18:38.406450 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-907d42c6-dd80-425b-a775-f8e6d7fe4f24 admin admin] Authenticating user token {{(pid=80378) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:18:38.422107 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-907d42c6-dd80-425b-a775-f8e6d7fe4f24 admin admin] Validating token access rules against request {{(pid=80378) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:18:38.508020 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e68b8e46-1dbf-442d-a412-0bb4d447b7d3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '891005fa5ff74041b75747d16cc66798', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7047912c5acd4adeba5d87c3bf895e39', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80378) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:18:38.509033 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e68b8e46-1dbf-442d-a412-0bb4d447b7d3 admin admin] REQUEST_METHOD: `POST` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:18:38.509254 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e68b8e46-1dbf-442d-a412-0bb4d447b7d3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:18:38.509622 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e68b8e46-1dbf-442d-a412-0bb4d447b7d3 admin admin] PATH_INFO: `/v3/users` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:18:38.510536 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e68b8e46-1dbf-442d-a412-0bb4d447b7d3 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:18:38.512719 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e68b8e46-1dbf-442d-a412-0bb4d447b7d3 admin admin] RBAC: Authorization granted {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:18:38.538096 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 1201/2416] 10.222.0.82 () {60 vars in 1144 bytes} [Tue Aug 2 10:18:38 2022] POST /identity/v3/users => generated 295 bytes in 133 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 02 10:18:38.541880 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5f48645-258a-4f87-adc2-ab3955dc95db admin admin] Authenticating user token {{(pid=80377) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:18:38.555023 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5f48645-258a-4f87-adc2-ab3955dc95db admin admin] Validating token access rules against request {{(pid=80377) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:18:38.560773 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c2a6e8f-4ab7-42ae-8ac3-33e471885778 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '891005fa5ff74041b75747d16cc66798', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7047912c5acd4adeba5d87c3bf895e39', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80377) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:18:38.561524 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c2a6e8f-4ab7-42ae-8ac3-33e471885778 admin admin] REQUEST_METHOD: `GET` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:18:38.561847 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c2a6e8f-4ab7-42ae-8ac3-33e471885778 admin admin] SCRIPT_NAME: `/identity` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:18:38.562135 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c2a6e8f-4ab7-42ae-8ac3-33e471885778 admin admin] PATH_INFO: `/v3/roles` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:18:38.562852 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c2a6e8f-4ab7-42ae-8ac3-33e471885778 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:18:38.564657 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c2a6e8f-4ab7-42ae-8ac3-33e471885778 admin admin] RBAC: Authorization granted {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:18:38.575608 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: [pid: 80377|app: 0|req: 1216/2417] 10.222.0.82 () {58 vars in 1122 bytes} [Tue Aug 2 10:18:38 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 02 10:18:38.581107 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e68b8e46-1dbf-442d-a412-0bb4d447b7d3 admin admin] Authenticating user token {{(pid=80378) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:18:38.591267 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e68b8e46-1dbf-442d-a412-0bb4d447b7d3 admin admin] Validating token access rules against request {{(pid=80378) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:18:38.596993 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da29d478-870e-45d2-89e3-525d2b2b67ca admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '891005fa5ff74041b75747d16cc66798', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7047912c5acd4adeba5d87c3bf895e39', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80378) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:18:38.597729 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da29d478-870e-45d2-89e3-525d2b2b67ca admin admin] REQUEST_METHOD: `GET` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:18:38.598054 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da29d478-870e-45d2-89e3-525d2b2b67ca admin admin] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:18:38.598332 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da29d478-870e-45d2-89e3-525d2b2b67ca admin admin] PATH_INFO: `/v3/roles` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:18:38.599076 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da29d478-870e-45d2-89e3-525d2b2b67ca admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:18:38.600924 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da29d478-870e-45d2-89e3-525d2b2b67ca admin admin] RBAC: Authorization granted {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:18:38.616014 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 1202/2418] 10.222.0.82 () {58 vars in 1122 bytes} [Tue Aug 2 10:18:38 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 02 10:18:38.616571 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c2a6e8f-4ab7-42ae-8ac3-33e471885778 admin admin] Authenticating user token {{(pid=80377) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:18:38.627271 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c2a6e8f-4ab7-42ae-8ac3-33e471885778 admin admin] Validating token access rules against request {{(pid=80377) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:18:38.632191 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da0bb1b1-5f6a-4245-9284-8400c042c0cd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '891005fa5ff74041b75747d16cc66798', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7047912c5acd4adeba5d87c3bf895e39', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80377) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:18:38.632868 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da0bb1b1-5f6a-4245-9284-8400c042c0cd admin admin] REQUEST_METHOD: `PUT` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:18:38.633154 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da0bb1b1-5f6a-4245-9284-8400c042c0cd admin admin] SCRIPT_NAME: `/identity` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:18:38.633434 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da0bb1b1-5f6a-4245-9284-8400c042c0cd admin admin] PATH_INFO: `/v3/projects/a6a5d6b2448d4402bbed3e8e7e84de60/users/de8c39731658450db5cdb58330292590/roles/e1c04b19fc4543b8b54f58bbcffbc598` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:18:38.649649 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da0bb1b1-5f6a-4245-9284-8400c042c0cd admin admin] RBAC: Authorizing `identity:create_grant(project_id=a6a5d6b2448d4402bbed3e8e7e84de60, user_id=de8c39731658450db5cdb58330292590, role_id=e1c04b19fc4543b8b54f58bbcffbc598)` {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:18:38.652369 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da0bb1b1-5f6a-4245-9284-8400c042c0cd admin admin] RBAC: Authorization granted {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:18:38.665376 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: [pid: 80377|app: 0|req: 1217/2419] 10.222.0.82 () {60 vars in 1483 bytes} [Tue Aug 2 10:18:38 2022] PUT /identity/v3/projects/a6a5d6b2448d4402bbed3e8e7e84de60/users/de8c39731658450db5cdb58330292590/roles/e1c04b19fc4543b8b54f58bbcffbc598 => generated 0 bytes in 49 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 02 10:18:38.675865 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b81b3569-eb34-4ad6-8e7b-1a0bb3f01ffb None None] REQUEST_METHOD: `POST` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:18:38.676153 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b81b3569-eb34-4ad6-8e7b-1a0bb3f01ffb None None] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:18:38.676454 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b81b3569-eb34-4ad6-8e7b-1a0bb3f01ffb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:18:38.705748 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.auth.core [None req-b81b3569-eb34-4ad6-8e7b-1a0bb3f01ffb None None] MFA Rules not processed for user `de8c39731658450db5cdb58330292590`. Rule list: `[]` (Enabled: `True`). {{(pid=80378) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 02 10:18:38.706979 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.fernet_utils [None req-b81b3569-eb34-4ad6-8e7b-1a0bb3f01ffb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80378) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 10:18:38.780092 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 1203/2420] 10.222.0.82 () {58 vars in 958 bytes} [Tue Aug 2 10:18:38 2022] POST /identity/v3/auth/tokens => generated 3266 bytes in 105 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 02 10:18:38.793311 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da0bb1b1-5f6a-4245-9284-8400c042c0cd admin admin] Authenticating user token {{(pid=80377) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:18:38.795049 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.fernet_utils [None req-da0bb1b1-5f6a-4245-9284-8400c042c0cd admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80377) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 10:18:38.856541 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da0bb1b1-5f6a-4245-9284-8400c042c0cd admin admin] Validating token access rules against request {{(pid=80377) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:18:38.860352 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8d69496-2fa1-4fb8-9f18-eb037cc8932d service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1023de281ae74e9d8a6d6c312c1c97a7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f1b38ebb804c41c69772c998aae72c41', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80377) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:18:38.860983 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8d69496-2fa1-4fb8-9f18-eb037cc8932d service cinder] REQUEST_METHOD: `GET` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:18:38.861268 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8d69496-2fa1-4fb8-9f18-eb037cc8932d service cinder] SCRIPT_NAME: `/identity` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:18:38.861549 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8d69496-2fa1-4fb8-9f18-eb037cc8932d service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:18:38.870374 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8d69496-2fa1-4fb8-9f18-eb037cc8932d service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:18:38.871820 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8d69496-2fa1-4fb8-9f18-eb037cc8932d service cinder] RBAC: Authorization granted {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:18:38.882696 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: [pid: 80377|app: 0|req: 1218/2421] 10.222.0.82 () {60 vars in 1365 bytes} [Tue Aug 2 10:18:38 2022] GET /identity/v3/auth/tokens => generated 3266 bytes in 90 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 02 10:18:48.777152 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG dbcounter [-] [80378] Writing DB stats keystone:SELECT=83,keystone:INSERT=3 {{(pid=80378) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 10:18:48.878673 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG dbcounter [-] [80377] Writing DB stats keystone:SELECT=84,keystone:DELETE=7,keystone:INSERT=3 {{(pid=80377) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 10:19:07.646607 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b81b3569-eb34-4ad6-8e7b-1a0bb3f01ffb None None] Authenticating user token {{(pid=80378) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:19:07.709250 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b81b3569-eb34-4ad6-8e7b-1a0bb3f01ffb None None] Validating token access rules against request {{(pid=80378) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:19:07.713700 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d076994-1808-4494-86b9-513a3e4bd5e4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '891005fa5ff74041b75747d16cc66798', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7047912c5acd4adeba5d87c3bf895e39', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80378) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:19:07.714368 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d076994-1808-4494-86b9-513a3e4bd5e4 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:19:07.714650 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d076994-1808-4494-86b9-513a3e4bd5e4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:19:07.714925 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d076994-1808-4494-86b9-513a3e4bd5e4 admin admin] PATH_INFO: `/v3/users/de8c39731658450db5cdb58330292590` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:19:07.716090 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d076994-1808-4494-86b9-513a3e4bd5e4 admin admin] RBAC: Authorizing `identity:delete_user(user_id=de8c39731658450db5cdb58330292590)` {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:19:07.718046 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d076994-1808-4494-86b9-513a3e4bd5e4 admin admin] RBAC: Authorization granted {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:19:07.763946 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.notifications [None req-4d076994-1808-4494-86b9-513a3e4bd5e4 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'de8c39731658450db5cdb58330292590'} {{(pid=80378) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 10:19:07.772140 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.notifications [None req-4d076994-1808-4494-86b9-513a3e4bd5e4 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'de8c39731658450db5cdb58330292590'} {{(pid=80378) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 10:19:07.779233 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.notifications [None req-4d076994-1808-4494-86b9-513a3e4bd5e4 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'de8c39731658450db5cdb58330292590'} {{(pid=80378) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 10:19:07.779898 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.notifications [None req-4d076994-1808-4494-86b9-513a3e4bd5e4 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'de8c39731658450db5cdb58330292590'} {{(pid=80378) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 10:19:07.793368 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.notifications [None req-4d076994-1808-4494-86b9-513a3e4bd5e4 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'de8c39731658450db5cdb58330292590'} {{(pid=80378) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 10:19:07.800003 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 1204/2422] 10.222.0.82 () {58 vars in 1224 bytes} [Tue Aug 2 10:19:07 2022] DELETE /identity/v3/users/de8c39731658450db5cdb58330292590 => generated 0 bytes in 154 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 02 10:19:07.804370 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87a59c82-8ef1-49f4-b9e1-d0491a27f767 None None] REQUEST_METHOD: `POST` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:19:07.804823 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87a59c82-8ef1-49f4-b9e1-d0491a27f767 None None] SCRIPT_NAME: `/identity` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:19:07.804953 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87a59c82-8ef1-49f4-b9e1-d0491a27f767 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:19:07.824847 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.auth.core [None req-87a59c82-8ef1-49f4-b9e1-d0491a27f767 None None] MFA Rules not processed for user `891005fa5ff74041b75747d16cc66798`. Rule list: `[]` (Enabled: `True`). {{(pid=80377) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 02 10:19:07.826081 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.fernet_utils [None req-87a59c82-8ef1-49f4-b9e1-d0491a27f767 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80377) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 10:19:07.885547 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: [pid: 80377|app: 0|req: 1219/2423] 10.222.0.82 () {58 vars in 958 bytes} [Tue Aug 2 10:19:07 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 82 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 02 10:19:07.895635 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d076994-1808-4494-86b9-513a3e4bd5e4 admin admin] Authenticating user token {{(pid=80378) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:19:07.896994 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.fernet_utils [None req-4d076994-1808-4494-86b9-513a3e4bd5e4 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80378) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 10:19:07.954167 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d076994-1808-4494-86b9-513a3e4bd5e4 admin admin] Validating token access rules against request {{(pid=80378) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:19:07.957739 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0130f8f-b477-4de4-8c8c-d8ca850b1777 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '33b87170f0634b67bce60750847b5cbe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f1b38ebb804c41c69772c998aae72c41', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80378) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:19:07.958546 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0130f8f-b477-4de4-8c8c-d8ca850b1777 service neutron] REQUEST_METHOD: `GET` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:19:07.958833 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0130f8f-b477-4de4-8c8c-d8ca850b1777 service neutron] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:19:07.959965 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0130f8f-b477-4de4-8c8c-d8ca850b1777 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:19:07.969532 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0130f8f-b477-4de4-8c8c-d8ca850b1777 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:19:07.971004 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0130f8f-b477-4de4-8c8c-d8ca850b1777 service neutron] RBAC: Authorization granted {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:19:07.980466 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 1205/2424] 10.222.0.82 () {60 vars in 1365 bytes} [Tue Aug 2 10:19:07 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 85 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 02 10:19:08.267572 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87a59c82-8ef1-49f4-b9e1-d0491a27f767 None None] Authenticating user token {{(pid=80377) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:19:08.269039 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.fernet_utils [None req-87a59c82-8ef1-49f4-b9e1-d0491a27f767 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80377) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 10:19:08.281066 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87a59c82-8ef1-49f4-b9e1-d0491a27f767 None None] Validating token access rules against request {{(pid=80377) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:19:08.286780 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c58823de-7714-4881-aeea-bd6b8de645ed admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '891005fa5ff74041b75747d16cc66798', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7047912c5acd4adeba5d87c3bf895e39', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80377) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:19:08.288372 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c58823de-7714-4881-aeea-bd6b8de645ed admin admin] REQUEST_METHOD: `DELETE` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:19:08.288665 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c58823de-7714-4881-aeea-bd6b8de645ed admin admin] SCRIPT_NAME: `/identity` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:19:08.288944 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c58823de-7714-4881-aeea-bd6b8de645ed admin admin] PATH_INFO: `/v3/projects/a6a5d6b2448d4402bbed3e8e7e84de60` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:19:08.290184 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c58823de-7714-4881-aeea-bd6b8de645ed admin admin] RBAC: Authorizing `identity:delete_project(project_id=a6a5d6b2448d4402bbed3e8e7e84de60)` {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:19:08.292867 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c58823de-7714-4881-aeea-bd6b8de645ed admin admin] RBAC: Authorization granted {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:19:08.348471 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.notifications [None req-c58823de-7714-4881-aeea-bd6b8de645ed admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'a6a5d6b2448d4402bbed3e8e7e84de60', 'request_id': 'req-c58823de-7714-4881-aeea-bd6b8de645ed'} {{(pid=80377) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 10:19:08.354876 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.notifications [None req-c58823de-7714-4881-aeea-bd6b8de645ed admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'a6a5d6b2448d4402bbed3e8e7e84de60', 'request_id': 'req-c58823de-7714-4881-aeea-bd6b8de645ed'} {{(pid=80377) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 10:19:08.361462 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.notifications [None req-c58823de-7714-4881-aeea-bd6b8de645ed admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'a6a5d6b2448d4402bbed3e8e7e84de60', 'request_id': 'req-c58823de-7714-4881-aeea-bd6b8de645ed'} {{(pid=80377) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 10:19:08.380015 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.notifications [None req-c58823de-7714-4881-aeea-bd6b8de645ed admin admin] The token cache is being invalidate because project a6a5d6b2448d4402bbed3e8e7e84de60 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80377) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 02 10:19:08.380348 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.notifications [None req-c58823de-7714-4881-aeea-bd6b8de645ed admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80377) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 10:19:08.381226 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.notifications [None req-c58823de-7714-4881-aeea-bd6b8de645ed admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80377) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 10:19:08.383351 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: [pid: 80377|app: 0|req: 1220/2425] 10.222.0.82 () {58 vars in 1233 bytes} [Tue Aug 2 10:19:08 2022] DELETE /identity/v3/projects/a6a5d6b2448d4402bbed3e8e7e84de60 => generated 0 bytes in 116 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 02 10:19:17.976152 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG dbcounter [-] [80378] Writing DB stats keystone:SELECT=79,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80378) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 10:19:18.376012 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG dbcounter [-] [80377] Writing DB stats keystone:SELECT=54,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80377) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 10:19:18.850480 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0130f8f-b477-4de4-8c8c-d8ca850b1777 service neutron] Authenticating user token {{(pid=80378) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:19:18.851951 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.fernet_utils [None req-b0130f8f-b477-4de4-8c8c-d8ca850b1777 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80378) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 10:19:18.923376 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0130f8f-b477-4de4-8c8c-d8ca850b1777 service neutron] Validating token access rules against request {{(pid=80378) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:19:18.981585 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9236be3c-0b60-4ba2-8318-e46eab508a76 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ae6e910e50d74f29bfb89bf18d95a908', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f1b38ebb804c41c69772c998aae72c41', 'project_domain_id': 'default', 'roles': ['admin', 'service', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80378) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:19:18.982258 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9236be3c-0b60-4ba2-8318-e46eab508a76 service placement] REQUEST_METHOD: `GET` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:19:18.982524 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9236be3c-0b60-4ba2-8318-e46eab508a76 service placement] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:19:18.982803 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9236be3c-0b60-4ba2-8318-e46eab508a76 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:19:18.984607 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.fernet_utils [None req-9236be3c-0b60-4ba2-8318-e46eab508a76 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80378) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 10:19:19.001113 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9236be3c-0b60-4ba2-8318-e46eab508a76 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:19:19.002790 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9236be3c-0b60-4ba2-8318-e46eab508a76 service placement] RBAC: Authorization granted {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:19:19.015085 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 1206/2426] 10.222.0.82 () {60 vars in 1365 bytes} [Tue Aug 2 10:19:18 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 165 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 02 10:19:20.712506 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c58823de-7714-4881-aeea-bd6b8de645ed admin admin] Authenticating user token {{(pid=80377) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:19:20.714054 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.fernet_utils [None req-c58823de-7714-4881-aeea-bd6b8de645ed admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80377) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 10:19:20.797775 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c58823de-7714-4881-aeea-bd6b8de645ed admin admin] Validating token access rules against request {{(pid=80377) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:19:20.802605 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ee9f4ca-a8c8-4f67-bc13-ddf84981cea6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '891005fa5ff74041b75747d16cc66798', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7047912c5acd4adeba5d87c3bf895e39', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80377) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:19:20.803262 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ee9f4ca-a8c8-4f67-bc13-ddf84981cea6 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:19:20.803540 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ee9f4ca-a8c8-4f67-bc13-ddf84981cea6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:19:20.803810 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ee9f4ca-a8c8-4f67-bc13-ddf84981cea6 admin admin] PATH_INFO: `/v3/users/1e7b3d7f3da64f90b6ca4f19b7ca9a02` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:19:20.804974 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ee9f4ca-a8c8-4f67-bc13-ddf84981cea6 admin admin] RBAC: Authorizing `identity:delete_user(user_id=1e7b3d7f3da64f90b6ca4f19b7ca9a02)` {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:19:20.806991 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ee9f4ca-a8c8-4f67-bc13-ddf84981cea6 admin admin] RBAC: Authorization granted {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:19:20.871792 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.notifications [None req-1ee9f4ca-a8c8-4f67-bc13-ddf84981cea6 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '1e7b3d7f3da64f90b6ca4f19b7ca9a02'} {{(pid=80377) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 10:19:20.888583 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.notifications [None req-1ee9f4ca-a8c8-4f67-bc13-ddf84981cea6 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '1e7b3d7f3da64f90b6ca4f19b7ca9a02'} {{(pid=80377) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 10:19:20.897329 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.notifications [None req-1ee9f4ca-a8c8-4f67-bc13-ddf84981cea6 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '1e7b3d7f3da64f90b6ca4f19b7ca9a02'} {{(pid=80377) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 10:19:20.898022 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.notifications [None req-1ee9f4ca-a8c8-4f67-bc13-ddf84981cea6 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '1e7b3d7f3da64f90b6ca4f19b7ca9a02'} {{(pid=80377) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 10:19:20.911228 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.notifications [None req-1ee9f4ca-a8c8-4f67-bc13-ddf84981cea6 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '1e7b3d7f3da64f90b6ca4f19b7ca9a02'} {{(pid=80377) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 10:19:20.918124 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: [pid: 80377|app: 0|req: 1221/2427] 10.222.0.82 () {58 vars in 1224 bytes} [Tue Aug 2 10:19:20 2022] DELETE /identity/v3/users/1e7b3d7f3da64f90b6ca4f19b7ca9a02 => generated 0 bytes in 206 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 02 10:19:20.921910 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48b433d5-c7a7-4bbb-adeb-cf4c4bc63418 None None] REQUEST_METHOD: `POST` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:19:20.922423 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48b433d5-c7a7-4bbb-adeb-cf4c4bc63418 None None] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:19:20.922544 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48b433d5-c7a7-4bbb-adeb-cf4c4bc63418 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:19:20.942697 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.auth.core [None req-48b433d5-c7a7-4bbb-adeb-cf4c4bc63418 None None] MFA Rules not processed for user `891005fa5ff74041b75747d16cc66798`. Rule list: `[]` (Enabled: `True`). {{(pid=80378) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 02 10:19:20.943918 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.fernet_utils [None req-48b433d5-c7a7-4bbb-adeb-cf4c4bc63418 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80378) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 10:19:21.008234 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 1207/2428] 10.222.0.82 () {58 vars in 958 bytes} [Tue Aug 2 10:19:20 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 02 10:19:21.018621 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ee9f4ca-a8c8-4f67-bc13-ddf84981cea6 admin admin] Authenticating user token {{(pid=80377) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:19:21.021033 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.fernet_utils [None req-1ee9f4ca-a8c8-4f67-bc13-ddf84981cea6 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80377) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 10:19:21.082787 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ee9f4ca-a8c8-4f67-bc13-ddf84981cea6 admin admin] Validating token access rules against request {{(pid=80377) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:19:21.086392 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb0d9f57-71bf-4a47-9fea-8fd3d4d0c4da service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '33b87170f0634b67bce60750847b5cbe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f1b38ebb804c41c69772c998aae72c41', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80377) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:19:21.087017 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb0d9f57-71bf-4a47-9fea-8fd3d4d0c4da service neutron] REQUEST_METHOD: `GET` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:19:21.087356 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb0d9f57-71bf-4a47-9fea-8fd3d4d0c4da service neutron] SCRIPT_NAME: `/identity` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:19:21.087632 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb0d9f57-71bf-4a47-9fea-8fd3d4d0c4da service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:19:21.097609 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb0d9f57-71bf-4a47-9fea-8fd3d4d0c4da service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:19:21.099379 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb0d9f57-71bf-4a47-9fea-8fd3d4d0c4da service neutron] RBAC: Authorization granted {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:19:21.110611 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: [pid: 80377|app: 0|req: 1222/2429] 10.222.0.82 () {60 vars in 1365 bytes} [Tue Aug 2 10:19:21 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 92 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 02 10:19:21.543471 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48b433d5-c7a7-4bbb-adeb-cf4c4bc63418 None None] Authenticating user token {{(pid=80378) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:19:21.545614 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.fernet_utils [None req-48b433d5-c7a7-4bbb-adeb-cf4c4bc63418 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80378) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 10:19:21.561272 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48b433d5-c7a7-4bbb-adeb-cf4c4bc63418 None None] Validating token access rules against request {{(pid=80378) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:19:21.571912 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-055283ca-4a64-4937-961f-987876a0ec1a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '891005fa5ff74041b75747d16cc66798', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7047912c5acd4adeba5d87c3bf895e39', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80378) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:19:21.572934 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-055283ca-4a64-4937-961f-987876a0ec1a admin admin] REQUEST_METHOD: `DELETE` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:19:21.572934 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-055283ca-4a64-4937-961f-987876a0ec1a admin admin] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:19:21.573190 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-055283ca-4a64-4937-961f-987876a0ec1a admin admin] PATH_INFO: `/v3/projects/ccf59d67afaf4235bcc5eb1b7abb71fe` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:19:21.574946 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-055283ca-4a64-4937-961f-987876a0ec1a admin admin] RBAC: Authorizing `identity:delete_project(project_id=ccf59d67afaf4235bcc5eb1b7abb71fe)` {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:19:21.576999 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-055283ca-4a64-4937-961f-987876a0ec1a admin admin] RBAC: Authorization granted {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:19:21.632282 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG dbcounter [-] [80378] Writing DB stats keystone:SELECT=98,keystone:DELETE=2 {{(pid=80378) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 10:19:21.656653 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.notifications [None req-055283ca-4a64-4937-961f-987876a0ec1a admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'ccf59d67afaf4235bcc5eb1b7abb71fe', 'request_id': 'req-055283ca-4a64-4937-961f-987876a0ec1a'} {{(pid=80378) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 10:19:21.663761 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.notifications [None req-055283ca-4a64-4937-961f-987876a0ec1a admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'ccf59d67afaf4235bcc5eb1b7abb71fe', 'request_id': 'req-055283ca-4a64-4937-961f-987876a0ec1a'} {{(pid=80378) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 10:19:21.670381 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.notifications [None req-055283ca-4a64-4937-961f-987876a0ec1a admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'ccf59d67afaf4235bcc5eb1b7abb71fe', 'request_id': 'req-055283ca-4a64-4937-961f-987876a0ec1a'} {{(pid=80378) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 10:19:21.691957 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.notifications [None req-055283ca-4a64-4937-961f-987876a0ec1a admin admin] The token cache is being invalidate because project ccf59d67afaf4235bcc5eb1b7abb71fe was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80378) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 02 10:19:21.692522 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.notifications [None req-055283ca-4a64-4937-961f-987876a0ec1a admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80378) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 10:19:21.694510 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.notifications [None req-055283ca-4a64-4937-961f-987876a0ec1a admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80378) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 10:19:21.696410 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 1208/2430] 10.222.0.82 () {58 vars in 1233 bytes} [Tue Aug 2 10:19:21 2022] DELETE /identity/v3/projects/ccf59d67afaf4235bcc5eb1b7abb71fe => generated 0 bytes in 154 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 02 10:19:22.878518 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79641c0d-1dce-46ca-a5cb-2850587b102e None None] REQUEST_METHOD: `POST` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:19:22.879048 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79641c0d-1dce-46ca-a5cb-2850587b102e None None] SCRIPT_NAME: `/identity` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:19:22.879048 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79641c0d-1dce-46ca-a5cb-2850587b102e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:19:22.899140 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.auth.core [None req-79641c0d-1dce-46ca-a5cb-2850587b102e None None] MFA Rules not processed for user `a5f2ec904f214a708a3691da24fb516d`. Rule list: `[]` (Enabled: `True`). {{(pid=80377) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 02 10:19:22.900349 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.fernet_utils [None req-79641c0d-1dce-46ca-a5cb-2850587b102e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80377) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 10:19:22.917230 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG dbcounter [-] [80377] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80377) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 10:19:22.960065 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: [pid: 80377|app: 0|req: 1223/2431] 10.222.0.82 () {58 vars in 958 bytes} [Tue Aug 2 10:19:22 2022] POST /identity/v3/auth/tokens => generated 3248 bytes in 83 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 02 10:19:24.630290 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-055283ca-4a64-4937-961f-987876a0ec1a admin admin] Authenticating user token {{(pid=80378) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:19:24.631752 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.fernet_utils [None req-055283ca-4a64-4937-961f-987876a0ec1a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80378) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 10:19:24.696828 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-055283ca-4a64-4937-961f-987876a0ec1a admin admin] Validating token access rules against request {{(pid=80378) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:19:24.701417 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15abc22b-c05b-44c4-88f5-4c08ccd26e64 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '33b87170f0634b67bce60750847b5cbe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f1b38ebb804c41c69772c998aae72c41', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80378) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:19:24.702078 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15abc22b-c05b-44c4-88f5-4c08ccd26e64 service neutron] REQUEST_METHOD: `GET` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:19:24.702363 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15abc22b-c05b-44c4-88f5-4c08ccd26e64 service neutron] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:19:24.702701 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15abc22b-c05b-44c4-88f5-4c08ccd26e64 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:19:24.705223 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.fernet_utils [None req-15abc22b-c05b-44c4-88f5-4c08ccd26e64 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80378) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 10:19:24.770731 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15abc22b-c05b-44c4-88f5-4c08ccd26e64 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:19:24.772175 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15abc22b-c05b-44c4-88f5-4c08ccd26e64 service neutron] RBAC: Authorization granted {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:19:24.782766 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 1209/2432] 10.222.0.82 () {60 vars in 1365 bytes} [Tue Aug 2 10:19:24 2022] GET /identity/v3/auth/tokens => generated 3315 bytes in 153 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 02 10:19:31.240754 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79641c0d-1dce-46ca-a5cb-2850587b102e None None] Authenticating user token {{(pid=80377) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:19:31.252057 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79641c0d-1dce-46ca-a5cb-2850587b102e None None] Validating token access rules against request {{(pid=80377) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:19:31.256317 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18209d0d-9e0a-42cb-87a1-11bef80f1fd6 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '33b87170f0634b67bce60750847b5cbe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f1b38ebb804c41c69772c998aae72c41', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80377) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:19:31.256949 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18209d0d-9e0a-42cb-87a1-11bef80f1fd6 service neutron] REQUEST_METHOD: `GET` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:19:31.257235 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18209d0d-9e0a-42cb-87a1-11bef80f1fd6 service neutron] SCRIPT_NAME: `/identity` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:19:31.257511 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18209d0d-9e0a-42cb-87a1-11bef80f1fd6 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:19:31.260047 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.fernet_utils [None req-18209d0d-9e0a-42cb-87a1-11bef80f1fd6 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80377) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 10:19:31.346027 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18209d0d-9e0a-42cb-87a1-11bef80f1fd6 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:19:31.347595 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18209d0d-9e0a-42cb-87a1-11bef80f1fd6 service neutron] RBAC: Authorization granted {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:19:31.358369 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: [pid: 80377|app: 0|req: 1224/2433] 10.222.0.82 () {60 vars in 1365 bytes} [Tue Aug 2 10:19:31 2022] GET /identity/v3/auth/tokens => generated 3325 bytes in 118 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 02 10:19:32.027206 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-675af185-30a7-4d05-9092-06f87a4cef7b None None] REQUEST_METHOD: `POST` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:19:32.027733 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-675af185-30a7-4d05-9092-06f87a4cef7b None None] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:19:32.027883 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-675af185-30a7-4d05-9092-06f87a4cef7b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:19:32.047839 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.auth.core [None req-675af185-30a7-4d05-9092-06f87a4cef7b None None] MFA Rules not processed for user `99616b362eac4b7dba445dafd3ca1426`. Rule list: `[]` (Enabled: `True`). {{(pid=80378) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 02 10:19:32.049052 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.fernet_utils [None req-675af185-30a7-4d05-9092-06f87a4cef7b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80378) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 10:19:32.111007 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 1210/2434] 10.222.0.82 () {58 vars in 958 bytes} [Tue Aug 2 10:19:32 2022] POST /identity/v3/auth/tokens => generated 3270 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 02 10:19:41.353208 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG dbcounter [-] [80377] Writing DB stats keystone:SELECT=45 {{(pid=80377) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 10:19:42.099126 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG dbcounter [-] [80378] Writing DB stats keystone:SELECT=89,keystone:DELETE=5,keystone:INSERT=1 {{(pid=80378) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 10:19:58.144444 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f702b04-a8d7-4937-a605-e011362f5707 None None] REQUEST_METHOD: `POST` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:19:58.144969 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f702b04-a8d7-4937-a605-e011362f5707 None None] SCRIPT_NAME: `/identity` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:19:58.144969 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f702b04-a8d7-4937-a605-e011362f5707 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:19:58.164656 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.auth.core [None req-8f702b04-a8d7-4937-a605-e011362f5707 None None] MFA Rules not processed for user `bd8dd72086db48b8aaac2ed0058872df`. Rule list: `[]` (Enabled: `True`). {{(pid=80377) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 02 10:19:58.165978 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.fernet_utils [None req-8f702b04-a8d7-4937-a605-e011362f5707 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80377) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 10:19:58.229203 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: [pid: 80377|app: 0|req: 1225/2435] 10.222.0.82 () {58 vars in 1009 bytes} [Tue Aug 2 10:19:58 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 02 10:19:58.271831 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-675af185-30a7-4d05-9092-06f87a4cef7b None None] Authenticating user token {{(pid=80378) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:19:58.273373 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.fernet_utils [None req-675af185-30a7-4d05-9092-06f87a4cef7b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80378) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 10:19:58.358493 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-675af185-30a7-4d05-9092-06f87a4cef7b None None] Validating token access rules against request {{(pid=80378) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:19:58.364577 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a388e546-0ab7-4037-ba60-24925f05f439 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bef1b2e714cb45c4a51eb6d3e15aad6d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f1b38ebb804c41c69772c998aae72c41', 'project_domain_id': 'default', 'roles': ['admin', 'service', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80378) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:19:58.365206 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a388e546-0ab7-4037-ba60-24925f05f439 service swift] REQUEST_METHOD: `GET` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:19:58.365503 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a388e546-0ab7-4037-ba60-24925f05f439 service swift] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:19:58.365767 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a388e546-0ab7-4037-ba60-24925f05f439 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:19:58.374569 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a388e546-0ab7-4037-ba60-24925f05f439 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:19:58.376194 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a388e546-0ab7-4037-ba60-24925f05f439 service swift] RBAC: Authorization granted {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:19:58.384737 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 1211/2436] 10.222.0.82 () {60 vars in 1384 bytes} [Tue Aug 2 10:19:58 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 113 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Aug 02 10:20:08.223672 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG dbcounter [-] [80377] Writing DB stats keystone:SELECT=25 {{(pid=80377) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 10:20:08.381639 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG dbcounter [-] [80378] Writing DB stats keystone:SELECT=32 {{(pid=80378) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 10:20:15.539474 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e419905-1c38-4bc2-988d-1ad4224f119b None None] REQUEST_METHOD: `POST` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:20:15.540130 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e419905-1c38-4bc2-988d-1ad4224f119b None None] SCRIPT_NAME: `/identity` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:20:15.540130 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e419905-1c38-4bc2-988d-1ad4224f119b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:20:15.559173 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.auth.core [None req-3e419905-1c38-4bc2-988d-1ad4224f119b None None] MFA Rules not processed for user `bd8dd72086db48b8aaac2ed0058872df`. Rule list: `[]` (Enabled: `True`). {{(pid=80377) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 02 10:20:15.560375 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.fernet_utils [None req-3e419905-1c38-4bc2-988d-1ad4224f119b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80377) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 10:20:15.567770 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: [pid: 80377|app: 0|req: 1226/2437] 10.222.0.82 () {58 vars in 967 bytes} [Tue Aug 2 10:20:15 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 02 10:20:15.581116 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a388e546-0ab7-4037-ba60-24925f05f439 service swift] Authenticating user token {{(pid=80378) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:20:15.596047 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a388e546-0ab7-4037-ba60-24925f05f439 service swift] Validating token access rules against request {{(pid=80378) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:20:15.604464 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca5e73c5-0a34-48cc-b9a8-2b2d453f8242 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bef1b2e714cb45c4a51eb6d3e15aad6d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f1b38ebb804c41c69772c998aae72c41', 'project_domain_id': 'default', 'roles': ['admin', 'service', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80378) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:20:15.605362 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca5e73c5-0a34-48cc-b9a8-2b2d453f8242 service swift] REQUEST_METHOD: `GET` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:20:15.606005 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca5e73c5-0a34-48cc-b9a8-2b2d453f8242 service swift] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:20:15.606534 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca5e73c5-0a34-48cc-b9a8-2b2d453f8242 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:20:15.617099 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca5e73c5-0a34-48cc-b9a8-2b2d453f8242 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:20:15.618721 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca5e73c5-0a34-48cc-b9a8-2b2d453f8242 service swift] RBAC: Authorization granted {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:20:15.627635 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 1212/2438] 10.222.0.82 () {60 vars in 1384 bytes} [Tue Aug 2 10:20:15 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 47 msecs (HTTP/1.1 200) 6 headers in 379 bytes (2 switches on core 0) Aug 02 10:20:17.554118 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e419905-1c38-4bc2-988d-1ad4224f119b None None] Authenticating user token {{(pid=80377) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:20:17.555542 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.fernet_utils [None req-3e419905-1c38-4bc2-988d-1ad4224f119b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80377) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 10:20:17.622963 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e419905-1c38-4bc2-988d-1ad4224f119b None None] Validating token access rules against request {{(pid=80377) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:20:17.627584 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab237aa9-7393-44da-8f14-8e6c00bc22cf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '891005fa5ff74041b75747d16cc66798', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7047912c5acd4adeba5d87c3bf895e39', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80377) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:20:17.628262 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab237aa9-7393-44da-8f14-8e6c00bc22cf admin admin] REQUEST_METHOD: `DELETE` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:20:17.628569 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab237aa9-7393-44da-8f14-8e6c00bc22cf admin admin] SCRIPT_NAME: `/identity` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:20:17.628877 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab237aa9-7393-44da-8f14-8e6c00bc22cf admin admin] PATH_INFO: `/v3/users/a5f2ec904f214a708a3691da24fb516d` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:20:17.630151 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab237aa9-7393-44da-8f14-8e6c00bc22cf admin admin] RBAC: Authorizing `identity:delete_user(user_id=a5f2ec904f214a708a3691da24fb516d)` {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:20:17.632139 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab237aa9-7393-44da-8f14-8e6c00bc22cf admin admin] RBAC: Authorization granted {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:20:17.681800 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.notifications [None req-ab237aa9-7393-44da-8f14-8e6c00bc22cf admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'a5f2ec904f214a708a3691da24fb516d'} {{(pid=80377) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 10:20:17.689503 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.notifications [None req-ab237aa9-7393-44da-8f14-8e6c00bc22cf admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'a5f2ec904f214a708a3691da24fb516d'} {{(pid=80377) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 10:20:17.697517 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.notifications [None req-ab237aa9-7393-44da-8f14-8e6c00bc22cf admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'a5f2ec904f214a708a3691da24fb516d'} {{(pid=80377) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 10:20:17.698312 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.notifications [None req-ab237aa9-7393-44da-8f14-8e6c00bc22cf admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'a5f2ec904f214a708a3691da24fb516d'} {{(pid=80377) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 10:20:17.713956 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.notifications [None req-ab237aa9-7393-44da-8f14-8e6c00bc22cf admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'a5f2ec904f214a708a3691da24fb516d'} {{(pid=80377) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 10:20:17.720774 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: [pid: 80377|app: 0|req: 1227/2439] 10.222.0.82 () {58 vars in 1224 bytes} [Tue Aug 2 10:20:17 2022] DELETE /identity/v3/users/a5f2ec904f214a708a3691da24fb516d => generated 0 bytes in 167 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 02 10:20:17.915569 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca5e73c5-0a34-48cc-b9a8-2b2d453f8242 service swift] Authenticating user token {{(pid=80378) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:20:17.932032 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca5e73c5-0a34-48cc-b9a8-2b2d453f8242 service swift] Validating token access rules against request {{(pid=80378) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:20:17.992880 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e3d31b2-392e-46c4-b624-9d565f936ebe admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '891005fa5ff74041b75747d16cc66798', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7047912c5acd4adeba5d87c3bf895e39', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80378) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:20:17.993574 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e3d31b2-392e-46c4-b624-9d565f936ebe admin admin] REQUEST_METHOD: `DELETE` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:20:17.993839 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e3d31b2-392e-46c4-b624-9d565f936ebe admin admin] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:20:17.994127 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e3d31b2-392e-46c4-b624-9d565f936ebe admin admin] PATH_INFO: `/v3/projects/b83d3a9d900c4e9383fa88fd1475c029` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:20:17.995303 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e3d31b2-392e-46c4-b624-9d565f936ebe admin admin] RBAC: Authorizing `identity:delete_project(project_id=b83d3a9d900c4e9383fa88fd1475c029)` {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:20:17.997225 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e3d31b2-392e-46c4-b624-9d565f936ebe admin admin] RBAC: Authorization granted {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:20:18.045577 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.notifications [None req-1e3d31b2-392e-46c4-b624-9d565f936ebe admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'b83d3a9d900c4e9383fa88fd1475c029', 'request_id': 'req-1e3d31b2-392e-46c4-b624-9d565f936ebe'} {{(pid=80378) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 10:20:18.052308 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.notifications [None req-1e3d31b2-392e-46c4-b624-9d565f936ebe admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'b83d3a9d900c4e9383fa88fd1475c029', 'request_id': 'req-1e3d31b2-392e-46c4-b624-9d565f936ebe'} {{(pid=80378) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 10:20:18.058397 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.notifications [None req-1e3d31b2-392e-46c4-b624-9d565f936ebe admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'b83d3a9d900c4e9383fa88fd1475c029', 'request_id': 'req-1e3d31b2-392e-46c4-b624-9d565f936ebe'} {{(pid=80378) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 10:20:18.072697 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.notifications [None req-1e3d31b2-392e-46c4-b624-9d565f936ebe admin admin] The token cache is being invalidate because project b83d3a9d900c4e9383fa88fd1475c029 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80378) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 02 10:20:18.073069 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.notifications [None req-1e3d31b2-392e-46c4-b624-9d565f936ebe admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80378) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 10:20:18.073917 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.notifications [None req-1e3d31b2-392e-46c4-b624-9d565f936ebe admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80378) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 10:20:18.075505 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 1213/2440] 10.222.0.82 () {58 vars in 1233 bytes} [Tue Aug 2 10:20:17 2022] DELETE /identity/v3/projects/b83d3a9d900c4e9383fa88fd1475c029 => generated 0 bytes in 160 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 02 10:20:22.466117 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70b59871-4366-4e2a-931d-720b68e29919 None None] REQUEST_METHOD: `POST` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:20:22.466612 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70b59871-4366-4e2a-931d-720b68e29919 None None] SCRIPT_NAME: `/identity` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:20:22.466717 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70b59871-4366-4e2a-931d-720b68e29919 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:20:22.489357 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.auth.core [None req-70b59871-4366-4e2a-931d-720b68e29919 None None] MFA Rules not processed for user `bd8dd72086db48b8aaac2ed0058872df`. Rule list: `[]` (Enabled: `True`). {{(pid=80377) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 02 10:20:22.490614 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.fernet_utils [None req-70b59871-4366-4e2a-931d-720b68e29919 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80377) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 10:20:22.552780 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: [pid: 80377|app: 0|req: 1228/2441] 10.222.0.82 () {58 vars in 1009 bytes} [Tue Aug 2 10:20:22 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 02 10:20:22.565361 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e3d31b2-392e-46c4-b624-9d565f936ebe admin admin] Authenticating user token {{(pid=80378) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:20:22.566785 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.fernet_utils [None req-1e3d31b2-392e-46c4-b624-9d565f936ebe admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80378) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 10:20:22.636997 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e3d31b2-392e-46c4-b624-9d565f936ebe admin admin] Validating token access rules against request {{(pid=80378) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:20:22.642136 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f640cad-dfbe-475a-803b-b7d588656ae6 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bef1b2e714cb45c4a51eb6d3e15aad6d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f1b38ebb804c41c69772c998aae72c41', 'project_domain_id': 'default', 'roles': ['admin', 'service', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80378) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:20:22.643025 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f640cad-dfbe-475a-803b-b7d588656ae6 service swift] REQUEST_METHOD: `GET` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:20:22.644080 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f640cad-dfbe-475a-803b-b7d588656ae6 service swift] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:20:22.644533 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f640cad-dfbe-475a-803b-b7d588656ae6 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:20:22.654718 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f640cad-dfbe-475a-803b-b7d588656ae6 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:20:22.656348 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f640cad-dfbe-475a-803b-b7d588656ae6 service swift] RBAC: Authorization granted {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:20:22.664951 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 1214/2442] 10.222.0.82 () {60 vars in 1384 bytes} [Tue Aug 2 10:20:22 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 100 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Aug 02 10:20:32.544368 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG dbcounter [-] [80377] Writing DB stats keystone:SELECT=81,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80377) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 10:20:32.661711 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG dbcounter [-] [80378] Writing DB stats keystone:SELECT=89,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80378) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 10:20:37.791370 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f80b115-af3d-44bb-a413-e3ab1d6f63dc None None] REQUEST_METHOD: `GET` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:20:37.791937 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f80b115-af3d-44bb-a413-e3ab1d6f63dc None None] SCRIPT_NAME: `/identity` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:20:37.791937 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f80b115-af3d-44bb-a413-e3ab1d6f63dc None None] PATH_INFO: `/` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:20:37.793319 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: [pid: 80377|app: 0|req: 1229/2443] 10.222.0.82 () {54 vars in 921 bytes} [Tue Aug 2 10:20:37 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 02 10:20:37.799320 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-434febc2-ba71-4cd2-b8ea-a287b930a03b None None] REQUEST_METHOD: `POST` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:20:37.799779 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-434febc2-ba71-4cd2-b8ea-a287b930a03b None None] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:20:37.800613 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-434febc2-ba71-4cd2-b8ea-a287b930a03b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:20:37.821695 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.auth.core [None req-434febc2-ba71-4cd2-b8ea-a287b930a03b None None] MFA Rules not processed for user `88af518196674fe593ad88d30111fb7d`. Rule list: `[]` (Enabled: `True`). {{(pid=80378) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 02 10:20:37.823023 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.fernet_utils [None req-434febc2-ba71-4cd2-b8ea-a287b930a03b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80378) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 10:20:37.897142 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 1215/2444] 10.222.0.82 () {58 vars in 1018 bytes} [Tue Aug 2 10:20:37 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 02 10:20:37.908888 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f80b115-af3d-44bb-a413-e3ab1d6f63dc None None] Authenticating user token {{(pid=80377) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:20:37.910666 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.fernet_utils [None req-1f80b115-af3d-44bb-a413-e3ab1d6f63dc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80377) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 10:20:37.968359 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f80b115-af3d-44bb-a413-e3ab1d6f63dc None None] Validating token access rules against request {{(pid=80377) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:20:37.971902 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e53be6e9-a74a-4012-b50f-053531dfd56c service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa964af53a2b4fb9ace6afb544d54d40', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f1b38ebb804c41c69772c998aae72c41', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80377) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:20:37.972519 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e53be6e9-a74a-4012-b50f-053531dfd56c service glance] REQUEST_METHOD: `GET` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:20:37.972803 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e53be6e9-a74a-4012-b50f-053531dfd56c service glance] SCRIPT_NAME: `/identity` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:20:37.973080 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e53be6e9-a74a-4012-b50f-053531dfd56c service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:20:37.983007 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e53be6e9-a74a-4012-b50f-053531dfd56c service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:20:37.984465 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e53be6e9-a74a-4012-b50f-053531dfd56c service glance] RBAC: Authorization granted {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:20:37.995478 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: [pid: 80377|app: 0|req: 1230/2445] 10.222.0.82 () {60 vars in 1365 bytes} [Tue Aug 2 10:20:37 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 87 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 02 10:20:38.064108 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c032c2c-a0d2-4c49-bee0-c2479f0f436c None None] REQUEST_METHOD: `GET` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:20:38.064386 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c032c2c-a0d2-4c49-bee0-c2479f0f436c None None] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:20:38.064663 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c032c2c-a0d2-4c49-bee0-c2479f0f436c None None] PATH_INFO: `/` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:20:38.066065 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 1216/2446] 10.222.0.82 () {54 vars in 921 bytes} [Tue Aug 2 10:20:38 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 02 10:20:38.070923 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bc3aac2-d5c9-4ef3-a1d4-be00a4645db0 None None] REQUEST_METHOD: `POST` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:20:38.071215 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bc3aac2-d5c9-4ef3-a1d4-be00a4645db0 None None] SCRIPT_NAME: `/identity` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:20:38.071496 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bc3aac2-d5c9-4ef3-a1d4-be00a4645db0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:20:38.090903 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.auth.core [None req-1bc3aac2-d5c9-4ef3-a1d4-be00a4645db0 None None] MFA Rules not processed for user `33b87170f0634b67bce60750847b5cbe`. Rule list: `[]` (Enabled: `True`). {{(pid=80377) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 02 10:20:38.092113 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.fernet_utils [None req-1bc3aac2-d5c9-4ef3-a1d4-be00a4645db0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80377) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 10:20:38.142833 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: [pid: 80377|app: 0|req: 1231/2447] 10.222.0.82 () {58 vars in 1018 bytes} [Tue Aug 2 10:20:38 2022] POST /identity/v3/auth/tokens => generated 3121 bytes in 73 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 02 10:20:38.155194 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c032c2c-a0d2-4c49-bee0-c2479f0f436c None None] Authenticating user token {{(pid=80378) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:20:38.156888 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.fernet_utils [None req-5c032c2c-a0d2-4c49-bee0-c2479f0f436c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80378) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 10:20:38.171002 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c032c2c-a0d2-4c49-bee0-c2479f0f436c None None] Validating token access rules against request {{(pid=80378) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:20:38.177322 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4adb00d8-7605-40b2-b2a1-97397e6958ee service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '33b87170f0634b67bce60750847b5cbe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f1b38ebb804c41c69772c998aae72c41', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80378) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:20:38.177977 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4adb00d8-7605-40b2-b2a1-97397e6958ee service neutron] REQUEST_METHOD: `GET` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:20:38.178259 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4adb00d8-7605-40b2-b2a1-97397e6958ee service neutron] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:20:38.178536 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4adb00d8-7605-40b2-b2a1-97397e6958ee service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:20:38.186530 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4adb00d8-7605-40b2-b2a1-97397e6958ee service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:20:38.188636 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4adb00d8-7605-40b2-b2a1-97397e6958ee service neutron] RBAC: Authorization granted {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:20:38.196906 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 1217/2448] 10.222.0.82 () {60 vars in 1365 bytes} [Tue Aug 2 10:20:38 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 42 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 02 10:20:38.280642 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-234efeb0-dbcb-4bf4-8dbe-2b339407b3a6 None None] REQUEST_METHOD: `GET` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:20:38.281020 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-234efeb0-dbcb-4bf4-8dbe-2b339407b3a6 None None] SCRIPT_NAME: `/identity` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:20:38.281298 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-234efeb0-dbcb-4bf4-8dbe-2b339407b3a6 None None] PATH_INFO: `/` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:20:38.282708 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: [pid: 80377|app: 0|req: 1232/2449] 10.222.0.82 () {54 vars in 921 bytes} [Tue Aug 2 10:20:38 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 02 10:20:38.287959 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b32666bb-e675-4917-bc5c-938aebf6fe82 None None] REQUEST_METHOD: `POST` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:20:38.288255 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b32666bb-e675-4917-bc5c-938aebf6fe82 None None] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:20:38.288535 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b32666bb-e675-4917-bc5c-938aebf6fe82 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:20:38.308759 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.auth.core [None req-b32666bb-e675-4917-bc5c-938aebf6fe82 None None] MFA Rules not processed for user `88af518196674fe593ad88d30111fb7d`. Rule list: `[]` (Enabled: `True`). {{(pid=80378) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 02 10:20:38.310032 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.fernet_utils [None req-b32666bb-e675-4917-bc5c-938aebf6fe82 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80378) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 10:20:38.320058 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 1218/2450] 10.222.0.82 () {58 vars in 1018 bytes} [Tue Aug 2 10:20:38 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 02 10:20:38.330439 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-234efeb0-dbcb-4bf4-8dbe-2b339407b3a6 None None] Authenticating user token {{(pid=80377) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:20:38.342106 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-234efeb0-dbcb-4bf4-8dbe-2b339407b3a6 None None] Validating token access rules against request {{(pid=80377) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:20:38.345702 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0acab81-6d98-4f43-adf5-b2b59ccbeeb3 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '33b87170f0634b67bce60750847b5cbe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f1b38ebb804c41c69772c998aae72c41', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80377) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:20:38.346364 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0acab81-6d98-4f43-adf5-b2b59ccbeeb3 service neutron] REQUEST_METHOD: `GET` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:20:38.346649 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0acab81-6d98-4f43-adf5-b2b59ccbeeb3 service neutron] SCRIPT_NAME: `/identity` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:20:38.346952 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0acab81-6d98-4f43-adf5-b2b59ccbeeb3 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:20:38.356870 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0acab81-6d98-4f43-adf5-b2b59ccbeeb3 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:20:38.358338 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0acab81-6d98-4f43-adf5-b2b59ccbeeb3 service neutron] RBAC: Authorization granted {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:20:38.369315 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: [pid: 80377|app: 0|req: 1233/2451] 10.222.0.82 () {60 vars in 1365 bytes} [Tue Aug 2 10:20:38 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 39 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 02 10:20:40.959470 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc31f32d-9cc8-4600-bf12-e846284dcf9d None None] REQUEST_METHOD: `POST` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:20:40.960282 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc31f32d-9cc8-4600-bf12-e846284dcf9d None None] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:20:40.960559 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc31f32d-9cc8-4600-bf12-e846284dcf9d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:20:40.981820 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.auth.core [None req-fc31f32d-9cc8-4600-bf12-e846284dcf9d None None] MFA Rules not processed for user `bd8dd72086db48b8aaac2ed0058872df`. Rule list: `[]` (Enabled: `True`). {{(pid=80378) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 02 10:20:40.983019 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.fernet_utils [None req-fc31f32d-9cc8-4600-bf12-e846284dcf9d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80378) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 10:20:40.990664 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 1219/2452] 10.222.0.82 () {58 vars in 1009 bytes} [Tue Aug 2 10:20:40 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 02 10:20:41.002521 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0acab81-6d98-4f43-adf5-b2b59ccbeeb3 service neutron] Authenticating user token {{(pid=80377) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:20:41.013555 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0acab81-6d98-4f43-adf5-b2b59ccbeeb3 service neutron] Validating token access rules against request {{(pid=80377) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:20:41.018447 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-174b73f9-71c7-4e57-a502-3b981d55047f service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bef1b2e714cb45c4a51eb6d3e15aad6d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f1b38ebb804c41c69772c998aae72c41', 'project_domain_id': 'default', 'roles': ['admin', 'service', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80377) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:20:41.019043 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-174b73f9-71c7-4e57-a502-3b981d55047f service swift] REQUEST_METHOD: `GET` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:20:41.019367 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-174b73f9-71c7-4e57-a502-3b981d55047f service swift] SCRIPT_NAME: `/identity` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:20:41.019734 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-174b73f9-71c7-4e57-a502-3b981d55047f service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:20:41.027571 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-174b73f9-71c7-4e57-a502-3b981d55047f service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:20:41.029130 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-174b73f9-71c7-4e57-a502-3b981d55047f service swift] RBAC: Authorization granted {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:20:41.037904 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: [pid: 80377|app: 0|req: 1234/2453] 10.222.0.82 () {60 vars in 1384 bytes} [Tue Aug 2 10:20:41 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 36 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Aug 02 10:20:50.976427 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG dbcounter [-] [80378] Writing DB stats keystone:SELECT=43 {{(pid=80378) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 10:20:51.034228 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG dbcounter [-] [80377] Writing DB stats keystone:SELECT=61 {{(pid=80377) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 10:21:01.640010 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc31f32d-9cc8-4600-bf12-e846284dcf9d None None] Authenticating user token {{(pid=80378) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:21:01.641482 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.fernet_utils [None req-fc31f32d-9cc8-4600-bf12-e846284dcf9d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80378) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 10:21:01.653403 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc31f32d-9cc8-4600-bf12-e846284dcf9d None None] Validating token access rules against request {{(pid=80378) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:21:01.656871 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32924a2c-3e98-4ba8-983b-21a5790bf6bc service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '33b87170f0634b67bce60750847b5cbe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f1b38ebb804c41c69772c998aae72c41', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80378) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:21:01.657473 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32924a2c-3e98-4ba8-983b-21a5790bf6bc service neutron] REQUEST_METHOD: `GET` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:21:01.657750 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32924a2c-3e98-4ba8-983b-21a5790bf6bc service neutron] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:21:01.658050 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32924a2c-3e98-4ba8-983b-21a5790bf6bc service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:21:01.659716 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.fernet_utils [None req-32924a2c-3e98-4ba8-983b-21a5790bf6bc service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80378) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 10:21:01.673019 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32924a2c-3e98-4ba8-983b-21a5790bf6bc service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:21:01.674471 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32924a2c-3e98-4ba8-983b-21a5790bf6bc service neutron] RBAC: Authorization granted {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:21:01.685002 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 1220/2454] 10.222.0.82 () {60 vars in 1365 bytes} [Tue Aug 2 10:21:01 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 45 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 02 10:21:01.929368 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-174b73f9-71c7-4e57-a502-3b981d55047f service swift] Authenticating user token {{(pid=80377) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:21:01.938874 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-174b73f9-71c7-4e57-a502-3b981d55047f service swift] Validating token access rules against request {{(pid=80377) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:21:01.942431 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46b9e229-71fa-4980-819d-9c47993dcb6e service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '33b87170f0634b67bce60750847b5cbe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f1b38ebb804c41c69772c998aae72c41', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80377) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:21:01.943033 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46b9e229-71fa-4980-819d-9c47993dcb6e service neutron] REQUEST_METHOD: `GET` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:21:01.943319 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46b9e229-71fa-4980-819d-9c47993dcb6e service neutron] SCRIPT_NAME: `/identity` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:21:01.943597 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46b9e229-71fa-4980-819d-9c47993dcb6e service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:21:01.945363 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.fernet_utils [None req-46b9e229-71fa-4980-819d-9c47993dcb6e service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80377) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 10:21:01.959214 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46b9e229-71fa-4980-819d-9c47993dcb6e service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:21:01.960668 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46b9e229-71fa-4980-819d-9c47993dcb6e service neutron] RBAC: Authorization granted {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:21:01.971391 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: [pid: 80377|app: 0|req: 1235/2455] 10.222.0.82 () {60 vars in 1365 bytes} [Tue Aug 2 10:21:01 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 43 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 02 10:21:11.340594 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-506d9067-38d4-49b5-a61b-32def98ae17d None None] REQUEST_METHOD: `POST` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:21:11.341089 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-506d9067-38d4-49b5-a61b-32def98ae17d None None] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:21:11.341187 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-506d9067-38d4-49b5-a61b-32def98ae17d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:21:11.360510 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.auth.core [None req-506d9067-38d4-49b5-a61b-32def98ae17d None None] MFA Rules not processed for user `bd8dd72086db48b8aaac2ed0058872df`. Rule list: `[]` (Enabled: `True`). {{(pid=80378) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 02 10:21:11.361730 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.fernet_utils [None req-506d9067-38d4-49b5-a61b-32def98ae17d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80378) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 10:21:11.369742 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 1221/2456] 10.222.0.82 () {58 vars in 967 bytes} [Tue Aug 2 10:21:11 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 02 10:21:11.392386 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46b9e229-71fa-4980-819d-9c47993dcb6e service neutron] Authenticating user token {{(pid=80377) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:21:11.403905 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46b9e229-71fa-4980-819d-9c47993dcb6e service neutron] Validating token access rules against request {{(pid=80377) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:21:11.408932 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-304337e9-64db-4099-9d87-3ba3917990d5 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bef1b2e714cb45c4a51eb6d3e15aad6d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f1b38ebb804c41c69772c998aae72c41', 'project_domain_id': 'default', 'roles': ['admin', 'service', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80377) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:21:11.409544 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-304337e9-64db-4099-9d87-3ba3917990d5 service swift] REQUEST_METHOD: `GET` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:21:11.409861 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-304337e9-64db-4099-9d87-3ba3917990d5 service swift] SCRIPT_NAME: `/identity` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:21:11.410145 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-304337e9-64db-4099-9d87-3ba3917990d5 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:21:11.418656 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-304337e9-64db-4099-9d87-3ba3917990d5 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:21:11.420232 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-304337e9-64db-4099-9d87-3ba3917990d5 service swift] RBAC: Authorization granted {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:21:11.429150 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: [pid: 80377|app: 0|req: 1236/2457] 10.222.0.82 () {60 vars in 1384 bytes} [Tue Aug 2 10:21:11 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 37 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Aug 02 10:21:21.355675 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG dbcounter [-] [80378] Writing DB stats keystone:SELECT=9 {{(pid=80378) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 10:21:21.425650 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG dbcounter [-] [80377] Writing DB stats keystone:SELECT=12 {{(pid=80377) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 10:21:41.564076 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-506d9067-38d4-49b5-a61b-32def98ae17d None None] Authenticating user token {{(pid=80378) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:21:41.566068 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.fernet_utils [None req-506d9067-38d4-49b5-a61b-32def98ae17d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80378) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 10:21:41.582102 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-506d9067-38d4-49b5-a61b-32def98ae17d None None] Validating token access rules against request {{(pid=80378) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:21:41.587722 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d879843-407f-45ed-ad58-0836c81e6470 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88af518196674fe593ad88d30111fb7d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f1b38ebb804c41c69772c998aae72c41', 'project_domain_id': 'default', 'roles': ['admin', 'ResellerAdmin', 'reader', 'service', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80378) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:21:41.588364 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d879843-407f-45ed-ad58-0836c81e6470 service nova] REQUEST_METHOD: `GET` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:21:41.588648 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d879843-407f-45ed-ad58-0836c81e6470 service nova] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:21:41.588925 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d879843-407f-45ed-ad58-0836c81e6470 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:21:41.590623 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.fernet_utils [None req-0d879843-407f-45ed-ad58-0836c81e6470 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80378) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 10:21:41.648354 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d879843-407f-45ed-ad58-0836c81e6470 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:21:41.649996 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d879843-407f-45ed-ad58-0836c81e6470 service nova] RBAC: Authorization granted {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:21:41.660437 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 1222/2458] 10.222.0.82 () {60 vars in 1365 bytes} [Tue Aug 2 10:21:41 2022] GET /identity/v3/auth/tokens => generated 3252 bytes in 97 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 02 10:21:44.660943 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-304337e9-64db-4099-9d87-3ba3917990d5 service swift] Authenticating user token {{(pid=80377) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:21:44.673518 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-304337e9-64db-4099-9d87-3ba3917990d5 service swift] Validating token access rules against request {{(pid=80377) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:21:44.679016 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4377e6f0-e34f-4519-932f-f28f35df416b service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88af518196674fe593ad88d30111fb7d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f1b38ebb804c41c69772c998aae72c41', 'project_domain_id': 'default', 'roles': ['admin', 'ResellerAdmin', 'reader', 'service', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80377) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:21:44.679624 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4377e6f0-e34f-4519-932f-f28f35df416b service nova] REQUEST_METHOD: `GET` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:21:44.679907 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4377e6f0-e34f-4519-932f-f28f35df416b service nova] SCRIPT_NAME: `/identity` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:21:44.680185 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4377e6f0-e34f-4519-932f-f28f35df416b service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:21:44.681978 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.fernet_utils [None req-4377e6f0-e34f-4519-932f-f28f35df416b service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80377) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 10:21:44.739548 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4377e6f0-e34f-4519-932f-f28f35df416b service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:21:44.741128 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4377e6f0-e34f-4519-932f-f28f35df416b service nova] RBAC: Authorization granted {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:21:44.750779 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: [pid: 80377|app: 0|req: 1237/2459] 10.222.0.82 () {60 vars in 1365 bytes} [Tue Aug 2 10:21:44 2022] GET /identity/v3/auth/tokens => generated 3270 bytes in 90 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 02 10:21:51.655822 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG dbcounter [-] [80378] Writing DB stats keystone:SELECT=28 {{(pid=80378) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 10:21:52.775597 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d879843-407f-45ed-ad58-0836c81e6470 service nova] Authenticating user token {{(pid=80378) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:21:52.785067 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d879843-407f-45ed-ad58-0836c81e6470 service nova] Validating token access rules against request {{(pid=80378) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:21:52.788947 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6e4f882-18d0-4a57-b696-5345b05c0160 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '33b87170f0634b67bce60750847b5cbe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f1b38ebb804c41c69772c998aae72c41', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80378) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:21:52.790575 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6e4f882-18d0-4a57-b696-5345b05c0160 service neutron] REQUEST_METHOD: `GET` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:21:52.790575 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6e4f882-18d0-4a57-b696-5345b05c0160 service neutron] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:21:52.790575 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6e4f882-18d0-4a57-b696-5345b05c0160 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:21:52.793089 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.fernet_utils [None req-b6e4f882-18d0-4a57-b696-5345b05c0160 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80378) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 10:21:52.807265 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6e4f882-18d0-4a57-b696-5345b05c0160 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:21:52.809235 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6e4f882-18d0-4a57-b696-5345b05c0160 service neutron] RBAC: Authorization granted {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:21:52.819136 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 1223/2460] 10.222.0.82 () {60 vars in 1365 bytes} [Tue Aug 2 10:21:52 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 44 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 02 10:21:54.746025 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG dbcounter [-] [80377] Writing DB stats keystone:SELECT=28 {{(pid=80377) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 10:22:02.814618 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG dbcounter [-] [80378] Writing DB stats keystone:SELECT=6 {{(pid=80378) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 10:22:08.312926 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12a86250-c3aa-4664-88e5-7921021f3c42 None None] REQUEST_METHOD: `POST` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:22:08.313445 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12a86250-c3aa-4664-88e5-7921021f3c42 None None] SCRIPT_NAME: `/identity` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:22:08.313543 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12a86250-c3aa-4664-88e5-7921021f3c42 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:22:08.332573 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.auth.core [None req-12a86250-c3aa-4664-88e5-7921021f3c42 None None] MFA Rules not processed for user `99616b362eac4b7dba445dafd3ca1426`. Rule list: `[]` (Enabled: `True`). {{(pid=80377) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 02 10:22:08.333760 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.fernet_utils [None req-12a86250-c3aa-4664-88e5-7921021f3c42 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80377) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 10:22:08.341084 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: [pid: 80377|app: 0|req: 1238/2461] 10.222.0.82 () {58 vars in 958 bytes} [Tue Aug 2 10:22:08 2022] POST /identity/v3/auth/tokens => generated 3270 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 02 10:22:14.628587 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6e4f882-18d0-4a57-b696-5345b05c0160 service neutron] Authenticating user token {{(pid=80378) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:22:14.646411 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6e4f882-18d0-4a57-b696-5345b05c0160 service neutron] Validating token access rules against request {{(pid=80378) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:22:14.655061 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b15fab4c-33bf-4c67-8001-1003572ee01d service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88af518196674fe593ad88d30111fb7d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f1b38ebb804c41c69772c998aae72c41', 'project_domain_id': 'default', 'roles': ['admin', 'ResellerAdmin', 'reader', 'service', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80378) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:22:14.655747 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b15fab4c-33bf-4c67-8001-1003572ee01d service nova] REQUEST_METHOD: `GET` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:22:14.656065 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b15fab4c-33bf-4c67-8001-1003572ee01d service nova] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:22:14.656638 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b15fab4c-33bf-4c67-8001-1003572ee01d service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:22:14.658878 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.fernet_utils [None req-b15fab4c-33bf-4c67-8001-1003572ee01d service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80378) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 10:22:14.676416 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b15fab4c-33bf-4c67-8001-1003572ee01d service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:22:14.678057 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b15fab4c-33bf-4c67-8001-1003572ee01d service nova] RBAC: Authorization granted {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:22:14.690187 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 1224/2462] 10.222.0.82 () {60 vars in 1365 bytes} [Tue Aug 2 10:22:14 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 62 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 02 10:22:18.327139 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG dbcounter [-] [80377] Writing DB stats keystone:SELECT=3 {{(pid=80377) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 10:22:19.755265 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12a86250-c3aa-4664-88e5-7921021f3c42 None None] Authenticating user token {{(pid=80377) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:22:19.768481 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12a86250-c3aa-4664-88e5-7921021f3c42 None None] Validating token access rules against request {{(pid=80377) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:22:19.774585 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4326c400-7194-4c17-a3df-70bc6b65c65f service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88af518196674fe593ad88d30111fb7d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f1b38ebb804c41c69772c998aae72c41', 'project_domain_id': 'default', 'roles': ['admin', 'ResellerAdmin', 'reader', 'service', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80377) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:22:19.775242 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4326c400-7194-4c17-a3df-70bc6b65c65f service nova] REQUEST_METHOD: `GET` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:22:19.775525 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4326c400-7194-4c17-a3df-70bc6b65c65f service nova] SCRIPT_NAME: `/identity` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:22:19.775796 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4326c400-7194-4c17-a3df-70bc6b65c65f service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:22:19.777521 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.fernet_utils [None req-4326c400-7194-4c17-a3df-70bc6b65c65f service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80377) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 10:22:19.837217 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4326c400-7194-4c17-a3df-70bc6b65c65f service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:22:19.838895 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4326c400-7194-4c17-a3df-70bc6b65c65f service nova] RBAC: Authorization granted {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:22:19.848894 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: [pid: 80377|app: 0|req: 1239/2463] 10.222.0.82 () {60 vars in 1365 bytes} [Tue Aug 2 10:22:19 2022] GET /identity/v3/auth/tokens => generated 3266 bytes in 94 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 02 10:22:24.685526 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG dbcounter [-] [80378] Writing DB stats keystone:SELECT=6 {{(pid=80378) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 10:22:28.550605 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf7c2f0f-94e6-4b9a-8807-77969154b78d None None] REQUEST_METHOD: `POST` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:22:28.551289 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf7c2f0f-94e6-4b9a-8807-77969154b78d None None] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:22:28.551289 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf7c2f0f-94e6-4b9a-8807-77969154b78d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:22:28.573167 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.auth.core [None req-bf7c2f0f-94e6-4b9a-8807-77969154b78d None None] MFA Rules not processed for user `b6890a8127e04092920cb1d31ab56d91`. Rule list: `[]` (Enabled: `True`). {{(pid=80378) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 02 10:22:28.574448 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.fernet_utils [None req-bf7c2f0f-94e6-4b9a-8807-77969154b78d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80378) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 10:22:28.581393 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 1225/2464] 10.222.0.82 () {58 vars in 958 bytes} [Tue Aug 2 10:22:28 2022] POST /identity/v3/auth/tokens => generated 3266 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 02 10:22:29.013844 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4326c400-7194-4c17-a3df-70bc6b65c65f service nova] Authenticating user token {{(pid=80377) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:22:29.025132 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4326c400-7194-4c17-a3df-70bc6b65c65f service nova] Validating token access rules against request {{(pid=80377) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:22:29.029476 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6e87785-6741-4a72-b31d-c794f50dd4a3 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '33b87170f0634b67bce60750847b5cbe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f1b38ebb804c41c69772c998aae72c41', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80377) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:22:29.030382 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6e87785-6741-4a72-b31d-c794f50dd4a3 service neutron] REQUEST_METHOD: `GET` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:22:29.030729 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6e87785-6741-4a72-b31d-c794f50dd4a3 service neutron] SCRIPT_NAME: `/identity` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:22:29.031964 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6e87785-6741-4a72-b31d-c794f50dd4a3 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:22:29.034247 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.fernet_utils [None req-d6e87785-6741-4a72-b31d-c794f50dd4a3 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80377) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 10:22:29.046017 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6e87785-6741-4a72-b31d-c794f50dd4a3 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:22:29.047710 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6e87785-6741-4a72-b31d-c794f50dd4a3 service neutron] RBAC: Authorization granted {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:22:29.069437 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: [pid: 80377|app: 0|req: 1240/2465] 10.222.0.82 () {60 vars in 1365 bytes} [Tue Aug 2 10:22:29 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 56 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 02 10:22:29.527861 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf7c2f0f-94e6-4b9a-8807-77969154b78d None None] Authenticating user token {{(pid=80378) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:22:29.529478 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.fernet_utils [None req-bf7c2f0f-94e6-4b9a-8807-77969154b78d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80378) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 10:22:29.604406 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf7c2f0f-94e6-4b9a-8807-77969154b78d None None] Validating token access rules against request {{(pid=80378) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:22:29.609426 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e90e1f4-691c-4709-a021-9846f40b3045 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ae6e910e50d74f29bfb89bf18d95a908', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f1b38ebb804c41c69772c998aae72c41', 'project_domain_id': 'default', 'roles': ['admin', 'service', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80378) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:22:29.610072 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e90e1f4-691c-4709-a021-9846f40b3045 service placement] REQUEST_METHOD: `GET` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:22:29.610352 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e90e1f4-691c-4709-a021-9846f40b3045 service placement] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:22:29.610621 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e90e1f4-691c-4709-a021-9846f40b3045 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:22:29.612339 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.fernet_utils [None req-6e90e1f4-691c-4709-a021-9846f40b3045 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80378) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 10:22:29.624444 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e90e1f4-691c-4709-a021-9846f40b3045 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:22:29.626059 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e90e1f4-691c-4709-a021-9846f40b3045 service placement] RBAC: Authorization granted {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:22:29.637540 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 1226/2466] 10.222.0.82 () {60 vars in 1365 bytes} [Tue Aug 2 10:22:29 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 110 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 02 10:22:39.053874 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG dbcounter [-] [80377] Writing DB stats keystone:SELECT=34 {{(pid=80377) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 10:22:39.632472 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG dbcounter [-] [80378] Writing DB stats keystone:SELECT=35 {{(pid=80378) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 10:22:40.759885 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6e87785-6741-4a72-b31d-c794f50dd4a3 service neutron] Authenticating user token {{(pid=80377) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:22:40.761901 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.fernet_utils [None req-d6e87785-6741-4a72-b31d-c794f50dd4a3 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80377) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 10:22:40.779093 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6e87785-6741-4a72-b31d-c794f50dd4a3 service neutron] Validating token access rules against request {{(pid=80377) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:22:40.785404 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-412fc572-e4ee-4aae-b3e4-db0c23ecbd98 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88af518196674fe593ad88d30111fb7d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f1b38ebb804c41c69772c998aae72c41', 'project_domain_id': 'default', 'roles': ['admin', 'ResellerAdmin', 'reader', 'service', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80377) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:22:40.786069 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-412fc572-e4ee-4aae-b3e4-db0c23ecbd98 service nova] REQUEST_METHOD: `GET` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:22:40.786346 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-412fc572-e4ee-4aae-b3e4-db0c23ecbd98 service nova] SCRIPT_NAME: `/identity` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:22:40.786623 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-412fc572-e4ee-4aae-b3e4-db0c23ecbd98 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:22:40.788482 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.fernet_utils [None req-412fc572-e4ee-4aae-b3e4-db0c23ecbd98 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80377) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 10:22:40.801993 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-412fc572-e4ee-4aae-b3e4-db0c23ecbd98 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:22:40.803582 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-412fc572-e4ee-4aae-b3e4-db0c23ecbd98 service nova] RBAC: Authorization granted {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:22:40.816944 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: [pid: 80377|app: 0|req: 1241/2467] 10.222.0.82 () {60 vars in 1365 bytes} [Tue Aug 2 10:22:40 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 57 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 02 10:22:41.237629 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e90e1f4-691c-4709-a021-9846f40b3045 service placement] Authenticating user token {{(pid=80378) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:22:41.250784 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e90e1f4-691c-4709-a021-9846f40b3045 service placement] Validating token access rules against request {{(pid=80378) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:22:41.256238 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a27e64bf-5dc7-475f-86c2-4686d7ecb312 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88af518196674fe593ad88d30111fb7d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f1b38ebb804c41c69772c998aae72c41', 'project_domain_id': 'default', 'roles': ['admin', 'ResellerAdmin', 'reader', 'service', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80378) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:22:41.256849 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a27e64bf-5dc7-475f-86c2-4686d7ecb312 service nova] REQUEST_METHOD: `GET` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:22:41.257157 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a27e64bf-5dc7-475f-86c2-4686d7ecb312 service nova] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:22:41.257437 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a27e64bf-5dc7-475f-86c2-4686d7ecb312 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:22:41.259211 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.fernet_utils [None req-a27e64bf-5dc7-475f-86c2-4686d7ecb312 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80378) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 10:22:41.316344 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a27e64bf-5dc7-475f-86c2-4686d7ecb312 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:22:41.318214 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a27e64bf-5dc7-475f-86c2-4686d7ecb312 service nova] RBAC: Authorization granted {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:22:41.328322 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 1227/2468] 10.222.0.82 () {60 vars in 1365 bytes} [Tue Aug 2 10:22:41 2022] GET /identity/v3/auth/tokens => generated 3262 bytes in 91 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 02 10:22:41.577189 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d537f57-efb1-45b0-b39a-93fc76f1fca4 None None] REQUEST_METHOD: `POST` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:22:41.577485 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d537f57-efb1-45b0-b39a-93fc76f1fca4 None None] SCRIPT_NAME: `/identity` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:22:41.577772 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d537f57-efb1-45b0-b39a-93fc76f1fca4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:22:41.598052 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.auth.core [None req-6d537f57-efb1-45b0-b39a-93fc76f1fca4 None None] MFA Rules not processed for user `bd8dd72086db48b8aaac2ed0058872df`. Rule list: `[]` (Enabled: `True`). {{(pid=80377) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 02 10:22:41.599270 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.fernet_utils [None req-6d537f57-efb1-45b0-b39a-93fc76f1fca4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80377) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 10:22:41.607396 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: [pid: 80377|app: 0|req: 1242/2469] 10.222.0.82 () {58 vars in 967 bytes} [Tue Aug 2 10:22:41 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 02 10:22:41.652471 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a27e64bf-5dc7-475f-86c2-4686d7ecb312 service nova] Authenticating user token {{(pid=80378) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:22:41.665030 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a27e64bf-5dc7-475f-86c2-4686d7ecb312 service nova] Validating token access rules against request {{(pid=80378) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:22:41.670593 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55b11c90-c447-4778-b0ff-74000a341fc8 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bef1b2e714cb45c4a51eb6d3e15aad6d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f1b38ebb804c41c69772c998aae72c41', 'project_domain_id': 'default', 'roles': ['admin', 'service', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80378) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:22:41.671206 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55b11c90-c447-4778-b0ff-74000a341fc8 service swift] REQUEST_METHOD: `GET` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:22:41.671490 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55b11c90-c447-4778-b0ff-74000a341fc8 service swift] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:22:41.671766 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55b11c90-c447-4778-b0ff-74000a341fc8 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:22:41.679910 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55b11c90-c447-4778-b0ff-74000a341fc8 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:22:41.681474 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55b11c90-c447-4778-b0ff-74000a341fc8 service swift] RBAC: Authorization granted {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:22:41.690216 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 1228/2470] 10.222.0.82 () {60 vars in 1384 bytes} [Tue Aug 2 10:22:41 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 38 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Aug 02 10:22:42.022518 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d537f57-efb1-45b0-b39a-93fc76f1fca4 None None] Authenticating user token {{(pid=80377) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:22:42.033140 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d537f57-efb1-45b0-b39a-93fc76f1fca4 None None] Validating token access rules against request {{(pid=80377) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:22:42.036770 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c65f16ba-b893-40c6-a3a6-064cbb6195c8 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '33b87170f0634b67bce60750847b5cbe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f1b38ebb804c41c69772c998aae72c41', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80377) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:22:42.037400 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c65f16ba-b893-40c6-a3a6-064cbb6195c8 service neutron] REQUEST_METHOD: `GET` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:22:42.037693 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c65f16ba-b893-40c6-a3a6-064cbb6195c8 service neutron] SCRIPT_NAME: `/identity` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:22:42.038006 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c65f16ba-b893-40c6-a3a6-064cbb6195c8 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:22:42.039747 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.fernet_utils [None req-c65f16ba-b893-40c6-a3a6-064cbb6195c8 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80377) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 10:22:42.103089 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c65f16ba-b893-40c6-a3a6-064cbb6195c8 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:22:42.104559 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c65f16ba-b893-40c6-a3a6-064cbb6195c8 service neutron] RBAC: Authorization granted {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:22:42.114317 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: [pid: 80377|app: 0|req: 1243/2471] 10.222.0.82 () {60 vars in 1365 bytes} [Tue Aug 2 10:22:42 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 92 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 02 10:22:47.359724 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55b11c90-c447-4778-b0ff-74000a341fc8 service swift] Authenticating user token {{(pid=80378) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:22:47.372096 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55b11c90-c447-4778-b0ff-74000a341fc8 service swift] Validating token access rules against request {{(pid=80378) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:22:47.378195 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-164ebb41-513e-4ee6-8787-bee3cd8fb707 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ae6e910e50d74f29bfb89bf18d95a908', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f1b38ebb804c41c69772c998aae72c41', 'project_domain_id': 'default', 'roles': ['admin', 'service', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80378) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:22:47.379410 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-164ebb41-513e-4ee6-8787-bee3cd8fb707 service placement] REQUEST_METHOD: `GET` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:22:47.380207 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-164ebb41-513e-4ee6-8787-bee3cd8fb707 service placement] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:22:47.380490 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-164ebb41-513e-4ee6-8787-bee3cd8fb707 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:22:47.382964 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.fernet_utils [None req-164ebb41-513e-4ee6-8787-bee3cd8fb707 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80378) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 10:22:47.399828 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-164ebb41-513e-4ee6-8787-bee3cd8fb707 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:22:47.404299 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-164ebb41-513e-4ee6-8787-bee3cd8fb707 service placement] RBAC: Authorization granted {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:22:47.425968 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 1229/2472] 10.222.0.82 () {60 vars in 1365 bytes} [Tue Aug 2 10:22:47 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 66 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 02 10:22:50.701882 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c65f16ba-b893-40c6-a3a6-064cbb6195c8 service neutron] Authenticating user token {{(pid=80377) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:22:50.703426 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.fernet_utils [None req-c65f16ba-b893-40c6-a3a6-064cbb6195c8 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80377) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 10:22:50.716601 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c65f16ba-b893-40c6-a3a6-064cbb6195c8 service neutron] Validating token access rules against request {{(pid=80377) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:22:50.728613 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ff14aa1-141f-48aa-9fc8-c6dfc721f5a1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '891005fa5ff74041b75747d16cc66798', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7047912c5acd4adeba5d87c3bf895e39', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80377) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:22:50.729879 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ff14aa1-141f-48aa-9fc8-c6dfc721f5a1 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:22:50.730168 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ff14aa1-141f-48aa-9fc8-c6dfc721f5a1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:22:50.730449 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ff14aa1-141f-48aa-9fc8-c6dfc721f5a1 admin admin] PATH_INFO: `/v3/users/54bb78d9af854abf845b47f2bb925b74` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:22:50.731777 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ff14aa1-141f-48aa-9fc8-c6dfc721f5a1 admin admin] RBAC: Authorizing `identity:delete_user(user_id=54bb78d9af854abf845b47f2bb925b74)` {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:22:50.733714 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ff14aa1-141f-48aa-9fc8-c6dfc721f5a1 admin admin] RBAC: Authorization granted {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:22:50.783715 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.notifications [None req-2ff14aa1-141f-48aa-9fc8-c6dfc721f5a1 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '54bb78d9af854abf845b47f2bb925b74'} {{(pid=80377) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 10:22:50.790772 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.notifications [None req-2ff14aa1-141f-48aa-9fc8-c6dfc721f5a1 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '54bb78d9af854abf845b47f2bb925b74'} {{(pid=80377) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 10:22:50.798190 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.notifications [None req-2ff14aa1-141f-48aa-9fc8-c6dfc721f5a1 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '54bb78d9af854abf845b47f2bb925b74'} {{(pid=80377) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 10:22:50.798820 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.notifications [None req-2ff14aa1-141f-48aa-9fc8-c6dfc721f5a1 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '54bb78d9af854abf845b47f2bb925b74'} {{(pid=80377) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 10:22:50.811033 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.notifications [None req-2ff14aa1-141f-48aa-9fc8-c6dfc721f5a1 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '54bb78d9af854abf845b47f2bb925b74'} {{(pid=80377) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 10:22:50.816885 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: [pid: 80377|app: 0|req: 1244/2473] 10.222.0.82 () {58 vars in 1224 bytes} [Tue Aug 2 10:22:50 2022] DELETE /identity/v3/users/54bb78d9af854abf845b47f2bb925b74 => generated 0 bytes in 116 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 02 10:22:50.990374 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-164ebb41-513e-4ee6-8787-bee3cd8fb707 service placement] Authenticating user token {{(pid=80378) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:22:51.003710 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-164ebb41-513e-4ee6-8787-bee3cd8fb707 service placement] Validating token access rules against request {{(pid=80378) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:22:51.060814 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea4508a0-3e22-477e-a1fa-d7e97bc5808e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '891005fa5ff74041b75747d16cc66798', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7047912c5acd4adeba5d87c3bf895e39', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80378) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:22:51.061453 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea4508a0-3e22-477e-a1fa-d7e97bc5808e admin admin] REQUEST_METHOD: `DELETE` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:22:51.061733 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea4508a0-3e22-477e-a1fa-d7e97bc5808e admin admin] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:22:51.062041 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea4508a0-3e22-477e-a1fa-d7e97bc5808e admin admin] PATH_INFO: `/v3/projects/bdbf0111b2d84f12b205134fde982ae4` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:22:51.063194 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea4508a0-3e22-477e-a1fa-d7e97bc5808e admin admin] RBAC: Authorizing `identity:delete_project(project_id=bdbf0111b2d84f12b205134fde982ae4)` {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:22:51.065095 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea4508a0-3e22-477e-a1fa-d7e97bc5808e admin admin] RBAC: Authorization granted {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:22:51.112251 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.notifications [None req-ea4508a0-3e22-477e-a1fa-d7e97bc5808e admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'bdbf0111b2d84f12b205134fde982ae4', 'request_id': 'req-ea4508a0-3e22-477e-a1fa-d7e97bc5808e'} {{(pid=80378) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 10:22:51.118309 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.notifications [None req-ea4508a0-3e22-477e-a1fa-d7e97bc5808e admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'bdbf0111b2d84f12b205134fde982ae4', 'request_id': 'req-ea4508a0-3e22-477e-a1fa-d7e97bc5808e'} {{(pid=80378) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 10:22:51.123733 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.notifications [None req-ea4508a0-3e22-477e-a1fa-d7e97bc5808e admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'bdbf0111b2d84f12b205134fde982ae4', 'request_id': 'req-ea4508a0-3e22-477e-a1fa-d7e97bc5808e'} {{(pid=80378) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 10:22:51.135282 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.notifications [None req-ea4508a0-3e22-477e-a1fa-d7e97bc5808e admin admin] The token cache is being invalidate because project bdbf0111b2d84f12b205134fde982ae4 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80378) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 02 10:22:51.135540 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.notifications [None req-ea4508a0-3e22-477e-a1fa-d7e97bc5808e admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80378) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 10:22:51.136167 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.notifications [None req-ea4508a0-3e22-477e-a1fa-d7e97bc5808e admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80378) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 10:22:51.137674 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 1230/2474] 10.222.0.82 () {58 vars in 1233 bytes} [Tue Aug 2 10:22:50 2022] DELETE /identity/v3/projects/bdbf0111b2d84f12b205134fde982ae4 => generated 0 bytes in 148 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 02 10:22:51.146776 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ff14aa1-141f-48aa-9fc8-c6dfc721f5a1 admin admin] Authenticating user token {{(pid=80377) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:22:51.148213 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.fernet_utils [None req-2ff14aa1-141f-48aa-9fc8-c6dfc721f5a1 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80377) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 10:22:51.162187 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ff14aa1-141f-48aa-9fc8-c6dfc721f5a1 admin admin] Validating token access rules against request {{(pid=80377) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:22:51.205155 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG dbcounter [-] [80377] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80377) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 10:22:51.219915 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e272546-c7c9-4eeb-bfa8-839a766095f2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '891005fa5ff74041b75747d16cc66798', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7047912c5acd4adeba5d87c3bf895e39', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80377) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:22:51.220687 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e272546-c7c9-4eeb-bfa8-839a766095f2 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:22:51.220973 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e272546-c7c9-4eeb-bfa8-839a766095f2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:22:51.221249 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e272546-c7c9-4eeb-bfa8-839a766095f2 admin admin] PATH_INFO: `/v3/users/8db28092597b454dbfb040c0d8f35a96` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:22:51.222437 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e272546-c7c9-4eeb-bfa8-839a766095f2 admin admin] RBAC: Authorizing `identity:delete_user(user_id=8db28092597b454dbfb040c0d8f35a96)` {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:22:51.224373 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e272546-c7c9-4eeb-bfa8-839a766095f2 admin admin] RBAC: Authorization granted {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:22:51.271017 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.notifications [None req-7e272546-c7c9-4eeb-bfa8-839a766095f2 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '8db28092597b454dbfb040c0d8f35a96'} {{(pid=80377) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 10:22:51.277996 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.notifications [None req-7e272546-c7c9-4eeb-bfa8-839a766095f2 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '8db28092597b454dbfb040c0d8f35a96'} {{(pid=80377) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 10:22:51.285000 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.notifications [None req-7e272546-c7c9-4eeb-bfa8-839a766095f2 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '8db28092597b454dbfb040c0d8f35a96'} {{(pid=80377) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 10:22:51.285626 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.notifications [None req-7e272546-c7c9-4eeb-bfa8-839a766095f2 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '8db28092597b454dbfb040c0d8f35a96'} {{(pid=80377) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 10:22:51.297402 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.notifications [None req-7e272546-c7c9-4eeb-bfa8-839a766095f2 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '8db28092597b454dbfb040c0d8f35a96'} {{(pid=80377) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 10:22:51.303314 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: [pid: 80377|app: 0|req: 1245/2475] 10.222.0.82 () {58 vars in 1224 bytes} [Tue Aug 2 10:22:51 2022] DELETE /identity/v3/users/8db28092597b454dbfb040c0d8f35a96 => generated 0 bytes in 162 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 02 10:22:51.478721 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea4508a0-3e22-477e-a1fa-d7e97bc5808e admin admin] Authenticating user token {{(pid=80378) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:22:51.482908 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.fernet_utils [None req-ea4508a0-3e22-477e-a1fa-d7e97bc5808e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80378) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 10:22:51.489389 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG dbcounter [-] [80378] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80378) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 10:22:51.497149 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea4508a0-3e22-477e-a1fa-d7e97bc5808e admin admin] Validating token access rules against request {{(pid=80378) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:22:51.562472 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42f6f5be-ede1-4d32-b0fc-5ed311852313 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '891005fa5ff74041b75747d16cc66798', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7047912c5acd4adeba5d87c3bf895e39', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80378) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:22:51.563139 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42f6f5be-ede1-4d32-b0fc-5ed311852313 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:22:51.563419 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42f6f5be-ede1-4d32-b0fc-5ed311852313 admin admin] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:22:51.563691 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42f6f5be-ede1-4d32-b0fc-5ed311852313 admin admin] PATH_INFO: `/v3/projects/afe3b2ebd99a4affbccaf8effc92e5e3` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:22:51.564833 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42f6f5be-ede1-4d32-b0fc-5ed311852313 admin admin] RBAC: Authorizing `identity:delete_project(project_id=afe3b2ebd99a4affbccaf8effc92e5e3)` {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:22:51.566801 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42f6f5be-ede1-4d32-b0fc-5ed311852313 admin admin] RBAC: Authorization granted {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:22:51.612531 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.notifications [None req-42f6f5be-ede1-4d32-b0fc-5ed311852313 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'afe3b2ebd99a4affbccaf8effc92e5e3', 'request_id': 'req-42f6f5be-ede1-4d32-b0fc-5ed311852313'} {{(pid=80378) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 10:22:51.618708 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.notifications [None req-42f6f5be-ede1-4d32-b0fc-5ed311852313 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'afe3b2ebd99a4affbccaf8effc92e5e3', 'request_id': 'req-42f6f5be-ede1-4d32-b0fc-5ed311852313'} {{(pid=80378) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 10:22:51.624925 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.notifications [None req-42f6f5be-ede1-4d32-b0fc-5ed311852313 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'afe3b2ebd99a4affbccaf8effc92e5e3', 'request_id': 'req-42f6f5be-ede1-4d32-b0fc-5ed311852313'} {{(pid=80378) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 10:22:51.643264 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.notifications [None req-42f6f5be-ede1-4d32-b0fc-5ed311852313 admin admin] The token cache is being invalidate because project afe3b2ebd99a4affbccaf8effc92e5e3 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80378) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 02 10:22:51.643555 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.notifications [None req-42f6f5be-ede1-4d32-b0fc-5ed311852313 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80378) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 10:22:51.644181 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.notifications [None req-42f6f5be-ede1-4d32-b0fc-5ed311852313 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80378) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 10:22:51.645704 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 1231/2476] 10.222.0.82 () {58 vars in 1233 bytes} [Tue Aug 2 10:22:51 2022] DELETE /identity/v3/projects/afe3b2ebd99a4affbccaf8effc92e5e3 => generated 0 bytes in 168 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 02 10:22:51.650672 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f321281-78be-4775-aad6-a6dd9da75b8b None None] REQUEST_METHOD: `POST` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:22:51.650912 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f321281-78be-4775-aad6-a6dd9da75b8b None None] SCRIPT_NAME: `/identity` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:22:51.651195 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f321281-78be-4775-aad6-a6dd9da75b8b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:22:51.671299 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.auth.core [None req-1f321281-78be-4775-aad6-a6dd9da75b8b None None] MFA Rules not processed for user `891005fa5ff74041b75747d16cc66798`. Rule list: `[]` (Enabled: `True`). {{(pid=80377) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 02 10:22:51.672515 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.fernet_utils [None req-1f321281-78be-4775-aad6-a6dd9da75b8b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80377) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 10:22:51.732051 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: [pid: 80377|app: 0|req: 1246/2477] 10.222.0.82 () {58 vars in 958 bytes} [Tue Aug 2 10:22:51 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 82 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 02 10:22:51.740457 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1dcb48e6-ea97-4435-b2b4-0edb00992b27 None None] REQUEST_METHOD: `POST` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:22:51.740747 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1dcb48e6-ea97-4435-b2b4-0edb00992b27 None None] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:22:51.741023 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1dcb48e6-ea97-4435-b2b4-0edb00992b27 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:22:51.761157 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.auth.core [None req-1dcb48e6-ea97-4435-b2b4-0edb00992b27 None None] MFA Rules not processed for user `891005fa5ff74041b75747d16cc66798`. Rule list: `[]` (Enabled: `True`). {{(pid=80378) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 02 10:22:51.762389 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.fernet_utils [None req-1dcb48e6-ea97-4435-b2b4-0edb00992b27 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80378) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 10:22:51.770714 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 1232/2478] 10.222.0.82 () {58 vars in 958 bytes} [Tue Aug 2 10:22:51 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 02 10:22:51.774765 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f321281-78be-4775-aad6-a6dd9da75b8b None None] Authenticating user token {{(pid=80377) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:22:51.785055 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f321281-78be-4775-aad6-a6dd9da75b8b None None] Validating token access rules against request {{(pid=80377) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:22:51.789656 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2464dc73-3b1f-4936-9d35-328154116523 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '891005fa5ff74041b75747d16cc66798', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7047912c5acd4adeba5d87c3bf895e39', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80377) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:22:51.790321 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2464dc73-3b1f-4936-9d35-328154116523 admin admin] REQUEST_METHOD: `GET` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:22:51.790602 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2464dc73-3b1f-4936-9d35-328154116523 admin admin] SCRIPT_NAME: `/identity` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:22:51.790879 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2464dc73-3b1f-4936-9d35-328154116523 admin admin] PATH_INFO: `/v3/domains` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:22:51.792151 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2464dc73-3b1f-4936-9d35-328154116523 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80377) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 02 10:22:51.792462 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2464dc73-3b1f-4936-9d35-328154116523 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:22:51.794084 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2464dc73-3b1f-4936-9d35-328154116523 admin admin] RBAC: Authorization granted {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:22:51.807482 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: [pid: 80377|app: 0|req: 1247/2479] 10.222.0.82 () {58 vars in 1153 bytes} [Tue Aug 2 10:22:51 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 02 10:22:51.810106 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1dcb48e6-ea97-4435-b2b4-0edb00992b27 None None] Authenticating user token {{(pid=80378) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:22:51.819886 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1dcb48e6-ea97-4435-b2b4-0edb00992b27 None None] Validating token access rules against request {{(pid=80378) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:22:51.824877 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04454e14-6318-4e5a-a3bc-76ca66568150 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '891005fa5ff74041b75747d16cc66798', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7047912c5acd4adeba5d87c3bf895e39', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80378) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:22:51.825498 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04454e14-6318-4e5a-a3bc-76ca66568150 admin admin] REQUEST_METHOD: `POST` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:22:51.825817 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04454e14-6318-4e5a-a3bc-76ca66568150 admin admin] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:22:51.826092 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04454e14-6318-4e5a-a3bc-76ca66568150 admin admin] PATH_INFO: `/v3/projects` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:22:51.826772 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04454e14-6318-4e5a-a3bc-76ca66568150 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:22:51.828701 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04454e14-6318-4e5a-a3bc-76ca66568150 admin admin] RBAC: Authorization granted {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:22:51.861616 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 1233/2480] 10.222.0.82 () {60 vars in 1153 bytes} [Tue Aug 2 10:22:51 2022] POST /identity/v3/projects => generated 375 bytes in 52 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 02 10:22:51.865771 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2464dc73-3b1f-4936-9d35-328154116523 admin admin] Authenticating user token {{(pid=80377) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:22:51.875972 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2464dc73-3b1f-4936-9d35-328154116523 admin admin] Validating token access rules against request {{(pid=80377) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:22:51.928101 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG dbcounter [-] [80377] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80377) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 10:22:51.943854 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75d0cb6c-46cc-4e01-9a44-116edcb01b2e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '891005fa5ff74041b75747d16cc66798', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7047912c5acd4adeba5d87c3bf895e39', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80377) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:22:51.944463 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75d0cb6c-46cc-4e01-9a44-116edcb01b2e admin admin] REQUEST_METHOD: `POST` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:22:51.944746 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75d0cb6c-46cc-4e01-9a44-116edcb01b2e admin admin] SCRIPT_NAME: `/identity` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:22:51.945019 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75d0cb6c-46cc-4e01-9a44-116edcb01b2e admin admin] PATH_INFO: `/v3/users` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:22:51.945695 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75d0cb6c-46cc-4e01-9a44-116edcb01b2e admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:22:51.947754 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75d0cb6c-46cc-4e01-9a44-116edcb01b2e admin admin] RBAC: Authorization granted {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:22:51.967997 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: [pid: 80377|app: 0|req: 1248/2481] 10.222.0.82 () {60 vars in 1144 bytes} [Tue Aug 2 10:22:51 2022] POST /identity/v3/users => generated 288 bytes in 103 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 02 10:22:51.972191 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04454e14-6318-4e5a-a3bc-76ca66568150 admin admin] Authenticating user token {{(pid=80378) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:22:51.983038 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04454e14-6318-4e5a-a3bc-76ca66568150 admin admin] Validating token access rules against request {{(pid=80378) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:22:51.988711 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9db0d416-eebe-47f2-89de-6e5d2116926c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '891005fa5ff74041b75747d16cc66798', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7047912c5acd4adeba5d87c3bf895e39', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80378) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:22:51.989334 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9db0d416-eebe-47f2-89de-6e5d2116926c admin admin] REQUEST_METHOD: `GET` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:22:51.989617 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9db0d416-eebe-47f2-89de-6e5d2116926c admin admin] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:22:51.989940 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9db0d416-eebe-47f2-89de-6e5d2116926c admin admin] PATH_INFO: `/v3/roles` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:22:51.990658 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9db0d416-eebe-47f2-89de-6e5d2116926c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:22:51.993707 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9db0d416-eebe-47f2-89de-6e5d2116926c admin admin] RBAC: Authorization granted {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:22:52.005536 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 1234/2482] 10.222.0.82 () {58 vars in 1122 bytes} [Tue Aug 2 10:22:51 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 02 10:22:52.009598 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75d0cb6c-46cc-4e01-9a44-116edcb01b2e admin admin] Authenticating user token {{(pid=80377) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:22:52.019547 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75d0cb6c-46cc-4e01-9a44-116edcb01b2e admin admin] Validating token access rules against request {{(pid=80377) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:22:52.024134 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb569e6f-9a8c-496c-91b5-dfac49c942bf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '891005fa5ff74041b75747d16cc66798', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7047912c5acd4adeba5d87c3bf895e39', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80377) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:22:52.024742 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb569e6f-9a8c-496c-91b5-dfac49c942bf admin admin] REQUEST_METHOD: `GET` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:22:52.025026 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb569e6f-9a8c-496c-91b5-dfac49c942bf admin admin] SCRIPT_NAME: `/identity` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:22:52.025301 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb569e6f-9a8c-496c-91b5-dfac49c942bf admin admin] PATH_INFO: `/v3/roles` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:22:52.026544 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb569e6f-9a8c-496c-91b5-dfac49c942bf admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:22:52.028137 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb569e6f-9a8c-496c-91b5-dfac49c942bf admin admin] RBAC: Authorization granted {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:22:52.038786 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: [pid: 80377|app: 0|req: 1249/2483] 10.222.0.82 () {58 vars in 1122 bytes} [Tue Aug 2 10:22:52 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 02 10:22:52.043497 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9db0d416-eebe-47f2-89de-6e5d2116926c admin admin] Authenticating user token {{(pid=80378) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:22:52.053529 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9db0d416-eebe-47f2-89de-6e5d2116926c admin admin] Validating token access rules against request {{(pid=80378) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:22:52.058581 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-138d5fdf-b2fb-4315-9806-b871780a6731 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '891005fa5ff74041b75747d16cc66798', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7047912c5acd4adeba5d87c3bf895e39', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80378) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:22:52.059254 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-138d5fdf-b2fb-4315-9806-b871780a6731 admin admin] REQUEST_METHOD: `PUT` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:22:52.059534 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-138d5fdf-b2fb-4315-9806-b871780a6731 admin admin] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:22:52.059850 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-138d5fdf-b2fb-4315-9806-b871780a6731 admin admin] PATH_INFO: `/v3/projects/06660bbc73344c53b5a0337808514df7/users/e08bc25f122445f592652671790cc658/roles/e1c04b19fc4543b8b54f58bbcffbc598` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:22:52.075932 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-138d5fdf-b2fb-4315-9806-b871780a6731 admin admin] RBAC: Authorizing `identity:create_grant(project_id=06660bbc73344c53b5a0337808514df7, user_id=e08bc25f122445f592652671790cc658, role_id=e1c04b19fc4543b8b54f58bbcffbc598)` {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:22:52.078595 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-138d5fdf-b2fb-4315-9806-b871780a6731 admin admin] RBAC: Authorization granted {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:22:52.091397 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 1235/2484] 10.222.0.82 () {60 vars in 1483 bytes} [Tue Aug 2 10:22:52 2022] PUT /identity/v3/projects/06660bbc73344c53b5a0337808514df7/users/e08bc25f122445f592652671790cc658/roles/e1c04b19fc4543b8b54f58bbcffbc598 => generated 0 bytes in 48 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 02 10:22:52.096403 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0516100a-2693-4143-af37-6300bb04cc14 None None] REQUEST_METHOD: `POST` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:22:52.097100 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0516100a-2693-4143-af37-6300bb04cc14 None None] SCRIPT_NAME: `/identity` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:22:52.097386 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0516100a-2693-4143-af37-6300bb04cc14 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:22:52.115824 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.auth.core [None req-0516100a-2693-4143-af37-6300bb04cc14 None None] MFA Rules not processed for user `891005fa5ff74041b75747d16cc66798`. Rule list: `[]` (Enabled: `True`). {{(pid=80377) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 02 10:22:52.117132 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.fernet_utils [None req-0516100a-2693-4143-af37-6300bb04cc14 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80377) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 10:22:52.176189 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: [pid: 80377|app: 0|req: 1250/2485] 10.222.0.82 () {58 vars in 958 bytes} [Tue Aug 2 10:22:52 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 81 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 02 10:22:52.186295 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-138d5fdf-b2fb-4315-9806-b871780a6731 admin admin] Authenticating user token {{(pid=80378) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:22:52.187995 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.fernet_utils [None req-138d5fdf-b2fb-4315-9806-b871780a6731 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80378) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 10:22:52.219585 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG dbcounter [-] [80378] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=80378) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 10:22:52.247071 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-138d5fdf-b2fb-4315-9806-b871780a6731 admin admin] Validating token access rules against request {{(pid=80378) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:22:52.250663 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4056f4c9-4de3-497d-804f-d02b8da00b22 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '33b87170f0634b67bce60750847b5cbe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f1b38ebb804c41c69772c998aae72c41', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80378) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:22:52.251273 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4056f4c9-4de3-497d-804f-d02b8da00b22 service neutron] REQUEST_METHOD: `GET` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:22:52.251553 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4056f4c9-4de3-497d-804f-d02b8da00b22 service neutron] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:22:52.251823 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4056f4c9-4de3-497d-804f-d02b8da00b22 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:22:52.260526 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4056f4c9-4de3-497d-804f-d02b8da00b22 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:22:52.261982 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4056f4c9-4de3-497d-804f-d02b8da00b22 service neutron] RBAC: Authorization granted {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:22:52.271235 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 1236/2486] 10.222.0.82 () {60 vars in 1365 bytes} [Tue Aug 2 10:22:52 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 85 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 02 10:22:56.678046 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f44840e4-fab3-4ce7-bac2-f3088e13abf8 None None] REQUEST_METHOD: `POST` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:22:56.678669 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f44840e4-fab3-4ce7-bac2-f3088e13abf8 None None] SCRIPT_NAME: `/identity` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:22:56.678669 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f44840e4-fab3-4ce7-bac2-f3088e13abf8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:22:56.713000 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.auth.core [None req-f44840e4-fab3-4ce7-bac2-f3088e13abf8 None None] MFA Rules not processed for user `e08bc25f122445f592652671790cc658`. Rule list: `[]` (Enabled: `True`). {{(pid=80377) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 02 10:22:56.714902 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.fernet_utils [None req-f44840e4-fab3-4ce7-bac2-f3088e13abf8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80377) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 10:22:56.790921 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: [pid: 80377|app: 0|req: 1251/2487] 10.222.0.82 () {58 vars in 958 bytes} [Tue Aug 2 10:22:56 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 114 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 02 10:22:56.795427 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4056f4c9-4de3-497d-804f-d02b8da00b22 service neutron] Authenticating user token {{(pid=80378) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:22:56.809048 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4056f4c9-4de3-497d-804f-d02b8da00b22 service neutron] Validating token access rules against request {{(pid=80378) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:22:56.813721 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97c6e6a3-b52b-43df-ac93-6a7da4238a47 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '891005fa5ff74041b75747d16cc66798', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7047912c5acd4adeba5d87c3bf895e39', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80378) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:22:56.814404 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97c6e6a3-b52b-43df-ac93-6a7da4238a47 admin admin] REQUEST_METHOD: `POST` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:22:56.814670 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97c6e6a3-b52b-43df-ac93-6a7da4238a47 admin admin] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:22:56.815292 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97c6e6a3-b52b-43df-ac93-6a7da4238a47 admin admin] PATH_INFO: `/v3/projects` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:22:56.815616 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97c6e6a3-b52b-43df-ac93-6a7da4238a47 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:22:56.817611 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97c6e6a3-b52b-43df-ac93-6a7da4238a47 admin admin] RBAC: Authorization granted {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:22:56.849654 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 1237/2488] 10.222.0.82 () {60 vars in 1153 bytes} [Tue Aug 2 10:22:56 2022] POST /identity/v3/projects => generated 377 bytes in 55 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 02 10:22:56.854067 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f44840e4-fab3-4ce7-bac2-f3088e13abf8 None None] Authenticating user token {{(pid=80377) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:22:56.870968 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f44840e4-fab3-4ce7-bac2-f3088e13abf8 None None] Validating token access rules against request {{(pid=80377) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:22:56.939100 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a08d8da2-72d9-4920-95bc-e63278872d56 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '891005fa5ff74041b75747d16cc66798', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7047912c5acd4adeba5d87c3bf895e39', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80377) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:22:56.940033 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a08d8da2-72d9-4920-95bc-e63278872d56 admin admin] REQUEST_METHOD: `POST` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:22:56.940322 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a08d8da2-72d9-4920-95bc-e63278872d56 admin admin] SCRIPT_NAME: `/identity` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:22:56.940603 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a08d8da2-72d9-4920-95bc-e63278872d56 admin admin] PATH_INFO: `/v3/users` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:22:56.941309 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a08d8da2-72d9-4920-95bc-e63278872d56 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:22:56.943728 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a08d8da2-72d9-4920-95bc-e63278872d56 admin admin] RBAC: Authorization granted {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:22:56.953328 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG dbcounter [-] [80377] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=80377) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 10:22:56.968074 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: [pid: 80377|app: 0|req: 1252/2489] 10.222.0.82 () {60 vars in 1144 bytes} [Tue Aug 2 10:22:56 2022] POST /identity/v3/users => generated 295 bytes in 115 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 02 10:22:56.972040 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97c6e6a3-b52b-43df-ac93-6a7da4238a47 admin admin] Authenticating user token {{(pid=80378) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:22:56.983537 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97c6e6a3-b52b-43df-ac93-6a7da4238a47 admin admin] Validating token access rules against request {{(pid=80378) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:22:56.988407 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-524308d2-9965-4979-987d-596a781713c0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '891005fa5ff74041b75747d16cc66798', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7047912c5acd4adeba5d87c3bf895e39', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80378) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:22:56.989036 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-524308d2-9965-4979-987d-596a781713c0 admin admin] REQUEST_METHOD: `GET` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:22:56.989321 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-524308d2-9965-4979-987d-596a781713c0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:22:56.989598 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-524308d2-9965-4979-987d-596a781713c0 admin admin] PATH_INFO: `/v3/roles` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:22:56.990378 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-524308d2-9965-4979-987d-596a781713c0 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:22:56.992016 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-524308d2-9965-4979-987d-596a781713c0 admin admin] RBAC: Authorization granted {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:22:57.003916 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 1238/2490] 10.222.0.82 () {58 vars in 1122 bytes} [Tue Aug 2 10:22:56 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 02 10:22:57.008371 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a08d8da2-72d9-4920-95bc-e63278872d56 admin admin] Authenticating user token {{(pid=80377) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:22:57.018782 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a08d8da2-72d9-4920-95bc-e63278872d56 admin admin] Validating token access rules against request {{(pid=80377) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:22:57.024850 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5423694-1c0f-439f-9499-e8c0c7b1ac0c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '891005fa5ff74041b75747d16cc66798', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7047912c5acd4adeba5d87c3bf895e39', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80377) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:22:57.025595 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5423694-1c0f-439f-9499-e8c0c7b1ac0c admin admin] REQUEST_METHOD: `PUT` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:22:57.026433 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5423694-1c0f-439f-9499-e8c0c7b1ac0c admin admin] SCRIPT_NAME: `/identity` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:22:57.026718 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5423694-1c0f-439f-9499-e8c0c7b1ac0c admin admin] PATH_INFO: `/v3/projects/d79d77dd089c4b3895fa177c86127bd7/users/1b65b5367f034856b93e957ce0897170/roles/237966442f05411191105bf336bc14f3` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:22:57.043467 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5423694-1c0f-439f-9499-e8c0c7b1ac0c admin admin] RBAC: Authorizing `identity:create_grant(project_id=d79d77dd089c4b3895fa177c86127bd7, user_id=1b65b5367f034856b93e957ce0897170, role_id=237966442f05411191105bf336bc14f3)` {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:22:57.046343 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5423694-1c0f-439f-9499-e8c0c7b1ac0c admin admin] RBAC: Authorization granted {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:22:57.061172 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: [pid: 80377|app: 0|req: 1253/2491] 10.222.0.82 () {60 vars in 1483 bytes} [Tue Aug 2 10:22:57 2022] PUT /identity/v3/projects/d79d77dd089c4b3895fa177c86127bd7/users/1b65b5367f034856b93e957ce0897170/roles/237966442f05411191105bf336bc14f3 => generated 0 bytes in 53 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 02 10:22:57.065025 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-524308d2-9965-4979-987d-596a781713c0 admin admin] Authenticating user token {{(pid=80378) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:22:57.076646 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-524308d2-9965-4979-987d-596a781713c0 admin admin] Validating token access rules against request {{(pid=80378) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:22:57.136086 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c178fdb-c706-4372-b82f-abfaac25701c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '891005fa5ff74041b75747d16cc66798', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7047912c5acd4adeba5d87c3bf895e39', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80378) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:22:57.136720 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c178fdb-c706-4372-b82f-abfaac25701c admin admin] REQUEST_METHOD: `GET` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:22:57.137013 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c178fdb-c706-4372-b82f-abfaac25701c admin admin] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:22:57.137291 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c178fdb-c706-4372-b82f-abfaac25701c admin admin] PATH_INFO: `/v3/roles` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:22:57.138477 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c178fdb-c706-4372-b82f-abfaac25701c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:22:57.139800 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c178fdb-c706-4372-b82f-abfaac25701c admin admin] RBAC: Authorization granted {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:22:57.153600 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 1239/2492] 10.222.0.82 () {58 vars in 1122 bytes} [Tue Aug 2 10:22:57 2022] GET /identity/v3/roles => generated 1420 bytes in 89 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 02 10:22:57.157843 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5423694-1c0f-439f-9499-e8c0c7b1ac0c admin admin] Authenticating user token {{(pid=80377) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:22:57.168663 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5423694-1c0f-439f-9499-e8c0c7b1ac0c admin admin] Validating token access rules against request {{(pid=80377) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:22:57.173489 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7913f40f-aa21-4203-805e-7b98b739683e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '891005fa5ff74041b75747d16cc66798', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7047912c5acd4adeba5d87c3bf895e39', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80377) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:22:57.174193 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7913f40f-aa21-4203-805e-7b98b739683e admin admin] REQUEST_METHOD: `PUT` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:22:57.174472 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7913f40f-aa21-4203-805e-7b98b739683e admin admin] SCRIPT_NAME: `/identity` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:22:57.174789 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7913f40f-aa21-4203-805e-7b98b739683e admin admin] PATH_INFO: `/v3/projects/d79d77dd089c4b3895fa177c86127bd7/users/1b65b5367f034856b93e957ce0897170/roles/237966442f05411191105bf336bc14f3` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:22:57.176691 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7913f40f-aa21-4203-805e-7b98b739683e admin admin] RBAC: Authorizing `identity:create_grant(project_id=d79d77dd089c4b3895fa177c86127bd7, user_id=1b65b5367f034856b93e957ce0897170, role_id=237966442f05411191105bf336bc14f3)` {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:22:57.179345 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7913f40f-aa21-4203-805e-7b98b739683e admin admin] RBAC: Authorization granted {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:22:57.191094 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: [pid: 80377|app: 0|req: 1254/2493] 10.222.0.82 () {60 vars in 1483 bytes} [Tue Aug 2 10:22:57 2022] PUT /identity/v3/projects/d79d77dd089c4b3895fa177c86127bd7/users/1b65b5367f034856b93e957ce0897170/roles/237966442f05411191105bf336bc14f3 => generated 0 bytes in 34 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 02 10:22:58.320596 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c178fdb-c706-4372-b82f-abfaac25701c admin admin] Authenticating user token {{(pid=80378) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:22:58.323541 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.fernet_utils [None req-0c178fdb-c706-4372-b82f-abfaac25701c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80378) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 10:22:58.403120 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c178fdb-c706-4372-b82f-abfaac25701c admin admin] Validating token access rules against request {{(pid=80378) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:22:58.454591 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG dbcounter [-] [80378] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80378) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 10:22:58.509818 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8977c627-0dbc-4d30-afab-f1b5ddd599d5 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88af518196674fe593ad88d30111fb7d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f1b38ebb804c41c69772c998aae72c41', 'project_domain_id': 'default', 'roles': ['admin', 'ResellerAdmin', 'reader', 'service', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80378) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:22:58.511193 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8977c627-0dbc-4d30-afab-f1b5ddd599d5 service nova] REQUEST_METHOD: `GET` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:22:58.511535 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8977c627-0dbc-4d30-afab-f1b5ddd599d5 service nova] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:22:58.512478 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8977c627-0dbc-4d30-afab-f1b5ddd599d5 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:22:58.515104 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.fernet_utils [None req-8977c627-0dbc-4d30-afab-f1b5ddd599d5 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80378) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 10:22:58.582154 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8977c627-0dbc-4d30-afab-f1b5ddd599d5 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:22:58.583895 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8977c627-0dbc-4d30-afab-f1b5ddd599d5 service nova] RBAC: Authorization granted {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:22:58.593832 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 1240/2494] 10.222.0.82 () {60 vars in 1365 bytes} [Tue Aug 2 10:22:58 2022] GET /identity/v3/auth/tokens => generated 3250 bytes in 274 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 02 10:23:01.908045 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55748cda-8e43-4be7-8268-c1fb27b38176 None None] REQUEST_METHOD: `POST` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:23:01.908512 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55748cda-8e43-4be7-8268-c1fb27b38176 None None] SCRIPT_NAME: `/identity` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:23:01.908612 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55748cda-8e43-4be7-8268-c1fb27b38176 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:23:01.942161 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.auth.core [None req-55748cda-8e43-4be7-8268-c1fb27b38176 None None] MFA Rules not processed for user `1b65b5367f034856b93e957ce0897170`. Rule list: `[]` (Enabled: `True`). {{(pid=80377) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 02 10:23:01.943973 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.fernet_utils [None req-55748cda-8e43-4be7-8268-c1fb27b38176 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80377) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 10:23:02.023110 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: [pid: 80377|app: 0|req: 1255/2495] 10.222.0.82 () {58 vars in 958 bytes} [Tue Aug 2 10:23:01 2022] POST /identity/v3/auth/tokens => generated 3321 bytes in 116 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 02 10:23:02.037094 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8977c627-0dbc-4d30-afab-f1b5ddd599d5 service nova] Authenticating user token {{(pid=80378) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:23:02.038665 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.fernet_utils [None req-8977c627-0dbc-4d30-afab-f1b5ddd599d5 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80378) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 10:23:02.054327 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8977c627-0dbc-4d30-afab-f1b5ddd599d5 service nova] Validating token access rules against request {{(pid=80378) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:23:02.063158 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e477ba86-34c2-473b-93b4-f8d8a7f0b92a service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88af518196674fe593ad88d30111fb7d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f1b38ebb804c41c69772c998aae72c41', 'project_domain_id': 'default', 'roles': ['admin', 'ResellerAdmin', 'reader', 'service', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80378) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:23:02.064003 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e477ba86-34c2-473b-93b4-f8d8a7f0b92a service nova] REQUEST_METHOD: `GET` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:23:02.064290 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e477ba86-34c2-473b-93b4-f8d8a7f0b92a service nova] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:23:02.064560 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e477ba86-34c2-473b-93b4-f8d8a7f0b92a service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:23:02.122915 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e477ba86-34c2-473b-93b4-f8d8a7f0b92a service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:23:02.124779 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e477ba86-34c2-473b-93b4-f8d8a7f0b92a service nova] RBAC: Authorization granted {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:23:02.134799 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 1241/2496] 10.222.0.82 () {60 vars in 1365 bytes} [Tue Aug 2 10:23:02 2022] GET /identity/v3/auth/tokens => generated 3252 bytes in 98 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 02 10:23:11.303434 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e05d136-7dfd-4d6b-baae-702f31e21eae None None] REQUEST_METHOD: `POST` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:23:11.303954 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e05d136-7dfd-4d6b-baae-702f31e21eae None None] SCRIPT_NAME: `/identity` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:23:11.304069 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e05d136-7dfd-4d6b-baae-702f31e21eae None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:23:11.323463 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.auth.core [None req-3e05d136-7dfd-4d6b-baae-702f31e21eae None None] MFA Rules not processed for user `e881ae7c60254c12860fff0a40480641`. Rule list: `[]` (Enabled: `True`). {{(pid=80377) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 02 10:23:11.324680 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.fernet_utils [None req-3e05d136-7dfd-4d6b-baae-702f31e21eae None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80377) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 10:23:11.382469 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: [pid: 80377|app: 0|req: 1256/2497] 10.222.0.82 () {58 vars in 958 bytes} [Tue Aug 2 10:23:11 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 80 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 02 10:23:11.695827 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e477ba86-34c2-473b-93b4-f8d8a7f0b92a service nova] Authenticating user token {{(pid=80378) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:23:11.697231 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.fernet_utils [None req-e477ba86-34c2-473b-93b4-f8d8a7f0b92a service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80378) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 10:23:11.764265 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e477ba86-34c2-473b-93b4-f8d8a7f0b92a service nova] Validating token access rules against request {{(pid=80378) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:23:11.768059 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12dbe92b-e850-4d77-ab53-0905be219e2b service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '33b87170f0634b67bce60750847b5cbe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f1b38ebb804c41c69772c998aae72c41', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80378) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:23:11.768688 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12dbe92b-e850-4d77-ab53-0905be219e2b service neutron] REQUEST_METHOD: `GET` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:23:11.769000 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12dbe92b-e850-4d77-ab53-0905be219e2b service neutron] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:23:11.769282 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12dbe92b-e850-4d77-ab53-0905be219e2b service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:23:11.771469 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.fernet_utils [None req-12dbe92b-e850-4d77-ab53-0905be219e2b service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80378) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 10:23:11.781467 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12dbe92b-e850-4d77-ab53-0905be219e2b service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:23:11.783011 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12dbe92b-e850-4d77-ab53-0905be219e2b service neutron] RBAC: Authorization granted {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:23:11.791206 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 1242/2498] 10.222.0.82 () {60 vars in 1365 bytes} [Tue Aug 2 10:23:11 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 96 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 02 10:23:21.375640 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG dbcounter [-] [80377] Writing DB stats keystone:INSERT=4,keystone:SELECT=76 {{(pid=80377) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 10:23:21.787559 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG dbcounter [-] [80378] Writing DB stats keystone:SELECT=95 {{(pid=80378) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 10:23:27.840666 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e05d136-7dfd-4d6b-baae-702f31e21eae None None] Authenticating user token {{(pid=80377) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:23:27.856652 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e05d136-7dfd-4d6b-baae-702f31e21eae None None] Validating token access rules against request {{(pid=80377) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:23:27.862895 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67c5dd98-777e-437d-9c84-cc4623372014 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88af518196674fe593ad88d30111fb7d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f1b38ebb804c41c69772c998aae72c41', 'project_domain_id': 'default', 'roles': ['admin', 'ResellerAdmin', 'reader', 'service', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80377) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:23:27.863530 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67c5dd98-777e-437d-9c84-cc4623372014 service nova] REQUEST_METHOD: `GET` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:23:27.863994 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67c5dd98-777e-437d-9c84-cc4623372014 service nova] SCRIPT_NAME: `/identity` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:23:27.864476 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67c5dd98-777e-437d-9c84-cc4623372014 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:23:27.866766 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.fernet_utils [None req-67c5dd98-777e-437d-9c84-cc4623372014 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80377) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 10:23:27.884909 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67c5dd98-777e-437d-9c84-cc4623372014 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:23:27.886794 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67c5dd98-777e-437d-9c84-cc4623372014 service nova] RBAC: Authorization granted {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:23:27.908314 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: [pid: 80377|app: 0|req: 1257/2499] 10.222.0.82 () {60 vars in 1365 bytes} [Tue Aug 2 10:23:27 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 59 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 02 10:23:28.054874 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7333175d-a6b0-4a07-9c0b-c29331f189e0 None None] REQUEST_METHOD: `POST` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:23:28.055436 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7333175d-a6b0-4a07-9c0b-c29331f189e0 None None] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:23:28.055881 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7333175d-a6b0-4a07-9c0b-c29331f189e0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:23:28.079398 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.auth.core [None req-7333175d-a6b0-4a07-9c0b-c29331f189e0 None None] MFA Rules not processed for user `e08bc25f122445f592652671790cc658`. Rule list: `[]` (Enabled: `True`). {{(pid=80378) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 02 10:23:28.080704 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.fernet_utils [None req-7333175d-a6b0-4a07-9c0b-c29331f189e0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80378) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 10:23:28.088634 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 1243/2500] 10.222.0.82 () {58 vars in 958 bytes} [Tue Aug 2 10:23:28 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 34 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 02 10:23:37.894963 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG dbcounter [-] [80377] Writing DB stats keystone:SELECT=6 {{(pid=80377) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 10:23:38.072800 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG dbcounter [-] [80378] Writing DB stats keystone:SELECT=3 {{(pid=80378) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 10:23:40.035038 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67c5dd98-777e-437d-9c84-cc4623372014 service nova] Authenticating user token {{(pid=80377) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:23:40.046409 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67c5dd98-777e-437d-9c84-cc4623372014 service nova] Validating token access rules against request {{(pid=80377) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:23:40.051125 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6f532b7-de75-4480-a938-54f037bcb4ed service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '33b87170f0634b67bce60750847b5cbe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f1b38ebb804c41c69772c998aae72c41', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80377) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:23:40.052136 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6f532b7-de75-4480-a938-54f037bcb4ed service neutron] REQUEST_METHOD: `GET` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:23:40.052524 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6f532b7-de75-4480-a938-54f037bcb4ed service neutron] SCRIPT_NAME: `/identity` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:23:40.052897 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6f532b7-de75-4480-a938-54f037bcb4ed service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:23:40.054881 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.fernet_utils [None req-c6f532b7-de75-4480-a938-54f037bcb4ed service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80377) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 10:23:40.135311 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6f532b7-de75-4480-a938-54f037bcb4ed service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:23:40.137468 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6f532b7-de75-4480-a938-54f037bcb4ed service neutron] RBAC: Authorization granted {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:23:40.147706 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: [pid: 80377|app: 0|req: 1258/2501] 10.222.0.82 () {60 vars in 1365 bytes} [Tue Aug 2 10:23:40 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 114 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 02 10:23:43.303787 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7333175d-a6b0-4a07-9c0b-c29331f189e0 None None] Authenticating user token {{(pid=80378) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:23:43.305247 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.fernet_utils [None req-7333175d-a6b0-4a07-9c0b-c29331f189e0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80378) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 10:23:43.321402 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7333175d-a6b0-4a07-9c0b-c29331f189e0 None None] Validating token access rules against request {{(pid=80378) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:23:43.326686 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8874d64-8487-43f1-803f-4e0c416ccf2e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '891005fa5ff74041b75747d16cc66798', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7047912c5acd4adeba5d87c3bf895e39', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80378) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:23:43.327329 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8874d64-8487-43f1-803f-4e0c416ccf2e admin admin] REQUEST_METHOD: `DELETE` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:23:43.327614 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8874d64-8487-43f1-803f-4e0c416ccf2e admin admin] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:23:43.327890 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8874d64-8487-43f1-803f-4e0c416ccf2e admin admin] PATH_INFO: `/v3/users/99616b362eac4b7dba445dafd3ca1426` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:23:43.329061 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8874d64-8487-43f1-803f-4e0c416ccf2e admin admin] RBAC: Authorizing `identity:delete_user(user_id=99616b362eac4b7dba445dafd3ca1426)` {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:23:43.331018 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8874d64-8487-43f1-803f-4e0c416ccf2e admin admin] RBAC: Authorization granted {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:23:43.378159 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.notifications [None req-e8874d64-8487-43f1-803f-4e0c416ccf2e admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '99616b362eac4b7dba445dafd3ca1426'} {{(pid=80378) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 10:23:43.385254 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.notifications [None req-e8874d64-8487-43f1-803f-4e0c416ccf2e admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '99616b362eac4b7dba445dafd3ca1426'} {{(pid=80378) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 10:23:43.392787 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.notifications [None req-e8874d64-8487-43f1-803f-4e0c416ccf2e admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '99616b362eac4b7dba445dafd3ca1426'} {{(pid=80378) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 10:23:43.393415 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.notifications [None req-e8874d64-8487-43f1-803f-4e0c416ccf2e admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '99616b362eac4b7dba445dafd3ca1426'} {{(pid=80378) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 10:23:43.410777 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.notifications [None req-e8874d64-8487-43f1-803f-4e0c416ccf2e admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '99616b362eac4b7dba445dafd3ca1426'} {{(pid=80378) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 10:23:43.416757 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 1244/2502] 10.222.0.82 () {58 vars in 1224 bytes} [Tue Aug 2 10:23:43 2022] DELETE /identity/v3/users/99616b362eac4b7dba445dafd3ca1426 => generated 0 bytes in 114 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 02 10:23:43.593537 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6f532b7-de75-4480-a938-54f037bcb4ed service neutron] Authenticating user token {{(pid=80377) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:23:43.603525 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6f532b7-de75-4480-a938-54f037bcb4ed service neutron] Validating token access rules against request {{(pid=80377) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:23:43.671072 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-863a0718-99e8-46ae-b4b8-2884c992d419 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '891005fa5ff74041b75747d16cc66798', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7047912c5acd4adeba5d87c3bf895e39', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80377) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:23:43.671757 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-863a0718-99e8-46ae-b4b8-2884c992d419 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:23:43.672043 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-863a0718-99e8-46ae-b4b8-2884c992d419 admin admin] SCRIPT_NAME: `/identity` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:23:43.672323 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-863a0718-99e8-46ae-b4b8-2884c992d419 admin admin] PATH_INFO: `/v3/projects/ce6b9eda623740608e0c9998aaa74dab` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:23:43.673517 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-863a0718-99e8-46ae-b4b8-2884c992d419 admin admin] RBAC: Authorizing `identity:delete_project(project_id=ce6b9eda623740608e0c9998aaa74dab)` {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:23:43.675556 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-863a0718-99e8-46ae-b4b8-2884c992d419 admin admin] RBAC: Authorization granted {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:23:43.731611 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.notifications [None req-863a0718-99e8-46ae-b4b8-2884c992d419 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'ce6b9eda623740608e0c9998aaa74dab', 'request_id': 'req-863a0718-99e8-46ae-b4b8-2884c992d419'} {{(pid=80377) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 10:23:43.737269 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.notifications [None req-863a0718-99e8-46ae-b4b8-2884c992d419 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'ce6b9eda623740608e0c9998aaa74dab', 'request_id': 'req-863a0718-99e8-46ae-b4b8-2884c992d419'} {{(pid=80377) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 10:23:43.743833 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.notifications [None req-863a0718-99e8-46ae-b4b8-2884c992d419 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'ce6b9eda623740608e0c9998aaa74dab', 'request_id': 'req-863a0718-99e8-46ae-b4b8-2884c992d419'} {{(pid=80377) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 10:23:43.757035 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.notifications [None req-863a0718-99e8-46ae-b4b8-2884c992d419 admin admin] The token cache is being invalidate because project ce6b9eda623740608e0c9998aaa74dab was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80377) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 02 10:23:43.757325 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.notifications [None req-863a0718-99e8-46ae-b4b8-2884c992d419 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80377) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 10:23:43.757995 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.notifications [None req-863a0718-99e8-46ae-b4b8-2884c992d419 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80377) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 10:23:43.759536 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: [pid: 80377|app: 0|req: 1259/2503] 10.222.0.82 () {58 vars in 1233 bytes} [Tue Aug 2 10:23:43 2022] DELETE /identity/v3/projects/ce6b9eda623740608e0c9998aaa74dab => generated 0 bytes in 167 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 02 10:23:43.765134 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cda607c-288d-4a06-aaec-2044a0275a91 None None] REQUEST_METHOD: `POST` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:23:43.765419 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cda607c-288d-4a06-aaec-2044a0275a91 None None] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:23:43.765689 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cda607c-288d-4a06-aaec-2044a0275a91 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:23:43.784791 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.auth.core [None req-2cda607c-288d-4a06-aaec-2044a0275a91 None None] MFA Rules not processed for user `891005fa5ff74041b75747d16cc66798`. Rule list: `[]` (Enabled: `True`). {{(pid=80378) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 02 10:23:43.786016 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.fernet_utils [None req-2cda607c-288d-4a06-aaec-2044a0275a91 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80378) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 10:23:43.845151 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 1245/2504] 10.222.0.82 () {58 vars in 958 bytes} [Tue Aug 2 10:23:43 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 81 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 02 10:23:43.853070 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a695cb4-001f-4a24-9e1c-bb4747b70b7d None None] REQUEST_METHOD: `POST` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:23:43.853070 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a695cb4-001f-4a24-9e1c-bb4747b70b7d None None] SCRIPT_NAME: `/identity` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:23:43.853370 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a695cb4-001f-4a24-9e1c-bb4747b70b7d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:23:43.874155 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.auth.core [None req-4a695cb4-001f-4a24-9e1c-bb4747b70b7d None None] MFA Rules not processed for user `891005fa5ff74041b75747d16cc66798`. Rule list: `[]` (Enabled: `True`). {{(pid=80377) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 02 10:23:43.875368 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.fernet_utils [None req-4a695cb4-001f-4a24-9e1c-bb4747b70b7d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80377) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 10:23:43.883762 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: [pid: 80377|app: 0|req: 1260/2505] 10.222.0.82 () {58 vars in 958 bytes} [Tue Aug 2 10:23:43 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 02 10:23:43.887931 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2cda607c-288d-4a06-aaec-2044a0275a91 None None] Authenticating user token {{(pid=80378) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:23:43.897672 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2cda607c-288d-4a06-aaec-2044a0275a91 None None] Validating token access rules against request {{(pid=80378) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:23:43.902293 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d050a0d-6813-48a5-b33c-e8487534ddb2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '891005fa5ff74041b75747d16cc66798', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7047912c5acd4adeba5d87c3bf895e39', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80378) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:23:43.902895 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d050a0d-6813-48a5-b33c-e8487534ddb2 admin admin] REQUEST_METHOD: `GET` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:23:43.903174 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d050a0d-6813-48a5-b33c-e8487534ddb2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:23:43.903441 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d050a0d-6813-48a5-b33c-e8487534ddb2 admin admin] PATH_INFO: `/v3/domains` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:23:43.904128 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d050a0d-6813-48a5-b33c-e8487534ddb2 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80378) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 02 10:23:43.904434 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d050a0d-6813-48a5-b33c-e8487534ddb2 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:23:43.906048 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d050a0d-6813-48a5-b33c-e8487534ddb2 admin admin] RBAC: Authorization granted {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:23:43.917197 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 1246/2506] 10.222.0.82 () {58 vars in 1153 bytes} [Tue Aug 2 10:23:43 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 02 10:23:43.921774 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a695cb4-001f-4a24-9e1c-bb4747b70b7d None None] Authenticating user token {{(pid=80377) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:23:43.931897 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a695cb4-001f-4a24-9e1c-bb4747b70b7d None None] Validating token access rules against request {{(pid=80377) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:23:43.937409 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-224f9026-9965-413e-a7fc-ca7fbc8e3511 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '891005fa5ff74041b75747d16cc66798', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7047912c5acd4adeba5d87c3bf895e39', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80377) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:23:43.938087 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-224f9026-9965-413e-a7fc-ca7fbc8e3511 admin admin] REQUEST_METHOD: `POST` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:23:43.938489 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-224f9026-9965-413e-a7fc-ca7fbc8e3511 admin admin] SCRIPT_NAME: `/identity` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:23:43.938772 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-224f9026-9965-413e-a7fc-ca7fbc8e3511 admin admin] PATH_INFO: `/v3/projects` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:23:43.939513 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-224f9026-9965-413e-a7fc-ca7fbc8e3511 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:23:43.941428 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-224f9026-9965-413e-a7fc-ca7fbc8e3511 admin admin] RBAC: Authorization granted {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:23:43.961014 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG dbcounter [-] [80377] Writing DB stats keystone:SELECT=91,keystone:DELETE=7,keystone:INSERT=2 {{(pid=80377) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 10:23:43.976294 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: [pid: 80377|app: 0|req: 1261/2507] 10.222.0.82 () {60 vars in 1153 bytes} [Tue Aug 2 10:23:43 2022] POST /identity/v3/projects => generated 385 bytes in 55 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 02 10:23:43.980779 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d050a0d-6813-48a5-b33c-e8487534ddb2 admin admin] Authenticating user token {{(pid=80378) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:23:43.990977 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d050a0d-6813-48a5-b33c-e8487534ddb2 admin admin] Validating token access rules against request {{(pid=80378) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:23:44.046556 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b56edd2-15a8-4898-b620-647c865a5ae3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '891005fa5ff74041b75747d16cc66798', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7047912c5acd4adeba5d87c3bf895e39', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80378) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:23:44.047168 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b56edd2-15a8-4898-b620-647c865a5ae3 admin admin] REQUEST_METHOD: `POST` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:23:44.047446 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b56edd2-15a8-4898-b620-647c865a5ae3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:23:44.047716 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b56edd2-15a8-4898-b620-647c865a5ae3 admin admin] PATH_INFO: `/v3/users` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:23:44.048385 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b56edd2-15a8-4898-b620-647c865a5ae3 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:23:44.050321 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b56edd2-15a8-4898-b620-647c865a5ae3 admin admin] RBAC: Authorization granted {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:23:44.058032 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG dbcounter [-] [80378] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80378) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 10:23:44.074262 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 1247/2508] 10.222.0.82 () {60 vars in 1144 bytes} [Tue Aug 2 10:23:43 2022] POST /identity/v3/users => generated 293 bytes in 94 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 02 10:23:44.078423 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-224f9026-9965-413e-a7fc-ca7fbc8e3511 admin admin] Authenticating user token {{(pid=80377) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:23:44.089073 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-224f9026-9965-413e-a7fc-ca7fbc8e3511 admin admin] Validating token access rules against request {{(pid=80377) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:23:44.093767 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b059639-5c71-47f6-af01-5c16483e2409 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '891005fa5ff74041b75747d16cc66798', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7047912c5acd4adeba5d87c3bf895e39', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80377) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:23:44.094404 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b059639-5c71-47f6-af01-5c16483e2409 admin admin] REQUEST_METHOD: `GET` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:23:44.094689 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b059639-5c71-47f6-af01-5c16483e2409 admin admin] SCRIPT_NAME: `/identity` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:23:44.094965 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b059639-5c71-47f6-af01-5c16483e2409 admin admin] PATH_INFO: `/v3/roles` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:23:44.095710 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b059639-5c71-47f6-af01-5c16483e2409 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:23:44.097264 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b059639-5c71-47f6-af01-5c16483e2409 admin admin] RBAC: Authorization granted {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:23:44.107677 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: [pid: 80377|app: 0|req: 1262/2509] 10.222.0.82 () {58 vars in 1122 bytes} [Tue Aug 2 10:23:44 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 02 10:23:44.111544 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b56edd2-15a8-4898-b620-647c865a5ae3 admin admin] Authenticating user token {{(pid=80378) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:23:44.121900 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b56edd2-15a8-4898-b620-647c865a5ae3 admin admin] Validating token access rules against request {{(pid=80378) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:23:44.126431 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ac2f8bd-a8e5-49fd-8aa4-f5ae5740ce2e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '891005fa5ff74041b75747d16cc66798', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7047912c5acd4adeba5d87c3bf895e39', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80378) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:23:44.127032 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ac2f8bd-a8e5-49fd-8aa4-f5ae5740ce2e admin admin] REQUEST_METHOD: `GET` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:23:44.127315 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ac2f8bd-a8e5-49fd-8aa4-f5ae5740ce2e admin admin] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:23:44.127590 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ac2f8bd-a8e5-49fd-8aa4-f5ae5740ce2e admin admin] PATH_INFO: `/v3/roles` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:23:44.128283 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ac2f8bd-a8e5-49fd-8aa4-f5ae5740ce2e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:23:44.129938 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ac2f8bd-a8e5-49fd-8aa4-f5ae5740ce2e admin admin] RBAC: Authorization granted {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:23:44.140238 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 1248/2510] 10.222.0.82 () {58 vars in 1122 bytes} [Tue Aug 2 10:23:44 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 02 10:23:44.144130 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b059639-5c71-47f6-af01-5c16483e2409 admin admin] Authenticating user token {{(pid=80377) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:23:44.157566 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b059639-5c71-47f6-af01-5c16483e2409 admin admin] Validating token access rules against request {{(pid=80377) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:23:44.163141 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ced8ed50-c536-4c7a-8b28-adde27f64f07 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '891005fa5ff74041b75747d16cc66798', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7047912c5acd4adeba5d87c3bf895e39', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80377) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:23:44.163852 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ced8ed50-c536-4c7a-8b28-adde27f64f07 admin admin] REQUEST_METHOD: `PUT` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:23:44.164136 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ced8ed50-c536-4c7a-8b28-adde27f64f07 admin admin] SCRIPT_NAME: `/identity` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:23:44.164413 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ced8ed50-c536-4c7a-8b28-adde27f64f07 admin admin] PATH_INFO: `/v3/projects/9389c81a2710466ea8be45d2fe2342ef/users/8774306d95c0476b80cc8e53b62e8a4b/roles/e1c04b19fc4543b8b54f58bbcffbc598` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:23:44.180542 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ced8ed50-c536-4c7a-8b28-adde27f64f07 admin admin] RBAC: Authorizing `identity:create_grant(project_id=9389c81a2710466ea8be45d2fe2342ef, user_id=8774306d95c0476b80cc8e53b62e8a4b, role_id=e1c04b19fc4543b8b54f58bbcffbc598)` {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:23:44.183940 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ced8ed50-c536-4c7a-8b28-adde27f64f07 admin admin] RBAC: Authorization granted {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:23:44.200393 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: [pid: 80377|app: 0|req: 1263/2511] 10.222.0.82 () {60 vars in 1483 bytes} [Tue Aug 2 10:23:44 2022] PUT /identity/v3/projects/9389c81a2710466ea8be45d2fe2342ef/users/8774306d95c0476b80cc8e53b62e8a4b/roles/e1c04b19fc4543b8b54f58bbcffbc598 => generated 0 bytes in 57 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 02 10:23:44.205592 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da810ade-acbb-46fa-80ff-59fda064593d None None] REQUEST_METHOD: `POST` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:23:44.205923 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da810ade-acbb-46fa-80ff-59fda064593d None None] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:23:44.206205 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da810ade-acbb-46fa-80ff-59fda064593d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:23:44.224860 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.auth.core [None req-da810ade-acbb-46fa-80ff-59fda064593d None None] MFA Rules not processed for user `891005fa5ff74041b75747d16cc66798`. Rule list: `[]` (Enabled: `True`). {{(pid=80378) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 02 10:23:44.226102 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.fernet_utils [None req-da810ade-acbb-46fa-80ff-59fda064593d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80378) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 10:23:44.284236 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 1249/2512] 10.222.0.82 () {58 vars in 958 bytes} [Tue Aug 2 10:23:44 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 80 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 02 10:23:44.294665 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ced8ed50-c536-4c7a-8b28-adde27f64f07 admin admin] Authenticating user token {{(pid=80377) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:23:44.296380 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.fernet_utils [None req-ced8ed50-c536-4c7a-8b28-adde27f64f07 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80377) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 10:23:44.351836 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ced8ed50-c536-4c7a-8b28-adde27f64f07 admin admin] Validating token access rules against request {{(pid=80377) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:23:44.355992 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-104915cd-571a-4bd8-91fb-266c38052acb service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '33b87170f0634b67bce60750847b5cbe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f1b38ebb804c41c69772c998aae72c41', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80377) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:23:44.356607 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-104915cd-571a-4bd8-91fb-266c38052acb service neutron] REQUEST_METHOD: `GET` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:23:44.356887 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-104915cd-571a-4bd8-91fb-266c38052acb service neutron] SCRIPT_NAME: `/identity` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:23:44.357161 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-104915cd-571a-4bd8-91fb-266c38052acb service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:23:44.366287 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-104915cd-571a-4bd8-91fb-266c38052acb service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:23:44.367730 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-104915cd-571a-4bd8-91fb-266c38052acb service neutron] RBAC: Authorization granted {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:23:44.377314 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: [pid: 80377|app: 0|req: 1264/2513] 10.222.0.82 () {60 vars in 1365 bytes} [Tue Aug 2 10:23:44 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 83 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 02 10:23:45.149980 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da810ade-acbb-46fa-80ff-59fda064593d None None] Authenticating user token {{(pid=80378) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:23:45.161464 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da810ade-acbb-46fa-80ff-59fda064593d None None] Validating token access rules against request {{(pid=80378) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:23:45.165563 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffb65009-5302-4312-815c-cdc970d61049 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '33b87170f0634b67bce60750847b5cbe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f1b38ebb804c41c69772c998aae72c41', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80378) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:23:45.166317 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffb65009-5302-4312-815c-cdc970d61049 service neutron] REQUEST_METHOD: `GET` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:23:45.166698 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffb65009-5302-4312-815c-cdc970d61049 service neutron] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:23:45.167072 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffb65009-5302-4312-815c-cdc970d61049 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:23:45.169640 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.fernet_utils [None req-ffb65009-5302-4312-815c-cdc970d61049 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80378) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 10:23:45.249064 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ffb65009-5302-4312-815c-cdc970d61049 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:23:45.250809 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ffb65009-5302-4312-815c-cdc970d61049 service neutron] RBAC: Authorization granted {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:23:45.261544 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 1250/2514] 10.222.0.82 () {60 vars in 1365 bytes} [Tue Aug 2 10:23:45 2022] GET /identity/v3/auth/tokens => generated 3321 bytes in 112 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 02 10:23:49.234593 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-171d7d16-b08c-451c-ac85-ca7e24e5ad01 None None] REQUEST_METHOD: `POST` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:23:49.235144 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-171d7d16-b08c-451c-ac85-ca7e24e5ad01 None None] SCRIPT_NAME: `/identity` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:23:49.235299 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-171d7d16-b08c-451c-ac85-ca7e24e5ad01 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:23:49.264875 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.auth.core [None req-171d7d16-b08c-451c-ac85-ca7e24e5ad01 None None] MFA Rules not processed for user `8774306d95c0476b80cc8e53b62e8a4b`. Rule list: `[]` (Enabled: `True`). {{(pid=80377) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 02 10:23:49.266361 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.fernet_utils [None req-171d7d16-b08c-451c-ac85-ca7e24e5ad01 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80377) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 10:23:49.335598 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: [pid: 80377|app: 0|req: 1265/2515] 10.222.0.82 () {58 vars in 958 bytes} [Tue Aug 2 10:23:49 2022] POST /identity/v3/auth/tokens => generated 3262 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 02 10:23:49.349697 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffb65009-5302-4312-815c-cdc970d61049 service neutron] Authenticating user token {{(pid=80378) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:23:49.351281 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.fernet_utils [None req-ffb65009-5302-4312-815c-cdc970d61049 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80378) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 10:23:49.431992 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffb65009-5302-4312-815c-cdc970d61049 service neutron] Validating token access rules against request {{(pid=80378) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:23:49.438097 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfa540d8-1709-4073-b930-752de4878f5b service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88af518196674fe593ad88d30111fb7d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f1b38ebb804c41c69772c998aae72c41', 'project_domain_id': 'default', 'roles': ['admin', 'ResellerAdmin', 'reader', 'service', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80378) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:23:49.438722 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfa540d8-1709-4073-b930-752de4878f5b service nova] REQUEST_METHOD: `GET` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:23:49.439013 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfa540d8-1709-4073-b930-752de4878f5b service nova] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:23:49.439290 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfa540d8-1709-4073-b930-752de4878f5b service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:23:49.447204 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dfa540d8-1709-4073-b930-752de4878f5b service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:23:49.448787 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dfa540d8-1709-4073-b930-752de4878f5b service nova] RBAC: Authorization granted {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:23:49.452832 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG dbcounter [-] [80378] Writing DB stats keystone:INSERT=3,keystone:SELECT=97 {{(pid=80378) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 10:23:49.458916 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 1251/2516] 10.222.0.82 () {60 vars in 1365 bytes} [Tue Aug 2 10:23:49 2022] GET /identity/v3/auth/tokens => generated 3262 bytes in 110 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 02 10:23:50.951425 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-881c5e81-b8ea-410a-bc07-10bd26d16994 None None] REQUEST_METHOD: `POST` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:23:50.951886 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-881c5e81-b8ea-410a-bc07-10bd26d16994 None None] SCRIPT_NAME: `/identity` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:23:50.952102 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-881c5e81-b8ea-410a-bc07-10bd26d16994 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:23:50.972595 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.auth.core [None req-881c5e81-b8ea-410a-bc07-10bd26d16994 None None] MFA Rules not processed for user `8774306d95c0476b80cc8e53b62e8a4b`. Rule list: `[]` (Enabled: `True`). {{(pid=80377) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 02 10:23:50.974036 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.fernet_utils [None req-881c5e81-b8ea-410a-bc07-10bd26d16994 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80377) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 10:23:50.986111 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: [pid: 80377|app: 0|req: 1266/2517] 10.222.0.82 () {58 vars in 958 bytes} [Tue Aug 2 10:23:50 2022] POST /identity/v3/auth/tokens => generated 3262 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 02 10:23:59.461617 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG dbcounter [-] [80378] Writing DB stats keystone:SELECT=1 {{(pid=80378) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 10:24:00.966543 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG dbcounter [-] [80377] Writing DB stats keystone:SELECT=78,keystone:INSERT=1 {{(pid=80377) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 10:24:38.644110 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d2dda46-833d-43ba-9006-6c6196f26b9b None None] REQUEST_METHOD: `POST` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:24:38.644628 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d2dda46-833d-43ba-9006-6c6196f26b9b None None] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:24:38.644740 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d2dda46-833d-43ba-9006-6c6196f26b9b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:24:38.683100 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.auth.core [None req-5d2dda46-833d-43ba-9006-6c6196f26b9b None None] MFA Rules not processed for user `bd8dd72086db48b8aaac2ed0058872df`. Rule list: `[]` (Enabled: `True`). {{(pid=80378) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 02 10:24:38.684334 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.fernet_utils [None req-5d2dda46-833d-43ba-9006-6c6196f26b9b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80378) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 10:24:38.765659 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 1252/2518] 10.222.0.82 () {58 vars in 1009 bytes} [Tue Aug 2 10:24:38 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 122 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 02 10:24:38.812929 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-881c5e81-b8ea-410a-bc07-10bd26d16994 None None] Authenticating user token {{(pid=80377) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:24:38.814413 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.fernet_utils [None req-881c5e81-b8ea-410a-bc07-10bd26d16994 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80377) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 10:24:38.924726 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-881c5e81-b8ea-410a-bc07-10bd26d16994 None None] Validating token access rules against request {{(pid=80377) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:24:38.929751 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7080064-2480-4a58-8512-4451f3aea27d service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bef1b2e714cb45c4a51eb6d3e15aad6d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f1b38ebb804c41c69772c998aae72c41', 'project_domain_id': 'default', 'roles': ['admin', 'service', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80377) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:24:38.930393 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7080064-2480-4a58-8512-4451f3aea27d service swift] REQUEST_METHOD: `GET` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:24:38.930671 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7080064-2480-4a58-8512-4451f3aea27d service swift] SCRIPT_NAME: `/identity` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:24:38.930941 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7080064-2480-4a58-8512-4451f3aea27d service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:24:38.939361 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7080064-2480-4a58-8512-4451f3aea27d service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:24:38.940934 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7080064-2480-4a58-8512-4451f3aea27d service swift] RBAC: Authorization granted {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:24:38.948942 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: [pid: 80377|app: 0|req: 1267/2519] 10.222.0.82 () {60 vars in 1384 bytes} [Tue Aug 2 10:24:38 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 137 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Aug 02 10:24:48.763346 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG dbcounter [-] [80378] Writing DB stats keystone:SELECT=45 {{(pid=80378) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 10:24:48.831915 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d2dda46-833d-43ba-9006-6c6196f26b9b None None] Authenticating user token {{(pid=80378) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:24:48.834022 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.fernet_utils [None req-5d2dda46-833d-43ba-9006-6c6196f26b9b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80378) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 10:24:48.945937 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG dbcounter [-] [80377] Writing DB stats keystone:SELECT=47 {{(pid=80377) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 10:24:48.995294 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d2dda46-833d-43ba-9006-6c6196f26b9b None None] Validating token access rules against request {{(pid=80378) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:24:49.003811 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83cbd0d1-f56b-440f-993e-619dd28ca0fe service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ae6e910e50d74f29bfb89bf18d95a908', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f1b38ebb804c41c69772c998aae72c41', 'project_domain_id': 'default', 'roles': ['admin', 'service', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80378) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:24:49.003811 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83cbd0d1-f56b-440f-993e-619dd28ca0fe service placement] REQUEST_METHOD: `GET` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:24:49.003811 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83cbd0d1-f56b-440f-993e-619dd28ca0fe service placement] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:24:49.003811 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83cbd0d1-f56b-440f-993e-619dd28ca0fe service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:24:49.004550 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.fernet_utils [None req-83cbd0d1-f56b-440f-993e-619dd28ca0fe service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80378) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 10:24:49.019720 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83cbd0d1-f56b-440f-993e-619dd28ca0fe service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:24:49.021517 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83cbd0d1-f56b-440f-993e-619dd28ca0fe service placement] RBAC: Authorization granted {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:24:49.032541 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 1253/2520] 10.222.0.82 () {60 vars in 1365 bytes} [Tue Aug 2 10:24:48 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 201 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 02 10:24:50.213385 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7080064-2480-4a58-8512-4451f3aea27d service swift] Authenticating user token {{(pid=80377) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:24:50.238675 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7080064-2480-4a58-8512-4451f3aea27d service swift] Validating token access rules against request {{(pid=80377) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:24:50.242326 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6de863b-20a8-4c1d-bc88-b6698a7d31fa service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '33b87170f0634b67bce60750847b5cbe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f1b38ebb804c41c69772c998aae72c41', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80377) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:24:50.242951 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6de863b-20a8-4c1d-bc88-b6698a7d31fa service neutron] REQUEST_METHOD: `GET` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:24:50.243236 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6de863b-20a8-4c1d-bc88-b6698a7d31fa service neutron] SCRIPT_NAME: `/identity` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:24:50.243513 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6de863b-20a8-4c1d-bc88-b6698a7d31fa service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:24:50.245255 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.fernet_utils [None req-e6de863b-20a8-4c1d-bc88-b6698a7d31fa service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80377) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 10:24:50.280160 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6de863b-20a8-4c1d-bc88-b6698a7d31fa service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:24:50.281604 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6de863b-20a8-4c1d-bc88-b6698a7d31fa service neutron] RBAC: Authorization granted {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:24:50.301037 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: [pid: 80377|app: 0|req: 1268/2521] 10.222.0.82 () {60 vars in 1365 bytes} [Tue Aug 2 10:24:50 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 88 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 02 10:24:54.035143 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83cbd0d1-f56b-440f-993e-619dd28ca0fe service placement] Authenticating user token {{(pid=80378) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:24:54.046321 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83cbd0d1-f56b-440f-993e-619dd28ca0fe service placement] Validating token access rules against request {{(pid=80378) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:24:54.051688 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b76f626-1403-40b9-ac1e-6826ccb939cd service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ae6e910e50d74f29bfb89bf18d95a908', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f1b38ebb804c41c69772c998aae72c41', 'project_domain_id': 'default', 'roles': ['admin', 'service', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80378) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:24:54.052307 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b76f626-1403-40b9-ac1e-6826ccb939cd service placement] REQUEST_METHOD: `GET` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:24:54.052919 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b76f626-1403-40b9-ac1e-6826ccb939cd service placement] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:24:54.052919 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b76f626-1403-40b9-ac1e-6826ccb939cd service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:24:54.054777 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.fernet_utils [None req-6b76f626-1403-40b9-ac1e-6826ccb939cd service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80378) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 10:24:54.067174 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b76f626-1403-40b9-ac1e-6826ccb939cd service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:24:54.068783 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b76f626-1403-40b9-ac1e-6826ccb939cd service placement] RBAC: Authorization granted {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:24:54.080388 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 1254/2522] 10.222.0.82 () {60 vars in 1365 bytes} [Tue Aug 2 10:24:54 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 46 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 02 10:24:57.324313 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6de863b-20a8-4c1d-bc88-b6698a7d31fa service neutron] Authenticating user token {{(pid=80377) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:24:57.325856 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.fernet_utils [None req-e6de863b-20a8-4c1d-bc88-b6698a7d31fa service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80377) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 10:24:57.338388 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6de863b-20a8-4c1d-bc88-b6698a7d31fa service neutron] Validating token access rules against request {{(pid=80377) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:24:57.343575 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c1edfb9-6c8f-4e2a-af73-f7b3355e7a6d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '891005fa5ff74041b75747d16cc66798', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7047912c5acd4adeba5d87c3bf895e39', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80377) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:24:57.344225 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c1edfb9-6c8f-4e2a-af73-f7b3355e7a6d admin admin] REQUEST_METHOD: `DELETE` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:24:57.344510 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c1edfb9-6c8f-4e2a-af73-f7b3355e7a6d admin admin] SCRIPT_NAME: `/identity` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:24:57.344788 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c1edfb9-6c8f-4e2a-af73-f7b3355e7a6d admin admin] PATH_INFO: `/v3/users/7b672b80961445c3a3a9b2d784f64208` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:24:57.346020 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c1edfb9-6c8f-4e2a-af73-f7b3355e7a6d admin admin] RBAC: Authorizing `identity:delete_user(user_id=7b672b80961445c3a3a9b2d784f64208)` {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:24:57.348009 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c1edfb9-6c8f-4e2a-af73-f7b3355e7a6d admin admin] RBAC: Authorization granted {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:24:57.394628 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.notifications [None req-2c1edfb9-6c8f-4e2a-af73-f7b3355e7a6d admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '7b672b80961445c3a3a9b2d784f64208'} {{(pid=80377) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 10:24:57.402803 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.notifications [None req-2c1edfb9-6c8f-4e2a-af73-f7b3355e7a6d admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '7b672b80961445c3a3a9b2d784f64208'} {{(pid=80377) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 10:24:57.410333 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.notifications [None req-2c1edfb9-6c8f-4e2a-af73-f7b3355e7a6d admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '7b672b80961445c3a3a9b2d784f64208'} {{(pid=80377) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 10:24:57.410984 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.notifications [None req-2c1edfb9-6c8f-4e2a-af73-f7b3355e7a6d admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '7b672b80961445c3a3a9b2d784f64208'} {{(pid=80377) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 10:24:57.423404 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.notifications [None req-2c1edfb9-6c8f-4e2a-af73-f7b3355e7a6d admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '7b672b80961445c3a3a9b2d784f64208'} {{(pid=80377) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 10:24:57.429264 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: [pid: 80377|app: 0|req: 1269/2523] 10.222.0.82 () {58 vars in 1224 bytes} [Tue Aug 2 10:24:57 2022] DELETE /identity/v3/users/7b672b80961445c3a3a9b2d784f64208 => generated 0 bytes in 106 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 02 10:24:57.626768 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b76f626-1403-40b9-ac1e-6826ccb939cd service placement] Authenticating user token {{(pid=80378) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:24:57.640289 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b76f626-1403-40b9-ac1e-6826ccb939cd service placement] Validating token access rules against request {{(pid=80378) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:24:57.709559 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa938ea0-82e9-4e64-bba4-1df9a8967846 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '891005fa5ff74041b75747d16cc66798', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7047912c5acd4adeba5d87c3bf895e39', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80378) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:24:57.710240 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa938ea0-82e9-4e64-bba4-1df9a8967846 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:24:57.710525 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa938ea0-82e9-4e64-bba4-1df9a8967846 admin admin] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:24:57.711046 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa938ea0-82e9-4e64-bba4-1df9a8967846 admin admin] PATH_INFO: `/v3/projects/99a92325ab3d4ec2bbecffb9a6e609d5` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:24:57.712254 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa938ea0-82e9-4e64-bba4-1df9a8967846 admin admin] RBAC: Authorizing `identity:delete_project(project_id=99a92325ab3d4ec2bbecffb9a6e609d5)` {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:24:57.714275 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa938ea0-82e9-4e64-bba4-1df9a8967846 admin admin] RBAC: Authorization granted {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:24:57.773913 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.notifications [None req-fa938ea0-82e9-4e64-bba4-1df9a8967846 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '99a92325ab3d4ec2bbecffb9a6e609d5', 'request_id': 'req-fa938ea0-82e9-4e64-bba4-1df9a8967846'} {{(pid=80378) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 10:24:57.780721 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.notifications [None req-fa938ea0-82e9-4e64-bba4-1df9a8967846 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '99a92325ab3d4ec2bbecffb9a6e609d5', 'request_id': 'req-fa938ea0-82e9-4e64-bba4-1df9a8967846'} {{(pid=80378) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 10:24:57.784976 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG dbcounter [-] [80378] Writing DB stats keystone:SELECT=95,keystone:DELETE=5 {{(pid=80378) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 10:24:57.787421 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.notifications [None req-fa938ea0-82e9-4e64-bba4-1df9a8967846 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '99a92325ab3d4ec2bbecffb9a6e609d5', 'request_id': 'req-fa938ea0-82e9-4e64-bba4-1df9a8967846'} {{(pid=80378) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 10:24:57.809543 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.notifications [None req-fa938ea0-82e9-4e64-bba4-1df9a8967846 admin admin] The token cache is being invalidate because project 99a92325ab3d4ec2bbecffb9a6e609d5 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80378) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 02 10:24:57.809883 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.notifications [None req-fa938ea0-82e9-4e64-bba4-1df9a8967846 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80378) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 10:24:57.811265 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.notifications [None req-fa938ea0-82e9-4e64-bba4-1df9a8967846 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80378) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 10:24:57.813828 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 1255/2524] 10.222.0.82 () {58 vars in 1233 bytes} [Tue Aug 2 10:24:57 2022] DELETE /identity/v3/projects/99a92325ab3d4ec2bbecffb9a6e609d5 => generated 0 bytes in 186 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 02 10:24:57.816717 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c1edfb9-6c8f-4e2a-af73-f7b3355e7a6d admin admin] Authenticating user token {{(pid=80377) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:24:57.818935 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.fernet_utils [None req-2c1edfb9-6c8f-4e2a-af73-f7b3355e7a6d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80377) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 10:24:57.836044 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c1edfb9-6c8f-4e2a-af73-f7b3355e7a6d admin admin] Validating token access rules against request {{(pid=80377) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:24:57.910329 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-451bea1e-eabd-4447-8921-6b8a71943d64 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '891005fa5ff74041b75747d16cc66798', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7047912c5acd4adeba5d87c3bf895e39', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80377) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:24:57.911153 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-451bea1e-eabd-4447-8921-6b8a71943d64 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:24:57.912004 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-451bea1e-eabd-4447-8921-6b8a71943d64 admin admin] SCRIPT_NAME: `/identity` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:24:57.912534 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-451bea1e-eabd-4447-8921-6b8a71943d64 admin admin] PATH_INFO: `/v3/users/3d378bdfcbc5436f81a0e3ad04822ebe` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:24:57.914162 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-451bea1e-eabd-4447-8921-6b8a71943d64 admin admin] RBAC: Authorizing `identity:delete_user(user_id=3d378bdfcbc5436f81a0e3ad04822ebe)` {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:24:57.916323 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-451bea1e-eabd-4447-8921-6b8a71943d64 admin admin] RBAC: Authorization granted {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:24:57.943487 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG dbcounter [-] [80377] Writing DB stats keystone:SELECT=85,keystone:DELETE=14,keystone:INSERT=1 {{(pid=80377) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 10:24:57.972964 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.notifications [None req-451bea1e-eabd-4447-8921-6b8a71943d64 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '3d378bdfcbc5436f81a0e3ad04822ebe'} {{(pid=80377) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 10:24:57.981019 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.notifications [None req-451bea1e-eabd-4447-8921-6b8a71943d64 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '3d378bdfcbc5436f81a0e3ad04822ebe'} {{(pid=80377) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 10:24:57.989013 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.notifications [None req-451bea1e-eabd-4447-8921-6b8a71943d64 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '3d378bdfcbc5436f81a0e3ad04822ebe'} {{(pid=80377) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 10:24:57.990308 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.notifications [None req-451bea1e-eabd-4447-8921-6b8a71943d64 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '3d378bdfcbc5436f81a0e3ad04822ebe'} {{(pid=80377) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 10:24:58.003717 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.notifications [None req-451bea1e-eabd-4447-8921-6b8a71943d64 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '3d378bdfcbc5436f81a0e3ad04822ebe'} {{(pid=80377) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 10:24:58.009912 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: [pid: 80377|app: 0|req: 1270/2525] 10.222.0.82 () {58 vars in 1224 bytes} [Tue Aug 2 10:24:57 2022] DELETE /identity/v3/users/3d378bdfcbc5436f81a0e3ad04822ebe => generated 0 bytes in 194 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 02 10:24:58.188408 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa938ea0-82e9-4e64-bba4-1df9a8967846 admin admin] Authenticating user token {{(pid=80378) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:24:58.190482 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.fernet_utils [None req-fa938ea0-82e9-4e64-bba4-1df9a8967846 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80378) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 10:24:58.209929 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa938ea0-82e9-4e64-bba4-1df9a8967846 admin admin] Validating token access rules against request {{(pid=80378) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:24:58.276990 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78906b5a-bb45-428f-9af5-e860da33fc2c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '891005fa5ff74041b75747d16cc66798', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7047912c5acd4adeba5d87c3bf895e39', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80378) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:24:58.277955 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78906b5a-bb45-428f-9af5-e860da33fc2c admin admin] REQUEST_METHOD: `DELETE` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:24:58.278401 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78906b5a-bb45-428f-9af5-e860da33fc2c admin admin] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:24:58.278796 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78906b5a-bb45-428f-9af5-e860da33fc2c admin admin] PATH_INFO: `/v3/projects/e12bfff080094a6ca25073c5f68987e1` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:24:58.280238 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78906b5a-bb45-428f-9af5-e860da33fc2c admin admin] RBAC: Authorizing `identity:delete_project(project_id=e12bfff080094a6ca25073c5f68987e1)` {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:24:58.282315 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78906b5a-bb45-428f-9af5-e860da33fc2c admin admin] RBAC: Authorization granted {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:24:58.332584 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.notifications [None req-78906b5a-bb45-428f-9af5-e860da33fc2c admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'e12bfff080094a6ca25073c5f68987e1', 'request_id': 'req-78906b5a-bb45-428f-9af5-e860da33fc2c'} {{(pid=80378) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 10:24:58.338795 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.notifications [None req-78906b5a-bb45-428f-9af5-e860da33fc2c admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'e12bfff080094a6ca25073c5f68987e1', 'request_id': 'req-78906b5a-bb45-428f-9af5-e860da33fc2c'} {{(pid=80378) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 10:24:58.344686 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.notifications [None req-78906b5a-bb45-428f-9af5-e860da33fc2c admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'e12bfff080094a6ca25073c5f68987e1', 'request_id': 'req-78906b5a-bb45-428f-9af5-e860da33fc2c'} {{(pid=80378) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 10:24:58.357397 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.notifications [None req-78906b5a-bb45-428f-9af5-e860da33fc2c admin admin] The token cache is being invalidate because project e12bfff080094a6ca25073c5f68987e1 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80378) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 02 10:24:58.357677 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.notifications [None req-78906b5a-bb45-428f-9af5-e860da33fc2c admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80378) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 10:24:58.358335 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.notifications [None req-78906b5a-bb45-428f-9af5-e860da33fc2c admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80378) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 10:24:58.359845 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 1256/2526] 10.222.0.82 () {58 vars in 1233 bytes} [Tue Aug 2 10:24:58 2022] DELETE /identity/v3/projects/e12bfff080094a6ca25073c5f68987e1 => generated 0 bytes in 172 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 02 10:24:58.532869 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b49d752b-ee5d-46ea-b3f2-b8fadc943b96 None None] REQUEST_METHOD: `POST` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:24:58.533432 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b49d752b-ee5d-46ea-b3f2-b8fadc943b96 None None] SCRIPT_NAME: `/identity` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:24:58.533708 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b49d752b-ee5d-46ea-b3f2-b8fadc943b96 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:24:58.556093 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.auth.core [None req-b49d752b-ee5d-46ea-b3f2-b8fadc943b96 None None] MFA Rules not processed for user `bd8dd72086db48b8aaac2ed0058872df`. Rule list: `[]` (Enabled: `True`). {{(pid=80377) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 02 10:24:58.558042 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.fernet_utils [None req-b49d752b-ee5d-46ea-b3f2-b8fadc943b96 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80377) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 10:24:58.624221 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: [pid: 80377|app: 0|req: 1271/2527] 10.222.0.82 () {58 vars in 1009 bytes} [Tue Aug 2 10:24:58 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 92 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 02 10:24:58.637207 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78906b5a-bb45-428f-9af5-e860da33fc2c admin admin] Authenticating user token {{(pid=80378) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:24:58.638691 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.fernet_utils [None req-78906b5a-bb45-428f-9af5-e860da33fc2c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80378) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 10:24:58.707584 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78906b5a-bb45-428f-9af5-e860da33fc2c admin admin] Validating token access rules against request {{(pid=80378) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:24:58.712500 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28182698-4c20-4f9b-a9f4-a8f82324ae66 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bef1b2e714cb45c4a51eb6d3e15aad6d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f1b38ebb804c41c69772c998aae72c41', 'project_domain_id': 'default', 'roles': ['admin', 'service', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80378) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:24:58.713123 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28182698-4c20-4f9b-a9f4-a8f82324ae66 service swift] REQUEST_METHOD: `GET` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:24:58.713411 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28182698-4c20-4f9b-a9f4-a8f82324ae66 service swift] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:24:58.713715 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28182698-4c20-4f9b-a9f4-a8f82324ae66 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:24:58.721858 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28182698-4c20-4f9b-a9f4-a8f82324ae66 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:24:58.723446 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28182698-4c20-4f9b-a9f4-a8f82324ae66 service swift] RBAC: Authorization granted {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:24:58.731358 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 1257/2528] 10.222.0.82 () {60 vars in 1384 bytes} [Tue Aug 2 10:24:58 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 95 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Aug 02 10:25:08.616514 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG dbcounter [-] [80377] Writing DB stats keystone:SELECT=45,keystone:DELETE=6,keystone:INSERT=1 {{(pid=80377) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 10:25:08.728146 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG dbcounter [-] [80378] Writing DB stats keystone:DELETE=9,keystone:SELECT=86,keystone:INSERT=2 {{(pid=80378) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 10:25:20.765955 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6ef11b7-a7b0-4eb2-92fd-c4810dcb9e93 None None] REQUEST_METHOD: `POST` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:25:20.766413 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6ef11b7-a7b0-4eb2-92fd-c4810dcb9e93 None None] SCRIPT_NAME: `/identity` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:25:20.766554 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6ef11b7-a7b0-4eb2-92fd-c4810dcb9e93 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:25:20.785446 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.auth.core [None req-a6ef11b7-a7b0-4eb2-92fd-c4810dcb9e93 None None] MFA Rules not processed for user `bd8dd72086db48b8aaac2ed0058872df`. Rule list: `[]` (Enabled: `True`). {{(pid=80377) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 02 10:25:20.786677 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.fernet_utils [None req-a6ef11b7-a7b0-4eb2-92fd-c4810dcb9e93 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80377) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 10:25:20.794480 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: [pid: 80377|app: 0|req: 1272/2529] 10.222.0.82 () {58 vars in 967 bytes} [Tue Aug 2 10:25:20 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 02 10:25:20.806577 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28182698-4c20-4f9b-a9f4-a8f82324ae66 service swift] Authenticating user token {{(pid=80378) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:25:20.817585 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28182698-4c20-4f9b-a9f4-a8f82324ae66 service swift] Validating token access rules against request {{(pid=80378) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:25:20.822879 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c845710-eaa7-4808-977a-a1c2e4977489 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bef1b2e714cb45c4a51eb6d3e15aad6d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f1b38ebb804c41c69772c998aae72c41', 'project_domain_id': 'default', 'roles': ['admin', 'service', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80378) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:25:20.823491 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c845710-eaa7-4808-977a-a1c2e4977489 service swift] REQUEST_METHOD: `GET` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:25:20.823777 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c845710-eaa7-4808-977a-a1c2e4977489 service swift] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:25:20.824108 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c845710-eaa7-4808-977a-a1c2e4977489 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:25:20.832031 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c845710-eaa7-4808-977a-a1c2e4977489 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:25:20.833594 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c845710-eaa7-4808-977a-a1c2e4977489 service swift] RBAC: Authorization granted {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:25:20.841540 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 1258/2530] 10.222.0.82 () {60 vars in 1384 bytes} [Tue Aug 2 10:25:20 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 35 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Aug 02 10:25:30.780230 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG dbcounter [-] [80377] Writing DB stats keystone:SELECT=3 {{(pid=80377) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 10:25:30.838273 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG dbcounter [-] [80378] Writing DB stats keystone:SELECT=6 {{(pid=80378) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 10:25:45.523216 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6ef11b7-a7b0-4eb2-92fd-c4810dcb9e93 None None] Authenticating user token {{(pid=80377) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:25:45.524779 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.fernet_utils [None req-a6ef11b7-a7b0-4eb2-92fd-c4810dcb9e93 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80377) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 10:25:45.593158 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6ef11b7-a7b0-4eb2-92fd-c4810dcb9e93 None None] Validating token access rules against request {{(pid=80377) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:25:45.597247 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2b47217-35fe-4ecd-b8eb-b38df7af9983 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '33b87170f0634b67bce60750847b5cbe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f1b38ebb804c41c69772c998aae72c41', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80377) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:25:45.597898 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2b47217-35fe-4ecd-b8eb-b38df7af9983 service neutron] REQUEST_METHOD: `GET` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:25:45.598181 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2b47217-35fe-4ecd-b8eb-b38df7af9983 service neutron] SCRIPT_NAME: `/identity` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:25:45.598455 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2b47217-35fe-4ecd-b8eb-b38df7af9983 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:25:45.600186 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.fernet_utils [None req-f2b47217-35fe-4ecd-b8eb-b38df7af9983 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80377) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 10:25:45.662075 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2b47217-35fe-4ecd-b8eb-b38df7af9983 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:25:45.663551 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2b47217-35fe-4ecd-b8eb-b38df7af9983 service neutron] RBAC: Authorization granted {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:25:45.673147 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: [pid: 80377|app: 0|req: 1273/2531] 10.222.0.82 () {60 vars in 1365 bytes} [Tue Aug 2 10:25:45 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 150 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 02 10:25:48.768728 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c845710-eaa7-4808-977a-a1c2e4977489 service swift] Authenticating user token {{(pid=80378) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:25:48.770286 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.fernet_utils [None req-0c845710-eaa7-4808-977a-a1c2e4977489 service swift] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80378) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 10:25:48.785658 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c845710-eaa7-4808-977a-a1c2e4977489 service swift] Validating token access rules against request {{(pid=80378) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:25:48.791620 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fcd78c8-9088-4a0c-8628-8fcc9b5f2594 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '891005fa5ff74041b75747d16cc66798', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7047912c5acd4adeba5d87c3bf895e39', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80378) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:25:48.792286 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fcd78c8-9088-4a0c-8628-8fcc9b5f2594 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:25:48.792575 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fcd78c8-9088-4a0c-8628-8fcc9b5f2594 admin admin] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:25:48.792887 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fcd78c8-9088-4a0c-8628-8fcc9b5f2594 admin admin] PATH_INFO: `/v3/users/e881ae7c60254c12860fff0a40480641` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:25:48.794090 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9fcd78c8-9088-4a0c-8628-8fcc9b5f2594 admin admin] RBAC: Authorizing `identity:delete_user(user_id=e881ae7c60254c12860fff0a40480641)` {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:25:48.796064 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9fcd78c8-9088-4a0c-8628-8fcc9b5f2594 admin admin] RBAC: Authorization granted {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:25:48.846064 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.notifications [None req-9fcd78c8-9088-4a0c-8628-8fcc9b5f2594 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'e881ae7c60254c12860fff0a40480641'} {{(pid=80378) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 10:25:48.858571 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.notifications [None req-9fcd78c8-9088-4a0c-8628-8fcc9b5f2594 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'e881ae7c60254c12860fff0a40480641'} {{(pid=80378) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 10:25:48.866011 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.notifications [None req-9fcd78c8-9088-4a0c-8628-8fcc9b5f2594 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'e881ae7c60254c12860fff0a40480641'} {{(pid=80378) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 10:25:48.866698 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.notifications [None req-9fcd78c8-9088-4a0c-8628-8fcc9b5f2594 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'e881ae7c60254c12860fff0a40480641'} {{(pid=80378) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 10:25:48.880673 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.notifications [None req-9fcd78c8-9088-4a0c-8628-8fcc9b5f2594 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'e881ae7c60254c12860fff0a40480641'} {{(pid=80378) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 10:25:48.887018 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 1259/2532] 10.222.0.82 () {58 vars in 1224 bytes} [Tue Aug 2 10:25:48 2022] DELETE /identity/v3/users/e881ae7c60254c12860fff0a40480641 => generated 0 bytes in 119 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 02 10:25:49.328269 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2b47217-35fe-4ecd-b8eb-b38df7af9983 service neutron] Authenticating user token {{(pid=80377) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:25:49.344043 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2b47217-35fe-4ecd-b8eb-b38df7af9983 service neutron] Validating token access rules against request {{(pid=80377) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:25:49.407114 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-883f7909-db02-4cbd-ba31-ae31828a6c88 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '891005fa5ff74041b75747d16cc66798', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7047912c5acd4adeba5d87c3bf895e39', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80377) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:25:49.407924 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-883f7909-db02-4cbd-ba31-ae31828a6c88 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:25:49.408242 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-883f7909-db02-4cbd-ba31-ae31828a6c88 admin admin] SCRIPT_NAME: `/identity` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:25:49.408661 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-883f7909-db02-4cbd-ba31-ae31828a6c88 admin admin] PATH_INFO: `/v3/projects/4bc7ee5e1e7f4f5c8d9de8fa1e48991b` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:25:49.409885 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-883f7909-db02-4cbd-ba31-ae31828a6c88 admin admin] RBAC: Authorizing `identity:delete_project(project_id=4bc7ee5e1e7f4f5c8d9de8fa1e48991b)` {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:25:49.411899 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-883f7909-db02-4cbd-ba31-ae31828a6c88 admin admin] RBAC: Authorization granted {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:25:49.461526 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG dbcounter [-] [80377] Writing DB stats keystone:SELECT=97,keystone:DELETE=3 {{(pid=80377) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 10:25:49.470340 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.notifications [None req-883f7909-db02-4cbd-ba31-ae31828a6c88 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '4bc7ee5e1e7f4f5c8d9de8fa1e48991b', 'request_id': 'req-883f7909-db02-4cbd-ba31-ae31828a6c88'} {{(pid=80377) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 10:25:49.476507 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.notifications [None req-883f7909-db02-4cbd-ba31-ae31828a6c88 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '4bc7ee5e1e7f4f5c8d9de8fa1e48991b', 'request_id': 'req-883f7909-db02-4cbd-ba31-ae31828a6c88'} {{(pid=80377) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 10:25:49.486137 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.notifications [None req-883f7909-db02-4cbd-ba31-ae31828a6c88 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '4bc7ee5e1e7f4f5c8d9de8fa1e48991b', 'request_id': 'req-883f7909-db02-4cbd-ba31-ae31828a6c88'} {{(pid=80377) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 10:25:49.521963 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.notifications [None req-883f7909-db02-4cbd-ba31-ae31828a6c88 admin admin] The token cache is being invalidate because project 4bc7ee5e1e7f4f5c8d9de8fa1e48991b was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80377) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 02 10:25:49.522536 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.notifications [None req-883f7909-db02-4cbd-ba31-ae31828a6c88 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80377) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 10:25:49.523612 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.notifications [None req-883f7909-db02-4cbd-ba31-ae31828a6c88 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80377) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 10:25:49.526837 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: [pid: 80377|app: 0|req: 1274/2533] 10.222.0.82 () {58 vars in 1233 bytes} [Tue Aug 2 10:25:49 2022] DELETE /identity/v3/projects/4bc7ee5e1e7f4f5c8d9de8fa1e48991b => generated 0 bytes in 199 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 02 10:25:49.531594 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca8c5dee-7f64-40e7-a1be-d6d8d9fc6cd4 None None] REQUEST_METHOD: `POST` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:25:49.531890 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca8c5dee-7f64-40e7-a1be-d6d8d9fc6cd4 None None] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:25:49.532224 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca8c5dee-7f64-40e7-a1be-d6d8d9fc6cd4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:25:49.609049 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.auth.core [None req-ca8c5dee-7f64-40e7-a1be-d6d8d9fc6cd4 None None] MFA Rules not processed for user `891005fa5ff74041b75747d16cc66798`. Rule list: `[]` (Enabled: `True`). {{(pid=80378) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 02 10:25:49.610353 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.fernet_utils [None req-ca8c5dee-7f64-40e7-a1be-d6d8d9fc6cd4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80378) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 10:25:49.674822 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 1260/2534] 10.222.0.82 () {58 vars in 958 bytes} [Tue Aug 2 10:25:49 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 144 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 02 10:25:49.682751 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0354870-88d6-4845-bdef-e679ca6b1e74 None None] REQUEST_METHOD: `POST` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:25:49.683614 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0354870-88d6-4845-bdef-e679ca6b1e74 None None] SCRIPT_NAME: `/identity` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:25:49.683984 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0354870-88d6-4845-bdef-e679ca6b1e74 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:25:49.704929 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.auth.core [None req-f0354870-88d6-4845-bdef-e679ca6b1e74 None None] MFA Rules not processed for user `891005fa5ff74041b75747d16cc66798`. Rule list: `[]` (Enabled: `True`). {{(pid=80377) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 02 10:25:49.706272 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.fernet_utils [None req-f0354870-88d6-4845-bdef-e679ca6b1e74 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80377) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 10:25:49.715392 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: [pid: 80377|app: 0|req: 1275/2535] 10.222.0.82 () {58 vars in 958 bytes} [Tue Aug 2 10:25:49 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 34 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 02 10:25:49.719772 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca8c5dee-7f64-40e7-a1be-d6d8d9fc6cd4 None None] Authenticating user token {{(pid=80378) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:25:49.732635 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca8c5dee-7f64-40e7-a1be-d6d8d9fc6cd4 None None] Validating token access rules against request {{(pid=80378) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:25:49.737312 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0ec905a-6d23-480f-bf8d-62a8b6ce0507 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '891005fa5ff74041b75747d16cc66798', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7047912c5acd4adeba5d87c3bf895e39', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80378) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:25:49.737961 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0ec905a-6d23-480f-bf8d-62a8b6ce0507 admin admin] REQUEST_METHOD: `GET` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:25:49.738243 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0ec905a-6d23-480f-bf8d-62a8b6ce0507 admin admin] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:25:49.738513 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0ec905a-6d23-480f-bf8d-62a8b6ce0507 admin admin] PATH_INFO: `/v3/domains` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:25:49.739206 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0ec905a-6d23-480f-bf8d-62a8b6ce0507 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80378) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 02 10:25:49.739516 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0ec905a-6d23-480f-bf8d-62a8b6ce0507 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:25:49.741101 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0ec905a-6d23-480f-bf8d-62a8b6ce0507 admin admin] RBAC: Authorization granted {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:25:49.756516 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 1261/2536] 10.222.0.82 () {58 vars in 1153 bytes} [Tue Aug 2 10:25:49 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 02 10:25:49.760230 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0354870-88d6-4845-bdef-e679ca6b1e74 None None] Authenticating user token {{(pid=80377) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:25:49.770983 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0354870-88d6-4845-bdef-e679ca6b1e74 None None] Validating token access rules against request {{(pid=80377) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:25:49.776499 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be09586e-eb9c-4871-845c-607dd0495247 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '891005fa5ff74041b75747d16cc66798', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7047912c5acd4adeba5d87c3bf895e39', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80377) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:25:49.777065 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be09586e-eb9c-4871-845c-607dd0495247 admin admin] REQUEST_METHOD: `POST` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:25:49.777350 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be09586e-eb9c-4871-845c-607dd0495247 admin admin] SCRIPT_NAME: `/identity` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:25:49.777622 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be09586e-eb9c-4871-845c-607dd0495247 admin admin] PATH_INFO: `/v3/projects` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:25:49.778334 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be09586e-eb9c-4871-845c-607dd0495247 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:25:49.780247 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be09586e-eb9c-4871-845c-607dd0495247 admin admin] RBAC: Authorization granted {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:25:49.811207 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: [pid: 80377|app: 0|req: 1276/2537] 10.222.0.82 () {60 vars in 1153 bytes} [Tue Aug 2 10:25:49 2022] POST /identity/v3/projects => generated 385 bytes in 49 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 02 10:25:49.812487 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0ec905a-6d23-480f-bf8d-62a8b6ce0507 admin admin] Authenticating user token {{(pid=80378) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:25:49.823844 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0ec905a-6d23-480f-bf8d-62a8b6ce0507 admin admin] Validating token access rules against request {{(pid=80378) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:25:49.852187 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG dbcounter [-] [80378] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80378) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 10:25:49.892164 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58da5030-9caf-4337-862b-0dcd830278e8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '891005fa5ff74041b75747d16cc66798', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7047912c5acd4adeba5d87c3bf895e39', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80378) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:25:49.892777 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58da5030-9caf-4337-862b-0dcd830278e8 admin admin] REQUEST_METHOD: `POST` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:25:49.893055 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58da5030-9caf-4337-862b-0dcd830278e8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:25:49.893329 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58da5030-9caf-4337-862b-0dcd830278e8 admin admin] PATH_INFO: `/v3/users` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:25:49.894115 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58da5030-9caf-4337-862b-0dcd830278e8 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:25:49.896055 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58da5030-9caf-4337-862b-0dcd830278e8 admin admin] RBAC: Authorization granted {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:25:49.918898 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 1262/2538] 10.222.0.82 () {60 vars in 1144 bytes} [Tue Aug 2 10:25:49 2022] POST /identity/v3/users => generated 293 bytes in 107 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 02 10:25:49.923215 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be09586e-eb9c-4871-845c-607dd0495247 admin admin] Authenticating user token {{(pid=80377) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:25:49.934938 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be09586e-eb9c-4871-845c-607dd0495247 admin admin] Validating token access rules against request {{(pid=80377) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:25:49.939829 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5b2a320-4da9-4369-a973-2ca8d90436ff admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '891005fa5ff74041b75747d16cc66798', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7047912c5acd4adeba5d87c3bf895e39', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80377) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:25:49.940460 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5b2a320-4da9-4369-a973-2ca8d90436ff admin admin] REQUEST_METHOD: `GET` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:25:49.940873 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5b2a320-4da9-4369-a973-2ca8d90436ff admin admin] SCRIPT_NAME: `/identity` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:25:49.941154 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5b2a320-4da9-4369-a973-2ca8d90436ff admin admin] PATH_INFO: `/v3/roles` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:25:49.941900 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5b2a320-4da9-4369-a973-2ca8d90436ff admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:25:49.943770 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5b2a320-4da9-4369-a973-2ca8d90436ff admin admin] RBAC: Authorization granted {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:25:49.957471 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: [pid: 80377|app: 0|req: 1277/2539] 10.222.0.82 () {58 vars in 1122 bytes} [Tue Aug 2 10:25:49 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 02 10:25:49.958951 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58da5030-9caf-4337-862b-0dcd830278e8 admin admin] Authenticating user token {{(pid=80378) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:25:49.970636 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58da5030-9caf-4337-862b-0dcd830278e8 admin admin] Validating token access rules against request {{(pid=80378) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:25:49.976935 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba07136d-a535-4239-a98c-93084f4f51ff admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '891005fa5ff74041b75747d16cc66798', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7047912c5acd4adeba5d87c3bf895e39', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80378) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:25:49.977558 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba07136d-a535-4239-a98c-93084f4f51ff admin admin] REQUEST_METHOD: `GET` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:25:49.977878 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba07136d-a535-4239-a98c-93084f4f51ff admin admin] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:25:49.978166 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba07136d-a535-4239-a98c-93084f4f51ff admin admin] PATH_INFO: `/v3/roles` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:25:49.978926 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba07136d-a535-4239-a98c-93084f4f51ff admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:25:49.980542 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba07136d-a535-4239-a98c-93084f4f51ff admin admin] RBAC: Authorization granted {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:25:49.992605 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 1263/2540] 10.222.0.82 () {58 vars in 1122 bytes} [Tue Aug 2 10:25:49 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 02 10:25:49.997298 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5b2a320-4da9-4369-a973-2ca8d90436ff admin admin] Authenticating user token {{(pid=80377) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:25:50.008556 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5b2a320-4da9-4369-a973-2ca8d90436ff admin admin] Validating token access rules against request {{(pid=80377) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:25:50.013999 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27f4cb30-6cb0-422f-af71-334de49f9def admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '891005fa5ff74041b75747d16cc66798', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7047912c5acd4adeba5d87c3bf895e39', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80377) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:25:50.014815 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27f4cb30-6cb0-422f-af71-334de49f9def admin admin] REQUEST_METHOD: `PUT` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:25:50.015178 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27f4cb30-6cb0-422f-af71-334de49f9def admin admin] SCRIPT_NAME: `/identity` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:25:50.015320 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27f4cb30-6cb0-422f-af71-334de49f9def admin admin] PATH_INFO: `/v3/projects/f9fce1883f5f4bc085916d84be562333/users/00cbabc9063245688c7ab9b7d726c6f8/roles/e1c04b19fc4543b8b54f58bbcffbc598` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:25:50.032909 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27f4cb30-6cb0-422f-af71-334de49f9def admin admin] RBAC: Authorizing `identity:create_grant(project_id=f9fce1883f5f4bc085916d84be562333, user_id=00cbabc9063245688c7ab9b7d726c6f8, role_id=e1c04b19fc4543b8b54f58bbcffbc598)` {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:25:50.035673 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27f4cb30-6cb0-422f-af71-334de49f9def admin admin] RBAC: Authorization granted {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:25:50.050875 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: [pid: 80377|app: 0|req: 1278/2541] 10.222.0.82 () {60 vars in 1483 bytes} [Tue Aug 2 10:25:49 2022] PUT /identity/v3/projects/f9fce1883f5f4bc085916d84be562333/users/00cbabc9063245688c7ab9b7d726c6f8/roles/e1c04b19fc4543b8b54f58bbcffbc598 => generated 0 bytes in 54 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 02 10:25:50.062271 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc67ebe2-b730-46f1-b4dc-652f2f4dfe49 None None] REQUEST_METHOD: `POST` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:25:50.062572 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc67ebe2-b730-46f1-b4dc-652f2f4dfe49 None None] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:25:50.062855 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc67ebe2-b730-46f1-b4dc-652f2f4dfe49 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:25:50.093658 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.auth.core [None req-fc67ebe2-b730-46f1-b4dc-652f2f4dfe49 None None] MFA Rules not processed for user `00cbabc9063245688c7ab9b7d726c6f8`. Rule list: `[]` (Enabled: `True`). {{(pid=80378) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 02 10:25:50.094816 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.fernet_utils [None req-fc67ebe2-b730-46f1-b4dc-652f2f4dfe49 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80378) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 10:25:50.160142 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 1264/2542] 10.222.0.82 () {58 vars in 958 bytes} [Tue Aug 2 10:25:50 2022] POST /identity/v3/auth/tokens => generated 3262 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 02 10:25:50.170865 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27f4cb30-6cb0-422f-af71-334de49f9def admin admin] Authenticating user token {{(pid=80377) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:25:50.172582 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.fernet_utils [None req-27f4cb30-6cb0-422f-af71-334de49f9def admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80377) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 10:25:50.273829 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27f4cb30-6cb0-422f-af71-334de49f9def admin admin] Validating token access rules against request {{(pid=80377) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:25:50.279755 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8cc14cdc-00ad-493e-8fd5-366a3416fe4e service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88af518196674fe593ad88d30111fb7d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f1b38ebb804c41c69772c998aae72c41', 'project_domain_id': 'default', 'roles': ['admin', 'ResellerAdmin', 'reader', 'service', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80377) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:25:50.280387 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cc14cdc-00ad-493e-8fd5-366a3416fe4e service nova] REQUEST_METHOD: `GET` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:25:50.280670 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cc14cdc-00ad-493e-8fd5-366a3416fe4e service nova] SCRIPT_NAME: `/identity` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:25:50.280967 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cc14cdc-00ad-493e-8fd5-366a3416fe4e service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:25:50.289062 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8cc14cdc-00ad-493e-8fd5-366a3416fe4e service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:25:50.290865 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8cc14cdc-00ad-493e-8fd5-366a3416fe4e service nova] RBAC: Authorization granted {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:25:50.299774 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: [pid: 80377|app: 0|req: 1279/2543] 10.222.0.82 () {60 vars in 1365 bytes} [Tue Aug 2 10:25:50 2022] GET /identity/v3/auth/tokens => generated 3262 bytes in 129 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 02 10:25:50.829875 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4af58ae1-dc31-4875-bc8b-633b7876d883 None None] REQUEST_METHOD: `POST` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:25:50.830363 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4af58ae1-dc31-4875-bc8b-633b7876d883 None None] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:25:50.830479 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4af58ae1-dc31-4875-bc8b-633b7876d883 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:25:50.850451 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.auth.core [None req-4af58ae1-dc31-4875-bc8b-633b7876d883 None None] MFA Rules not processed for user `8774306d95c0476b80cc8e53b62e8a4b`. Rule list: `[]` (Enabled: `True`). {{(pid=80378) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 02 10:25:50.851662 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.fernet_utils [None req-4af58ae1-dc31-4875-bc8b-633b7876d883 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80378) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 10:25:50.906179 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 1265/2544] 10.222.0.82 () {58 vars in 958 bytes} [Tue Aug 2 10:25:50 2022] POST /identity/v3/auth/tokens => generated 3262 bytes in 77 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 02 10:25:59.700193 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8cc14cdc-00ad-493e-8fd5-366a3416fe4e service nova] Authenticating user token {{(pid=80377) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:25:59.731901 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG dbcounter [-] [80377] Writing DB stats keystone:SELECT=93,keystone:DELETE=4,keystone:INSERT=3 {{(pid=80377) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 10:25:59.764919 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8cc14cdc-00ad-493e-8fd5-366a3416fe4e service nova] Validating token access rules against request {{(pid=80377) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:25:59.769422 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-498fc90f-1318-45a8-88f7-08f1fb56b9de admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '891005fa5ff74041b75747d16cc66798', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7047912c5acd4adeba5d87c3bf895e39', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80377) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:25:59.770099 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-498fc90f-1318-45a8-88f7-08f1fb56b9de admin admin] REQUEST_METHOD: `DELETE` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:25:59.770384 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-498fc90f-1318-45a8-88f7-08f1fb56b9de admin admin] SCRIPT_NAME: `/identity` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:25:59.770662 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-498fc90f-1318-45a8-88f7-08f1fb56b9de admin admin] PATH_INFO: `/v3/users/00cbabc9063245688c7ab9b7d726c6f8` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:25:59.771824 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-498fc90f-1318-45a8-88f7-08f1fb56b9de admin admin] RBAC: Authorizing `identity:delete_user(user_id=00cbabc9063245688c7ab9b7d726c6f8)` {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:25:59.773737 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-498fc90f-1318-45a8-88f7-08f1fb56b9de admin admin] RBAC: Authorization granted {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:25:59.818749 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.notifications [None req-498fc90f-1318-45a8-88f7-08f1fb56b9de admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '00cbabc9063245688c7ab9b7d726c6f8'} {{(pid=80377) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 10:25:59.827111 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.notifications [None req-498fc90f-1318-45a8-88f7-08f1fb56b9de admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '00cbabc9063245688c7ab9b7d726c6f8'} {{(pid=80377) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 10:25:59.833937 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.notifications [None req-498fc90f-1318-45a8-88f7-08f1fb56b9de admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '00cbabc9063245688c7ab9b7d726c6f8'} {{(pid=80377) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 10:25:59.834566 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.notifications [None req-498fc90f-1318-45a8-88f7-08f1fb56b9de admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '00cbabc9063245688c7ab9b7d726c6f8'} {{(pid=80377) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 10:25:59.846751 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.notifications [None req-498fc90f-1318-45a8-88f7-08f1fb56b9de admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '00cbabc9063245688c7ab9b7d726c6f8'} {{(pid=80377) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 10:25:59.852883 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: [pid: 80377|app: 0|req: 1280/2545] 10.222.0.82 () {58 vars in 1224 bytes} [Tue Aug 2 10:25:59 2022] DELETE /identity/v3/users/00cbabc9063245688c7ab9b7d726c6f8 => generated 0 bytes in 155 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 02 10:25:59.857373 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9192f06b-c5a2-4657-9392-b966c73fbdb0 None None] REQUEST_METHOD: `POST` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:25:59.857872 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9192f06b-c5a2-4657-9392-b966c73fbdb0 None None] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:25:59.857985 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9192f06b-c5a2-4657-9392-b966c73fbdb0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:25:59.876506 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.auth.core [None req-9192f06b-c5a2-4657-9392-b966c73fbdb0 None None] MFA Rules not processed for user `891005fa5ff74041b75747d16cc66798`. Rule list: `[]` (Enabled: `True`). {{(pid=80378) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 02 10:25:59.877719 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.fernet_utils [None req-9192f06b-c5a2-4657-9392-b966c73fbdb0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80378) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 10:25:59.915405 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG dbcounter [-] [80378] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80378) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 10:25:59.937452 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 1266/2546] 10.222.0.82 () {58 vars in 958 bytes} [Tue Aug 2 10:25:59 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 81 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 02 10:25:59.947944 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-498fc90f-1318-45a8-88f7-08f1fb56b9de admin admin] Authenticating user token {{(pid=80377) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:25:59.949365 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.fernet_utils [None req-498fc90f-1318-45a8-88f7-08f1fb56b9de admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80377) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 10:26:00.005277 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-498fc90f-1318-45a8-88f7-08f1fb56b9de admin admin] Validating token access rules against request {{(pid=80377) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:26:00.008909 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4c15b79-42e8-47ce-a149-228a39970530 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '33b87170f0634b67bce60750847b5cbe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f1b38ebb804c41c69772c998aae72c41', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80377) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:26:00.010077 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4c15b79-42e8-47ce-a149-228a39970530 service neutron] REQUEST_METHOD: `GET` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:26:00.010367 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4c15b79-42e8-47ce-a149-228a39970530 service neutron] SCRIPT_NAME: `/identity` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:26:00.010642 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4c15b79-42e8-47ce-a149-228a39970530 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:26:00.019608 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4c15b79-42e8-47ce-a149-228a39970530 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:26:00.021050 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4c15b79-42e8-47ce-a149-228a39970530 service neutron] RBAC: Authorization granted {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:26:00.030334 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: [pid: 80377|app: 0|req: 1281/2547] 10.222.0.82 () {60 vars in 1365 bytes} [Tue Aug 2 10:25:59 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 83 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 02 10:26:00.289608 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9192f06b-c5a2-4657-9392-b966c73fbdb0 None None] Authenticating user token {{(pid=80378) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:26:00.291728 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.fernet_utils [None req-9192f06b-c5a2-4657-9392-b966c73fbdb0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80378) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 10:26:00.303564 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9192f06b-c5a2-4657-9392-b966c73fbdb0 None None] Validating token access rules against request {{(pid=80378) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:26:00.308458 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83b4907d-4d7e-4726-8eb1-f7433dd046b5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '891005fa5ff74041b75747d16cc66798', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7047912c5acd4adeba5d87c3bf895e39', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80378) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:26:00.309187 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83b4907d-4d7e-4726-8eb1-f7433dd046b5 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:26:00.309442 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83b4907d-4d7e-4726-8eb1-f7433dd046b5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:26:00.309724 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83b4907d-4d7e-4726-8eb1-f7433dd046b5 admin admin] PATH_INFO: `/v3/projects/f9fce1883f5f4bc085916d84be562333` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:26:00.311008 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83b4907d-4d7e-4726-8eb1-f7433dd046b5 admin admin] RBAC: Authorizing `identity:delete_project(project_id=f9fce1883f5f4bc085916d84be562333)` {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:26:00.312984 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83b4907d-4d7e-4726-8eb1-f7433dd046b5 admin admin] RBAC: Authorization granted {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:26:00.362512 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.notifications [None req-83b4907d-4d7e-4726-8eb1-f7433dd046b5 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'f9fce1883f5f4bc085916d84be562333', 'request_id': 'req-83b4907d-4d7e-4726-8eb1-f7433dd046b5'} {{(pid=80378) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 10:26:00.368782 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.notifications [None req-83b4907d-4d7e-4726-8eb1-f7433dd046b5 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'f9fce1883f5f4bc085916d84be562333', 'request_id': 'req-83b4907d-4d7e-4726-8eb1-f7433dd046b5'} {{(pid=80378) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 10:26:00.374286 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.notifications [None req-83b4907d-4d7e-4726-8eb1-f7433dd046b5 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'f9fce1883f5f4bc085916d84be562333', 'request_id': 'req-83b4907d-4d7e-4726-8eb1-f7433dd046b5'} {{(pid=80378) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 10:26:00.386722 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.notifications [None req-83b4907d-4d7e-4726-8eb1-f7433dd046b5 admin admin] The token cache is being invalidate because project f9fce1883f5f4bc085916d84be562333 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80378) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 02 10:26:00.387021 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.notifications [None req-83b4907d-4d7e-4726-8eb1-f7433dd046b5 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80378) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 10:26:00.387644 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.notifications [None req-83b4907d-4d7e-4726-8eb1-f7433dd046b5 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80378) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 10:26:00.389175 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 1267/2548] 10.222.0.82 () {58 vars in 1233 bytes} [Tue Aug 2 10:26:00 2022] DELETE /identity/v3/projects/f9fce1883f5f4bc085916d84be562333 => generated 0 bytes in 101 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 02 10:26:00.394083 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f683495c-ca44-4665-bddd-7143c3287b09 None None] REQUEST_METHOD: `POST` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:26:00.394372 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f683495c-ca44-4665-bddd-7143c3287b09 None None] SCRIPT_NAME: `/identity` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:26:00.394653 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f683495c-ca44-4665-bddd-7143c3287b09 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:26:00.414416 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.auth.core [None req-f683495c-ca44-4665-bddd-7143c3287b09 None None] MFA Rules not processed for user `891005fa5ff74041b75747d16cc66798`. Rule list: `[]` (Enabled: `True`). {{(pid=80377) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 02 10:26:00.415622 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.fernet_utils [None req-f683495c-ca44-4665-bddd-7143c3287b09 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80377) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 10:26:00.464431 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG dbcounter [-] [80377] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80377) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 10:26:00.480874 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: [pid: 80377|app: 0|req: 1282/2549] 10.222.0.82 () {58 vars in 958 bytes} [Tue Aug 2 10:26:00 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 02 10:26:00.488705 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49e173fa-65ff-43b5-b213-e666db9c3075 None None] REQUEST_METHOD: `POST` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:26:00.488999 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49e173fa-65ff-43b5-b213-e666db9c3075 None None] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:26:00.489283 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49e173fa-65ff-43b5-b213-e666db9c3075 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:26:00.508887 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.auth.core [None req-49e173fa-65ff-43b5-b213-e666db9c3075 None None] MFA Rules not processed for user `891005fa5ff74041b75747d16cc66798`. Rule list: `[]` (Enabled: `True`). {{(pid=80378) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 02 10:26:00.510696 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.fernet_utils [None req-49e173fa-65ff-43b5-b213-e666db9c3075 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80378) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 10:26:00.518892 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 1268/2550] 10.222.0.82 () {58 vars in 958 bytes} [Tue Aug 2 10:26:00 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 02 10:26:00.523243 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f683495c-ca44-4665-bddd-7143c3287b09 None None] Authenticating user token {{(pid=80377) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:26:00.533606 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f683495c-ca44-4665-bddd-7143c3287b09 None None] Validating token access rules against request {{(pid=80377) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:26:00.538312 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b56f832-fb2f-4451-bfd6-791e574d6b2d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '891005fa5ff74041b75747d16cc66798', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7047912c5acd4adeba5d87c3bf895e39', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80377) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:26:00.538932 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b56f832-fb2f-4451-bfd6-791e574d6b2d admin admin] REQUEST_METHOD: `GET` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:26:00.539213 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b56f832-fb2f-4451-bfd6-791e574d6b2d admin admin] SCRIPT_NAME: `/identity` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:26:00.539488 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b56f832-fb2f-4451-bfd6-791e574d6b2d admin admin] PATH_INFO: `/v3/domains` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:26:00.540190 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b56f832-fb2f-4451-bfd6-791e574d6b2d admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80377) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 02 10:26:00.540526 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b56f832-fb2f-4451-bfd6-791e574d6b2d admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:26:00.542236 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b56f832-fb2f-4451-bfd6-791e574d6b2d admin admin] RBAC: Authorization granted {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:26:00.554407 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: [pid: 80377|app: 0|req: 1283/2551] 10.222.0.82 () {58 vars in 1153 bytes} [Tue Aug 2 10:26:00 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 02 10:26:00.558910 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49e173fa-65ff-43b5-b213-e666db9c3075 None None] Authenticating user token {{(pid=80378) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:26:00.568622 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49e173fa-65ff-43b5-b213-e666db9c3075 None None] Validating token access rules against request {{(pid=80378) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:26:00.574145 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75ebd0c4-94d2-4843-a830-288a85f3c499 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '891005fa5ff74041b75747d16cc66798', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7047912c5acd4adeba5d87c3bf895e39', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80378) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:26:00.574766 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75ebd0c4-94d2-4843-a830-288a85f3c499 admin admin] REQUEST_METHOD: `POST` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:26:00.575051 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75ebd0c4-94d2-4843-a830-288a85f3c499 admin admin] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:26:00.575327 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75ebd0c4-94d2-4843-a830-288a85f3c499 admin admin] PATH_INFO: `/v3/projects` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:26:00.576006 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75ebd0c4-94d2-4843-a830-288a85f3c499 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:26:00.577935 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75ebd0c4-94d2-4843-a830-288a85f3c499 admin admin] RBAC: Authorization granted {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:26:00.620164 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 1269/2552] 10.222.0.82 () {60 vars in 1153 bytes} [Tue Aug 2 10:26:00 2022] POST /identity/v3/projects => generated 373 bytes in 62 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 02 10:26:00.624303 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b56f832-fb2f-4451-bfd6-791e574d6b2d admin admin] Authenticating user token {{(pid=80377) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:26:00.635051 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b56f832-fb2f-4451-bfd6-791e574d6b2d admin admin] Validating token access rules against request {{(pid=80377) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:26:00.695512 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc9f5972-86b2-49f5-be86-e04bf6a6c2a1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '891005fa5ff74041b75747d16cc66798', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7047912c5acd4adeba5d87c3bf895e39', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80377) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:26:00.696141 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc9f5972-86b2-49f5-be86-e04bf6a6c2a1 admin admin] REQUEST_METHOD: `POST` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:26:00.696424 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc9f5972-86b2-49f5-be86-e04bf6a6c2a1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:26:00.696698 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc9f5972-86b2-49f5-be86-e04bf6a6c2a1 admin admin] PATH_INFO: `/v3/users` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:26:00.697412 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc9f5972-86b2-49f5-be86-e04bf6a6c2a1 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:26:00.699411 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc9f5972-86b2-49f5-be86-e04bf6a6c2a1 admin admin] RBAC: Authorization granted {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:26:00.734246 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: [pid: 80377|app: 0|req: 1284/2553] 10.222.0.82 () {60 vars in 1144 bytes} [Tue Aug 2 10:26:00 2022] POST /identity/v3/users => generated 287 bytes in 107 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 02 10:26:00.735954 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75ebd0c4-94d2-4843-a830-288a85f3c499 admin admin] Authenticating user token {{(pid=80378) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:26:00.746271 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75ebd0c4-94d2-4843-a830-288a85f3c499 admin admin] Validating token access rules against request {{(pid=80378) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:26:00.750915 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb59f963-03f8-4435-8f4e-5862b2254d24 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '891005fa5ff74041b75747d16cc66798', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7047912c5acd4adeba5d87c3bf895e39', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80378) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:26:00.751528 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb59f963-03f8-4435-8f4e-5862b2254d24 admin admin] REQUEST_METHOD: `GET` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:26:00.751807 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb59f963-03f8-4435-8f4e-5862b2254d24 admin admin] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:26:00.752075 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb59f963-03f8-4435-8f4e-5862b2254d24 admin admin] PATH_INFO: `/v3/roles` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:26:00.752765 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb59f963-03f8-4435-8f4e-5862b2254d24 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:26:00.754395 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb59f963-03f8-4435-8f4e-5862b2254d24 admin admin] RBAC: Authorization granted {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:26:00.768641 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 1270/2554] 10.222.0.82 () {58 vars in 1122 bytes} [Tue Aug 2 10:26:00 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 02 10:26:00.770389 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc9f5972-86b2-49f5-be86-e04bf6a6c2a1 admin admin] Authenticating user token {{(pid=80377) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:26:00.781656 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc9f5972-86b2-49f5-be86-e04bf6a6c2a1 admin admin] Validating token access rules against request {{(pid=80377) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:26:00.786389 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b7b85fa-aed9-4f79-898b-ef2380c16864 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '891005fa5ff74041b75747d16cc66798', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7047912c5acd4adeba5d87c3bf895e39', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80377) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:26:00.787038 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b7b85fa-aed9-4f79-898b-ef2380c16864 admin admin] REQUEST_METHOD: `GET` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:26:00.787388 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b7b85fa-aed9-4f79-898b-ef2380c16864 admin admin] SCRIPT_NAME: `/identity` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:26:00.787597 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b7b85fa-aed9-4f79-898b-ef2380c16864 admin admin] PATH_INFO: `/v3/roles` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:26:00.788326 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b7b85fa-aed9-4f79-898b-ef2380c16864 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:26:00.789956 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b7b85fa-aed9-4f79-898b-ef2380c16864 admin admin] RBAC: Authorization granted {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:26:00.802167 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: [pid: 80377|app: 0|req: 1285/2555] 10.222.0.82 () {58 vars in 1122 bytes} [Tue Aug 2 10:26:00 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 02 10:26:00.807209 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb59f963-03f8-4435-8f4e-5862b2254d24 admin admin] Authenticating user token {{(pid=80378) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:26:00.817513 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb59f963-03f8-4435-8f4e-5862b2254d24 admin admin] Validating token access rules against request {{(pid=80378) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:26:00.823107 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e24c5803-0690-4f5b-91ef-4d52e3174c53 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '891005fa5ff74041b75747d16cc66798', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7047912c5acd4adeba5d87c3bf895e39', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80378) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:26:00.824390 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e24c5803-0690-4f5b-91ef-4d52e3174c53 admin admin] REQUEST_METHOD: `PUT` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:26:00.824685 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e24c5803-0690-4f5b-91ef-4d52e3174c53 admin admin] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:26:00.824966 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e24c5803-0690-4f5b-91ef-4d52e3174c53 admin admin] PATH_INFO: `/v3/projects/ee4b032eb52149a399dc9d30c3d15b53/users/9f54983a174e490ebed30d0347bffa4e/roles/e1c04b19fc4543b8b54f58bbcffbc598` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:26:00.840406 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e24c5803-0690-4f5b-91ef-4d52e3174c53 admin admin] RBAC: Authorizing `identity:create_grant(project_id=ee4b032eb52149a399dc9d30c3d15b53, user_id=9f54983a174e490ebed30d0347bffa4e, role_id=e1c04b19fc4543b8b54f58bbcffbc598)` {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:26:00.843112 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e24c5803-0690-4f5b-91ef-4d52e3174c53 admin admin] RBAC: Authorization granted {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:26:00.859302 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 1271/2556] 10.222.0.82 () {60 vars in 1483 bytes} [Tue Aug 2 10:26:00 2022] PUT /identity/v3/projects/ee4b032eb52149a399dc9d30c3d15b53/users/9f54983a174e490ebed30d0347bffa4e/roles/e1c04b19fc4543b8b54f58bbcffbc598 => generated 0 bytes in 53 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 02 10:26:00.870372 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e892629-55b3-41ab-ac5f-3b03dfa0f7f0 None None] REQUEST_METHOD: `POST` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:26:00.870847 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e892629-55b3-41ab-ac5f-3b03dfa0f7f0 None None] SCRIPT_NAME: `/identity` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:26:00.871124 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e892629-55b3-41ab-ac5f-3b03dfa0f7f0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:26:00.899649 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.auth.core [None req-9e892629-55b3-41ab-ac5f-3b03dfa0f7f0 None None] MFA Rules not processed for user `9f54983a174e490ebed30d0347bffa4e`. Rule list: `[]` (Enabled: `True`). {{(pid=80377) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 02 10:26:00.900914 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.fernet_utils [None req-9e892629-55b3-41ab-ac5f-3b03dfa0f7f0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80377) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 10:26:00.966267 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: [pid: 80377|app: 0|req: 1286/2557] 10.222.0.82 () {58 vars in 958 bytes} [Tue Aug 2 10:26:00 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 02 10:26:00.977138 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e24c5803-0690-4f5b-91ef-4d52e3174c53 admin admin] Authenticating user token {{(pid=80378) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:26:00.978905 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.fernet_utils [None req-e24c5803-0690-4f5b-91ef-4d52e3174c53 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80378) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 10:26:01.036745 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG dbcounter [-] [80378] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=80378) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 10:26:01.060509 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e24c5803-0690-4f5b-91ef-4d52e3174c53 admin admin] Validating token access rules against request {{(pid=80378) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:26:01.066363 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44902c35-2806-4346-875d-2c2ff6ca8067 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88af518196674fe593ad88d30111fb7d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f1b38ebb804c41c69772c998aae72c41', 'project_domain_id': 'default', 'roles': ['admin', 'ResellerAdmin', 'reader', 'service', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80378) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:26:01.066983 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44902c35-2806-4346-875d-2c2ff6ca8067 service nova] REQUEST_METHOD: `GET` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:26:01.067264 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44902c35-2806-4346-875d-2c2ff6ca8067 service nova] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:26:01.067539 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44902c35-2806-4346-875d-2c2ff6ca8067 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:26:01.076513 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44902c35-2806-4346-875d-2c2ff6ca8067 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:26:01.078140 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44902c35-2806-4346-875d-2c2ff6ca8067 service nova] RBAC: Authorization granted {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:26:01.086972 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 1272/2558] 10.222.0.82 () {60 vars in 1365 bytes} [Tue Aug 2 10:26:00 2022] GET /identity/v3/auth/tokens => generated 3250 bytes in 110 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 02 10:26:02.474857 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e892629-55b3-41ab-ac5f-3b03dfa0f7f0 None None] Authenticating user token {{(pid=80377) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:26:02.476538 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.fernet_utils [None req-9e892629-55b3-41ab-ac5f-3b03dfa0f7f0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80377) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 10:26:02.531067 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG dbcounter [-] [80377] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80377) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 10:26:02.559013 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e892629-55b3-41ab-ac5f-3b03dfa0f7f0 None None] Validating token access rules against request {{(pid=80377) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:26:02.562929 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8c30db9-afd3-4909-b64a-be8b921a84b3 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1023de281ae74e9d8a6d6c312c1c97a7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f1b38ebb804c41c69772c998aae72c41', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80377) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:26:02.563548 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8c30db9-afd3-4909-b64a-be8b921a84b3 service cinder] REQUEST_METHOD: `GET` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:26:02.563838 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8c30db9-afd3-4909-b64a-be8b921a84b3 service cinder] SCRIPT_NAME: `/identity` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:26:02.564117 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8c30db9-afd3-4909-b64a-be8b921a84b3 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:26:02.565924 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.fernet_utils [None req-b8c30db9-afd3-4909-b64a-be8b921a84b3 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80377) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 10:26:02.579614 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8c30db9-afd3-4909-b64a-be8b921a84b3 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:26:02.581051 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8c30db9-afd3-4909-b64a-be8b921a84b3 service cinder] RBAC: Authorization granted {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:26:02.592527 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: [pid: 80377|app: 0|req: 1287/2559] 10.222.0.82 () {60 vars in 1365 bytes} [Tue Aug 2 10:26:02 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 119 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 02 10:26:02.662188 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44902c35-2806-4346-875d-2c2ff6ca8067 service nova] Authenticating user token {{(pid=80378) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:26:02.663650 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.fernet_utils [None req-44902c35-2806-4346-875d-2c2ff6ca8067 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80378) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 10:26:02.680015 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44902c35-2806-4346-875d-2c2ff6ca8067 service nova] Validating token access rules against request {{(pid=80378) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:26:02.684376 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a726ac3c-a0d0-44ab-95cc-62d2b0fc5569 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1023de281ae74e9d8a6d6c312c1c97a7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f1b38ebb804c41c69772c998aae72c41', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80378) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:26:02.684996 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a726ac3c-a0d0-44ab-95cc-62d2b0fc5569 service cinder] REQUEST_METHOD: `GET` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:26:02.685280 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a726ac3c-a0d0-44ab-95cc-62d2b0fc5569 service cinder] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:26:02.685555 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a726ac3c-a0d0-44ab-95cc-62d2b0fc5569 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:26:02.687384 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.fernet_utils [None req-a726ac3c-a0d0-44ab-95cc-62d2b0fc5569 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80378) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 10:26:02.702997 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a726ac3c-a0d0-44ab-95cc-62d2b0fc5569 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:26:02.704439 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a726ac3c-a0d0-44ab-95cc-62d2b0fc5569 service cinder] RBAC: Authorization granted {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:26:02.717487 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 1273/2560] 10.222.0.82 () {60 vars in 1365 bytes} [Tue Aug 2 10:26:02 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 56 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 02 10:26:03.954845 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8c30db9-afd3-4909-b64a-be8b921a84b3 service cinder] Authenticating user token {{(pid=80377) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:26:04.014593 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8c30db9-afd3-4909-b64a-be8b921a84b3 service cinder] Validating token access rules against request {{(pid=80377) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:26:04.019801 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15fb156a-65f7-4a65-8a4d-06dd52f8252c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '891005fa5ff74041b75747d16cc66798', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7047912c5acd4adeba5d87c3bf895e39', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80377) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:26:04.020441 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15fb156a-65f7-4a65-8a4d-06dd52f8252c admin admin] REQUEST_METHOD: `DELETE` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:26:04.020729 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15fb156a-65f7-4a65-8a4d-06dd52f8252c admin admin] SCRIPT_NAME: `/identity` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:26:04.021000 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15fb156a-65f7-4a65-8a4d-06dd52f8252c admin admin] PATH_INFO: `/v3/users/9f54983a174e490ebed30d0347bffa4e` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:26:04.022210 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15fb156a-65f7-4a65-8a4d-06dd52f8252c admin admin] RBAC: Authorizing `identity:delete_user(user_id=9f54983a174e490ebed30d0347bffa4e)` {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:26:04.024128 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15fb156a-65f7-4a65-8a4d-06dd52f8252c admin admin] RBAC: Authorization granted {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:26:04.076203 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.notifications [None req-15fb156a-65f7-4a65-8a4d-06dd52f8252c admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '9f54983a174e490ebed30d0347bffa4e'} {{(pid=80377) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 10:26:04.084678 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.notifications [None req-15fb156a-65f7-4a65-8a4d-06dd52f8252c admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '9f54983a174e490ebed30d0347bffa4e'} {{(pid=80377) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 10:26:04.091610 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.notifications [None req-15fb156a-65f7-4a65-8a4d-06dd52f8252c admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '9f54983a174e490ebed30d0347bffa4e'} {{(pid=80377) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 10:26:04.092256 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.notifications [None req-15fb156a-65f7-4a65-8a4d-06dd52f8252c admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '9f54983a174e490ebed30d0347bffa4e'} {{(pid=80377) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 10:26:04.105244 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.notifications [None req-15fb156a-65f7-4a65-8a4d-06dd52f8252c admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '9f54983a174e490ebed30d0347bffa4e'} {{(pid=80377) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 10:26:04.111383 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: [pid: 80377|app: 0|req: 1288/2561] 10.222.0.82 () {58 vars in 1224 bytes} [Tue Aug 2 10:26:03 2022] DELETE /identity/v3/users/9f54983a174e490ebed30d0347bffa4e => generated 0 bytes in 159 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 02 10:26:04.115770 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a965740c-e465-467f-8d52-ffb623b5d01a None None] REQUEST_METHOD: `POST` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:26:04.116284 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a965740c-e465-467f-8d52-ffb623b5d01a None None] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:26:04.116456 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a965740c-e465-467f-8d52-ffb623b5d01a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:26:04.136454 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.auth.core [None req-a965740c-e465-467f-8d52-ffb623b5d01a None None] MFA Rules not processed for user `891005fa5ff74041b75747d16cc66798`. Rule list: `[]` (Enabled: `True`). {{(pid=80378) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 02 10:26:04.137767 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.fernet_utils [None req-a965740c-e465-467f-8d52-ffb623b5d01a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80378) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 10:26:04.202011 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 1274/2562] 10.222.0.82 () {58 vars in 958 bytes} [Tue Aug 2 10:26:04 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 02 10:26:04.211456 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15fb156a-65f7-4a65-8a4d-06dd52f8252c admin admin] Authenticating user token {{(pid=80377) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:26:04.212869 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.fernet_utils [None req-15fb156a-65f7-4a65-8a4d-06dd52f8252c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80377) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 10:26:04.269921 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15fb156a-65f7-4a65-8a4d-06dd52f8252c admin admin] Validating token access rules against request {{(pid=80377) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:26:04.273541 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f21654fc-b937-4dd7-b875-497005611937 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '33b87170f0634b67bce60750847b5cbe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f1b38ebb804c41c69772c998aae72c41', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80377) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:26:04.274192 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f21654fc-b937-4dd7-b875-497005611937 service neutron] REQUEST_METHOD: `GET` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:26:04.274471 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f21654fc-b937-4dd7-b875-497005611937 service neutron] SCRIPT_NAME: `/identity` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:26:04.274746 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f21654fc-b937-4dd7-b875-497005611937 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:26:04.280592 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG dbcounter [-] [80377] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80377) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 10:26:04.285262 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f21654fc-b937-4dd7-b875-497005611937 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:26:04.287177 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f21654fc-b937-4dd7-b875-497005611937 service neutron] RBAC: Authorization granted {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:26:04.298017 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: [pid: 80377|app: 0|req: 1289/2563] 10.222.0.82 () {60 vars in 1365 bytes} [Tue Aug 2 10:26:04 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 87 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 02 10:26:04.568235 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a965740c-e465-467f-8d52-ffb623b5d01a None None] Authenticating user token {{(pid=80378) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:26:04.569998 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.fernet_utils [None req-a965740c-e465-467f-8d52-ffb623b5d01a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80378) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 10:26:04.584117 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a965740c-e465-467f-8d52-ffb623b5d01a None None] Validating token access rules against request {{(pid=80378) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:26:04.589809 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d983b45-9d16-49a1-9581-85217d5308cb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '891005fa5ff74041b75747d16cc66798', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7047912c5acd4adeba5d87c3bf895e39', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80378) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:26:04.590473 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d983b45-9d16-49a1-9581-85217d5308cb admin admin] REQUEST_METHOD: `DELETE` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:26:04.590762 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d983b45-9d16-49a1-9581-85217d5308cb admin admin] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:26:04.591412 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d983b45-9d16-49a1-9581-85217d5308cb admin admin] PATH_INFO: `/v3/projects/ee4b032eb52149a399dc9d30c3d15b53` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:26:04.592245 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d983b45-9d16-49a1-9581-85217d5308cb admin admin] RBAC: Authorizing `identity:delete_project(project_id=ee4b032eb52149a399dc9d30c3d15b53)` {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:26:04.594218 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d983b45-9d16-49a1-9581-85217d5308cb admin admin] RBAC: Authorization granted {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:26:04.645890 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.notifications [None req-3d983b45-9d16-49a1-9581-85217d5308cb admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'ee4b032eb52149a399dc9d30c3d15b53', 'request_id': 'req-3d983b45-9d16-49a1-9581-85217d5308cb'} {{(pid=80378) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 10:26:04.652817 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.notifications [None req-3d983b45-9d16-49a1-9581-85217d5308cb admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'ee4b032eb52149a399dc9d30c3d15b53', 'request_id': 'req-3d983b45-9d16-49a1-9581-85217d5308cb'} {{(pid=80378) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 10:26:04.658388 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.notifications [None req-3d983b45-9d16-49a1-9581-85217d5308cb admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'ee4b032eb52149a399dc9d30c3d15b53', 'request_id': 'req-3d983b45-9d16-49a1-9581-85217d5308cb'} {{(pid=80378) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 10:26:04.672138 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.notifications [None req-3d983b45-9d16-49a1-9581-85217d5308cb admin admin] The token cache is being invalidate because project ee4b032eb52149a399dc9d30c3d15b53 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80378) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 02 10:26:04.672439 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.notifications [None req-3d983b45-9d16-49a1-9581-85217d5308cb admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80378) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 10:26:04.674042 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.notifications [None req-3d983b45-9d16-49a1-9581-85217d5308cb admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80378) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 10:26:04.675782 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 1275/2564] 10.222.0.82 () {58 vars in 1233 bytes} [Tue Aug 2 10:26:04 2022] DELETE /identity/v3/projects/ee4b032eb52149a399dc9d30c3d15b53 => generated 0 bytes in 108 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 02 10:26:04.681562 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08d59c4d-c872-4997-8181-a3bfc1983336 None None] REQUEST_METHOD: `POST` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:26:04.682778 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08d59c4d-c872-4997-8181-a3bfc1983336 None None] SCRIPT_NAME: `/identity` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:26:04.683038 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08d59c4d-c872-4997-8181-a3bfc1983336 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:26:04.704300 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.auth.core [None req-08d59c4d-c872-4997-8181-a3bfc1983336 None None] MFA Rules not processed for user `891005fa5ff74041b75747d16cc66798`. Rule list: `[]` (Enabled: `True`). {{(pid=80377) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 02 10:26:04.705684 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.fernet_utils [None req-08d59c4d-c872-4997-8181-a3bfc1983336 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80377) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 10:26:04.780820 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: [pid: 80377|app: 0|req: 1290/2565] 10.222.0.82 () {58 vars in 958 bytes} [Tue Aug 2 10:26:04 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 02 10:26:04.789286 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-311cd4a1-9b6d-447c-a765-46d033217226 None None] REQUEST_METHOD: `POST` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:26:04.789604 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-311cd4a1-9b6d-447c-a765-46d033217226 None None] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:26:04.790000 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-311cd4a1-9b6d-447c-a765-46d033217226 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:26:04.811634 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.auth.core [None req-311cd4a1-9b6d-447c-a765-46d033217226 None None] MFA Rules not processed for user `891005fa5ff74041b75747d16cc66798`. Rule list: `[]` (Enabled: `True`). {{(pid=80378) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 02 10:26:04.812859 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.fernet_utils [None req-311cd4a1-9b6d-447c-a765-46d033217226 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80378) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 10:26:04.822546 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 1276/2566] 10.222.0.82 () {58 vars in 958 bytes} [Tue Aug 2 10:26:04 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 35 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 02 10:26:04.826848 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08d59c4d-c872-4997-8181-a3bfc1983336 None None] Authenticating user token {{(pid=80377) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:26:04.837063 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08d59c4d-c872-4997-8181-a3bfc1983336 None None] Validating token access rules against request {{(pid=80377) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:26:04.841969 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17bfc1cc-767f-4362-a26d-f8c81f53e25a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '891005fa5ff74041b75747d16cc66798', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7047912c5acd4adeba5d87c3bf895e39', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80377) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:26:04.842584 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17bfc1cc-767f-4362-a26d-f8c81f53e25a admin admin] REQUEST_METHOD: `GET` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:26:04.842864 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17bfc1cc-767f-4362-a26d-f8c81f53e25a admin admin] SCRIPT_NAME: `/identity` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:26:04.843138 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17bfc1cc-767f-4362-a26d-f8c81f53e25a admin admin] PATH_INFO: `/v3/domains` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:26:04.843827 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17bfc1cc-767f-4362-a26d-f8c81f53e25a admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80377) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 02 10:26:04.844137 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17bfc1cc-767f-4362-a26d-f8c81f53e25a admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:26:04.845750 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17bfc1cc-767f-4362-a26d-f8c81f53e25a admin admin] RBAC: Authorization granted {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:26:04.861741 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: [pid: 80377|app: 0|req: 1291/2567] 10.222.0.82 () {58 vars in 1153 bytes} [Tue Aug 2 10:26:04 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 02 10:26:04.866189 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-311cd4a1-9b6d-447c-a765-46d033217226 None None] Authenticating user token {{(pid=80378) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:26:04.876938 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-311cd4a1-9b6d-447c-a765-46d033217226 None None] Validating token access rules against request {{(pid=80378) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:26:04.881736 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73de9d8e-030e-4727-9e24-84f2b4d5eec0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '891005fa5ff74041b75747d16cc66798', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7047912c5acd4adeba5d87c3bf895e39', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80378) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:26:04.882388 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73de9d8e-030e-4727-9e24-84f2b4d5eec0 admin admin] REQUEST_METHOD: `POST` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:26:04.882799 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73de9d8e-030e-4727-9e24-84f2b4d5eec0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:26:04.883000 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73de9d8e-030e-4727-9e24-84f2b4d5eec0 admin admin] PATH_INFO: `/v3/projects` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:26:04.883688 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73de9d8e-030e-4727-9e24-84f2b4d5eec0 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:26:04.886188 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73de9d8e-030e-4727-9e24-84f2b4d5eec0 admin admin] RBAC: Authorization granted {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:26:04.913368 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 1277/2568] 10.222.0.82 () {60 vars in 1153 bytes} [Tue Aug 2 10:26:04 2022] POST /identity/v3/projects => generated 379 bytes in 48 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 02 10:26:04.918124 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17bfc1cc-767f-4362-a26d-f8c81f53e25a admin admin] Authenticating user token {{(pid=80377) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:26:04.928433 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17bfc1cc-767f-4362-a26d-f8c81f53e25a admin admin] Validating token access rules against request {{(pid=80377) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:26:04.988116 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8da4333b-9421-4518-abb3-0e0baa499a7f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '891005fa5ff74041b75747d16cc66798', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7047912c5acd4adeba5d87c3bf895e39', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80377) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:26:04.989182 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8da4333b-9421-4518-abb3-0e0baa499a7f admin admin] REQUEST_METHOD: `POST` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:26:04.989476 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8da4333b-9421-4518-abb3-0e0baa499a7f admin admin] SCRIPT_NAME: `/identity` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:26:04.989752 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8da4333b-9421-4518-abb3-0e0baa499a7f admin admin] PATH_INFO: `/v3/users` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:26:04.990468 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8da4333b-9421-4518-abb3-0e0baa499a7f admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:26:04.992424 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8da4333b-9421-4518-abb3-0e0baa499a7f admin admin] RBAC: Authorization granted {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:26:05.015938 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: [pid: 80377|app: 0|req: 1292/2569] 10.222.0.82 () {60 vars in 1144 bytes} [Tue Aug 2 10:26:04 2022] POST /identity/v3/users => generated 290 bytes in 95 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 02 10:26:05.017216 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73de9d8e-030e-4727-9e24-84f2b4d5eec0 admin admin] Authenticating user token {{(pid=80378) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:26:05.027616 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73de9d8e-030e-4727-9e24-84f2b4d5eec0 admin admin] Validating token access rules against request {{(pid=80378) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:26:05.032660 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12a7133d-aecf-47bc-a955-d2bf5d19c081 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '891005fa5ff74041b75747d16cc66798', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7047912c5acd4adeba5d87c3bf895e39', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80378) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:26:05.033418 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12a7133d-aecf-47bc-a955-d2bf5d19c081 admin admin] REQUEST_METHOD: `GET` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:26:05.033703 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12a7133d-aecf-47bc-a955-d2bf5d19c081 admin admin] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:26:05.034005 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12a7133d-aecf-47bc-a955-d2bf5d19c081 admin admin] PATH_INFO: `/v3/roles` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:26:05.034698 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12a7133d-aecf-47bc-a955-d2bf5d19c081 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:26:05.036282 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12a7133d-aecf-47bc-a955-d2bf5d19c081 admin admin] RBAC: Authorization granted {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:26:05.047329 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 1278/2570] 10.222.0.82 () {58 vars in 1122 bytes} [Tue Aug 2 10:26:05 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 02 10:26:05.051175 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8da4333b-9421-4518-abb3-0e0baa499a7f admin admin] Authenticating user token {{(pid=80377) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:26:05.062061 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8da4333b-9421-4518-abb3-0e0baa499a7f admin admin] Validating token access rules against request {{(pid=80377) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:26:05.066674 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72716c32-5489-4616-8f09-efa94a04fa5f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '891005fa5ff74041b75747d16cc66798', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7047912c5acd4adeba5d87c3bf895e39', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80377) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:26:05.067304 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72716c32-5489-4616-8f09-efa94a04fa5f admin admin] REQUEST_METHOD: `GET` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:26:05.067585 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72716c32-5489-4616-8f09-efa94a04fa5f admin admin] SCRIPT_NAME: `/identity` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:26:05.068117 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72716c32-5489-4616-8f09-efa94a04fa5f admin admin] PATH_INFO: `/v3/roles` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:26:05.068822 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72716c32-5489-4616-8f09-efa94a04fa5f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:26:05.070452 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72716c32-5489-4616-8f09-efa94a04fa5f admin admin] RBAC: Authorization granted {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:26:05.080832 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: [pid: 80377|app: 0|req: 1293/2571] 10.222.0.82 () {58 vars in 1122 bytes} [Tue Aug 2 10:26:05 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 02 10:26:05.084843 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12a7133d-aecf-47bc-a955-d2bf5d19c081 admin admin] Authenticating user token {{(pid=80378) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:26:05.094807 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12a7133d-aecf-47bc-a955-d2bf5d19c081 admin admin] Validating token access rules against request {{(pid=80378) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:26:05.099439 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e9abb32-a7f2-4161-88fa-bcacf2a61c97 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '891005fa5ff74041b75747d16cc66798', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7047912c5acd4adeba5d87c3bf895e39', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80378) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:26:05.100086 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e9abb32-a7f2-4161-88fa-bcacf2a61c97 admin admin] REQUEST_METHOD: `PUT` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:26:05.100372 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e9abb32-a7f2-4161-88fa-bcacf2a61c97 admin admin] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:26:05.100648 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e9abb32-a7f2-4161-88fa-bcacf2a61c97 admin admin] PATH_INFO: `/v3/projects/fab5382451fe4e2db7711dcfc696c8d9/users/fba7f9d4a7ed4dd5a02dd4cd23f57e9f/roles/e1c04b19fc4543b8b54f58bbcffbc598` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:26:05.104214 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG dbcounter [-] [80378] Writing DB stats keystone:SELECT=91,keystone:DELETE=7,keystone:INSERT=2 {{(pid=80378) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 10:26:05.117762 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e9abb32-a7f2-4161-88fa-bcacf2a61c97 admin admin] RBAC: Authorizing `identity:create_grant(project_id=fab5382451fe4e2db7711dcfc696c8d9, user_id=fba7f9d4a7ed4dd5a02dd4cd23f57e9f, role_id=e1c04b19fc4543b8b54f58bbcffbc598)` {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:26:05.121232 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e9abb32-a7f2-4161-88fa-bcacf2a61c97 admin admin] RBAC: Authorization granted {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:26:05.133768 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 1279/2572] 10.222.0.82 () {60 vars in 1483 bytes} [Tue Aug 2 10:26:05 2022] PUT /identity/v3/projects/fab5382451fe4e2db7711dcfc696c8d9/users/fba7f9d4a7ed4dd5a02dd4cd23f57e9f/roles/e1c04b19fc4543b8b54f58bbcffbc598 => generated 0 bytes in 49 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 02 10:26:05.144613 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48f9a009-c39c-4cde-9ae1-74f56e5a152a None None] REQUEST_METHOD: `POST` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:26:05.144940 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48f9a009-c39c-4cde-9ae1-74f56e5a152a None None] SCRIPT_NAME: `/identity` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:26:05.145221 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48f9a009-c39c-4cde-9ae1-74f56e5a152a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:26:05.174475 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.auth.core [None req-48f9a009-c39c-4cde-9ae1-74f56e5a152a None None] MFA Rules not processed for user `fba7f9d4a7ed4dd5a02dd4cd23f57e9f`. Rule list: `[]` (Enabled: `True`). {{(pid=80377) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 02 10:26:05.175719 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.fernet_utils [None req-48f9a009-c39c-4cde-9ae1-74f56e5a152a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80377) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 10:26:05.223636 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG dbcounter [-] [80377] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80377) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 10:26:05.242059 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: [pid: 80377|app: 0|req: 1294/2573] 10.222.0.82 () {58 vars in 958 bytes} [Tue Aug 2 10:26:05 2022] POST /identity/v3/auth/tokens => generated 3256 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 02 10:26:05.247150 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e9abb32-a7f2-4161-88fa-bcacf2a61c97 admin admin] Authenticating user token {{(pid=80378) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:26:05.312233 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e9abb32-a7f2-4161-88fa-bcacf2a61c97 admin admin] Validating token access rules against request {{(pid=80378) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:26:05.317320 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c1a2656-47cc-4f39-a249-9c85fe37120f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '891005fa5ff74041b75747d16cc66798', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7047912c5acd4adeba5d87c3bf895e39', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80378) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:26:05.318347 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c1a2656-47cc-4f39-a249-9c85fe37120f admin admin] REQUEST_METHOD: `POST` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:26:05.318695 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c1a2656-47cc-4f39-a249-9c85fe37120f admin admin] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:26:05.319037 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c1a2656-47cc-4f39-a249-9c85fe37120f admin admin] PATH_INFO: `/v3/projects` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:26:05.319799 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c1a2656-47cc-4f39-a249-9c85fe37120f admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:26:05.322010 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c1a2656-47cc-4f39-a249-9c85fe37120f admin admin] RBAC: Authorization granted {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:26:05.353555 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 1280/2574] 10.222.0.82 () {60 vars in 1153 bytes} [Tue Aug 2 10:26:05 2022] POST /identity/v3/projects => generated 379 bytes in 104 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 02 10:26:05.354699 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48f9a009-c39c-4cde-9ae1-74f56e5a152a None None] Authenticating user token {{(pid=80377) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:26:05.365267 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48f9a009-c39c-4cde-9ae1-74f56e5a152a None None] Validating token access rules against request {{(pid=80377) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:26:05.420973 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-776ed612-d029-4472-b769-edfc8b4e3a5a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '891005fa5ff74041b75747d16cc66798', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7047912c5acd4adeba5d87c3bf895e39', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80377) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:26:05.421514 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-776ed612-d029-4472-b769-edfc8b4e3a5a admin admin] REQUEST_METHOD: `POST` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:26:05.421825 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-776ed612-d029-4472-b769-edfc8b4e3a5a admin admin] SCRIPT_NAME: `/identity` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:26:05.422110 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-776ed612-d029-4472-b769-edfc8b4e3a5a admin admin] PATH_INFO: `/v3/users` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:26:05.422783 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-776ed612-d029-4472-b769-edfc8b4e3a5a admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:26:05.424702 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-776ed612-d029-4472-b769-edfc8b4e3a5a admin admin] RBAC: Authorization granted {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:26:05.446383 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: [pid: 80377|app: 0|req: 1295/2575] 10.222.0.82 () {60 vars in 1144 bytes} [Tue Aug 2 10:26:05 2022] POST /identity/v3/users => generated 296 bytes in 92 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 02 10:26:05.450975 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c1a2656-47cc-4f39-a249-9c85fe37120f admin admin] Authenticating user token {{(pid=80378) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:26:05.461173 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c1a2656-47cc-4f39-a249-9c85fe37120f admin admin] Validating token access rules against request {{(pid=80378) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:26:05.465711 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5a7fbfd-7964-4385-89a9-2f93d209cfb0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '891005fa5ff74041b75747d16cc66798', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7047912c5acd4adeba5d87c3bf895e39', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80378) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:26:05.466347 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5a7fbfd-7964-4385-89a9-2f93d209cfb0 admin admin] REQUEST_METHOD: `GET` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:26:05.466629 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5a7fbfd-7964-4385-89a9-2f93d209cfb0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:26:05.466903 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5a7fbfd-7964-4385-89a9-2f93d209cfb0 admin admin] PATH_INFO: `/v3/roles` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:26:05.467593 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5a7fbfd-7964-4385-89a9-2f93d209cfb0 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:26:05.469189 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5a7fbfd-7964-4385-89a9-2f93d209cfb0 admin admin] RBAC: Authorization granted {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:26:05.479569 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 1281/2576] 10.222.0.82 () {58 vars in 1122 bytes} [Tue Aug 2 10:26:05 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 02 10:26:05.483502 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-776ed612-d029-4472-b769-edfc8b4e3a5a admin admin] Authenticating user token {{(pid=80377) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:26:05.493380 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-776ed612-d029-4472-b769-edfc8b4e3a5a admin admin] Validating token access rules against request {{(pid=80377) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:26:05.498554 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-639c5867-97cb-465f-b6b0-3739ed7e787d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '891005fa5ff74041b75747d16cc66798', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7047912c5acd4adeba5d87c3bf895e39', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80377) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:26:05.499212 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-639c5867-97cb-465f-b6b0-3739ed7e787d admin admin] REQUEST_METHOD: `PUT` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:26:05.499501 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-639c5867-97cb-465f-b6b0-3739ed7e787d admin admin] SCRIPT_NAME: `/identity` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:26:05.499786 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-639c5867-97cb-465f-b6b0-3739ed7e787d admin admin] PATH_INFO: `/v3/projects/cfede307bc0941b38a30fdbdeb09f473/users/6f521142f0604a5dad9294703b7e56cc/roles/237966442f05411191105bf336bc14f3` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:26:05.514595 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-639c5867-97cb-465f-b6b0-3739ed7e787d admin admin] RBAC: Authorizing `identity:create_grant(project_id=cfede307bc0941b38a30fdbdeb09f473, user_id=6f521142f0604a5dad9294703b7e56cc, role_id=237966442f05411191105bf336bc14f3)` {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:26:05.517243 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-639c5867-97cb-465f-b6b0-3739ed7e787d admin admin] RBAC: Authorization granted {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:26:05.530051 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: [pid: 80377|app: 0|req: 1296/2577] 10.222.0.82 () {60 vars in 1483 bytes} [Tue Aug 2 10:26:05 2022] PUT /identity/v3/projects/cfede307bc0941b38a30fdbdeb09f473/users/6f521142f0604a5dad9294703b7e56cc/roles/237966442f05411191105bf336bc14f3 => generated 0 bytes in 47 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 02 10:26:05.533939 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5a7fbfd-7964-4385-89a9-2f93d209cfb0 admin admin] Authenticating user token {{(pid=80378) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:26:05.544302 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5a7fbfd-7964-4385-89a9-2f93d209cfb0 admin admin] Validating token access rules against request {{(pid=80378) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:26:05.599380 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3e50fe6-26ee-487f-9504-6a087eaa109d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '891005fa5ff74041b75747d16cc66798', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7047912c5acd4adeba5d87c3bf895e39', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80378) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:26:05.600022 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3e50fe6-26ee-487f-9504-6a087eaa109d admin admin] REQUEST_METHOD: `GET` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:26:05.600305 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3e50fe6-26ee-487f-9504-6a087eaa109d admin admin] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:26:05.600579 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3e50fe6-26ee-487f-9504-6a087eaa109d admin admin] PATH_INFO: `/v3/roles` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:26:05.601265 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3e50fe6-26ee-487f-9504-6a087eaa109d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:26:05.603403 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3e50fe6-26ee-487f-9504-6a087eaa109d admin admin] RBAC: Authorization granted {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:26:05.612835 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 1282/2578] 10.222.0.82 () {58 vars in 1122 bytes} [Tue Aug 2 10:26:05 2022] GET /identity/v3/roles => generated 1420 bytes in 79 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 02 10:26:05.616906 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-639c5867-97cb-465f-b6b0-3739ed7e787d admin admin] Authenticating user token {{(pid=80377) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:26:05.627626 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-639c5867-97cb-465f-b6b0-3739ed7e787d admin admin] Validating token access rules against request {{(pid=80377) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:26:05.632285 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1daef0e4-0ba2-467d-89e6-9cd4b5102022 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '891005fa5ff74041b75747d16cc66798', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7047912c5acd4adeba5d87c3bf895e39', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80377) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:26:05.632983 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1daef0e4-0ba2-467d-89e6-9cd4b5102022 admin admin] REQUEST_METHOD: `PUT` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:26:05.633265 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1daef0e4-0ba2-467d-89e6-9cd4b5102022 admin admin] SCRIPT_NAME: `/identity` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:26:05.633543 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1daef0e4-0ba2-467d-89e6-9cd4b5102022 admin admin] PATH_INFO: `/v3/projects/cfede307bc0941b38a30fdbdeb09f473/users/6f521142f0604a5dad9294703b7e56cc/roles/237966442f05411191105bf336bc14f3` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:26:05.635608 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1daef0e4-0ba2-467d-89e6-9cd4b5102022 admin admin] RBAC: Authorizing `identity:create_grant(project_id=cfede307bc0941b38a30fdbdeb09f473, user_id=6f521142f0604a5dad9294703b7e56cc, role_id=237966442f05411191105bf336bc14f3)` {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:26:05.638290 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1daef0e4-0ba2-467d-89e6-9cd4b5102022 admin admin] RBAC: Authorization granted {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:26:05.648391 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: [pid: 80377|app: 0|req: 1297/2579] 10.222.0.82 () {60 vars in 1483 bytes} [Tue Aug 2 10:26:05 2022] PUT /identity/v3/projects/cfede307bc0941b38a30fdbdeb09f473/users/6f521142f0604a5dad9294703b7e56cc/roles/237966442f05411191105bf336bc14f3 => generated 0 bytes in 32 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 02 10:26:05.659845 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d799f8ea-7c60-42f9-838a-3c128aaa826b None None] REQUEST_METHOD: `POST` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:26:05.660140 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d799f8ea-7c60-42f9-838a-3c128aaa826b None None] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:26:05.660421 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d799f8ea-7c60-42f9-838a-3c128aaa826b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:26:05.690203 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.auth.core [None req-d799f8ea-7c60-42f9-838a-3c128aaa826b None None] MFA Rules not processed for user `6f521142f0604a5dad9294703b7e56cc`. Rule list: `[]` (Enabled: `True`). {{(pid=80378) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 02 10:26:05.691349 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.fernet_utils [None req-d799f8ea-7c60-42f9-838a-3c128aaa826b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80378) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 10:26:05.742699 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG dbcounter [-] [80378] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80378) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 10:26:05.767978 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 1283/2580] 10.222.0.82 () {58 vars in 958 bytes} [Tue Aug 2 10:26:05 2022] POST /identity/v3/auth/tokens => generated 3323 bytes in 109 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 02 10:26:05.780217 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1daef0e4-0ba2-467d-89e6-9cd4b5102022 admin admin] Authenticating user token {{(pid=80377) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:26:05.782096 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.fernet_utils [None req-1daef0e4-0ba2-467d-89e6-9cd4b5102022 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80377) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 10:26:05.840001 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1daef0e4-0ba2-467d-89e6-9cd4b5102022 admin admin] Validating token access rules against request {{(pid=80377) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:26:05.843830 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e953dd3-418e-4aae-b9d2-29dbe9bdca4d service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1023de281ae74e9d8a6d6c312c1c97a7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f1b38ebb804c41c69772c998aae72c41', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80377) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:26:05.844457 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e953dd3-418e-4aae-b9d2-29dbe9bdca4d service cinder] REQUEST_METHOD: `GET` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:26:05.844743 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e953dd3-418e-4aae-b9d2-29dbe9bdca4d service cinder] SCRIPT_NAME: `/identity` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:26:05.845022 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e953dd3-418e-4aae-b9d2-29dbe9bdca4d service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:26:05.854988 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e953dd3-418e-4aae-b9d2-29dbe9bdca4d service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:26:05.856419 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e953dd3-418e-4aae-b9d2-29dbe9bdca4d service cinder] RBAC: Authorization granted {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:26:05.865704 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: [pid: 80377|app: 0|req: 1298/2581] 10.222.0.82 () {60 vars in 1365 bytes} [Tue Aug 2 10:26:05 2022] GET /identity/v3/auth/tokens => generated 3323 bytes in 86 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 02 10:26:06.175793 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d799f8ea-7c60-42f9-838a-3c128aaa826b None None] Authenticating user token {{(pid=80378) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:26:06.185153 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d799f8ea-7c60-42f9-838a-3c128aaa826b None None] Validating token access rules against request {{(pid=80378) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:26:06.189722 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e0e3906-59e6-4f4d-8dd4-ebb8c0757247 tempest-VolumeQuotaClassesTest-440405426 tempest-VolumeQuotaClassesTest-440405426-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f521142f0604a5dad9294703b7e56cc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cfede307bc0941b38a30fdbdeb09f473', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80378) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:26:06.190373 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e0e3906-59e6-4f4d-8dd4-ebb8c0757247 tempest-VolumeQuotaClassesTest-440405426 tempest-VolumeQuotaClassesTest-440405426-project-admin] REQUEST_METHOD: `GET` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:26:06.190669 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e0e3906-59e6-4f4d-8dd4-ebb8c0757247 tempest-VolumeQuotaClassesTest-440405426 tempest-VolumeQuotaClassesTest-440405426-project-admin] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:26:06.190952 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e0e3906-59e6-4f4d-8dd4-ebb8c0757247 tempest-VolumeQuotaClassesTest-440405426 tempest-VolumeQuotaClassesTest-440405426-project-admin] PATH_INFO: `/v3/domains` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:26:06.191675 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e0e3906-59e6-4f4d-8dd4-ebb8c0757247 tempest-VolumeQuotaClassesTest-440405426 tempest-VolumeQuotaClassesTest-440405426-project-admin] RBAC: Adding query filter params (name=Default) {{(pid=80378) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 02 10:26:06.191972 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e0e3906-59e6-4f4d-8dd4-ebb8c0757247 tempest-VolumeQuotaClassesTest-440405426 tempest-VolumeQuotaClassesTest-440405426-project-admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:26:06.193928 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e0e3906-59e6-4f4d-8dd4-ebb8c0757247 tempest-VolumeQuotaClassesTest-440405426 tempest-VolumeQuotaClassesTest-440405426-project-admin] RBAC: Authorization granted {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:26:06.205137 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 1284/2582] 10.222.0.82 () {58 vars in 1153 bytes} [Tue Aug 2 10:26:06 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 02 10:26:06.209101 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e953dd3-418e-4aae-b9d2-29dbe9bdca4d service cinder] Authenticating user token {{(pid=80377) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:26:06.218862 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e953dd3-418e-4aae-b9d2-29dbe9bdca4d service cinder] Validating token access rules against request {{(pid=80377) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:26:06.223355 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17ce002d-cb8a-4171-b51c-9bb00d6b3fac tempest-VolumeQuotaClassesTest-440405426 tempest-VolumeQuotaClassesTest-440405426-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f521142f0604a5dad9294703b7e56cc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cfede307bc0941b38a30fdbdeb09f473', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80377) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:26:06.223954 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17ce002d-cb8a-4171-b51c-9bb00d6b3fac tempest-VolumeQuotaClassesTest-440405426 tempest-VolumeQuotaClassesTest-440405426-project-admin] REQUEST_METHOD: `POST` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:26:06.224239 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17ce002d-cb8a-4171-b51c-9bb00d6b3fac tempest-VolumeQuotaClassesTest-440405426 tempest-VolumeQuotaClassesTest-440405426-project-admin] SCRIPT_NAME: `/identity` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:26:06.224516 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17ce002d-cb8a-4171-b51c-9bb00d6b3fac tempest-VolumeQuotaClassesTest-440405426 tempest-VolumeQuotaClassesTest-440405426-project-admin] PATH_INFO: `/v3/projects` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:26:06.225176 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17ce002d-cb8a-4171-b51c-9bb00d6b3fac tempest-VolumeQuotaClassesTest-440405426 tempest-VolumeQuotaClassesTest-440405426-project-admin] RBAC: Authorizing `identity:create_project()` {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:26:06.227223 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17ce002d-cb8a-4171-b51c-9bb00d6b3fac tempest-VolumeQuotaClassesTest-440405426 tempest-VolumeQuotaClassesTest-440405426-project-admin] RBAC: Authorization granted {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:26:06.254288 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: [pid: 80377|app: 0|req: 1299/2583] 10.222.0.82 () {60 vars in 1153 bytes} [Tue Aug 2 10:26:06 2022] POST /identity/v3/projects => generated 354 bytes in 46 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 02 10:26:06.258564 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e0e3906-59e6-4f4d-8dd4-ebb8c0757247 tempest-VolumeQuotaClassesTest-440405426 tempest-VolumeQuotaClassesTest-440405426-project-admin] Authenticating user token {{(pid=80378) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:26:06.269131 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e0e3906-59e6-4f4d-8dd4-ebb8c0757247 tempest-VolumeQuotaClassesTest-440405426 tempest-VolumeQuotaClassesTest-440405426-project-admin] Validating token access rules against request {{(pid=80378) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:26:06.323909 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2ce9990-f614-457b-ba1c-2b024d39a8ac tempest-VolumeQuotaClassesTest-440405426 tempest-VolumeQuotaClassesTest-440405426-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f521142f0604a5dad9294703b7e56cc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cfede307bc0941b38a30fdbdeb09f473', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80378) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:26:06.324534 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2ce9990-f614-457b-ba1c-2b024d39a8ac tempest-VolumeQuotaClassesTest-440405426 tempest-VolumeQuotaClassesTest-440405426-project-admin] REQUEST_METHOD: `GET` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:26:06.324822 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2ce9990-f614-457b-ba1c-2b024d39a8ac tempest-VolumeQuotaClassesTest-440405426 tempest-VolumeQuotaClassesTest-440405426-project-admin] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:26:06.325102 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2ce9990-f614-457b-ba1c-2b024d39a8ac tempest-VolumeQuotaClassesTest-440405426 tempest-VolumeQuotaClassesTest-440405426-project-admin] PATH_INFO: `/v3/domains` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:26:06.325805 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2ce9990-f614-457b-ba1c-2b024d39a8ac tempest-VolumeQuotaClassesTest-440405426 tempest-VolumeQuotaClassesTest-440405426-project-admin] RBAC: Adding query filter params (name=Default) {{(pid=80378) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 02 10:26:06.326203 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2ce9990-f614-457b-ba1c-2b024d39a8ac tempest-VolumeQuotaClassesTest-440405426 tempest-VolumeQuotaClassesTest-440405426-project-admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:26:06.327803 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2ce9990-f614-457b-ba1c-2b024d39a8ac tempest-VolumeQuotaClassesTest-440405426 tempest-VolumeQuotaClassesTest-440405426-project-admin] RBAC: Authorization granted {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:26:06.339823 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 1285/2584] 10.222.0.82 () {58 vars in 1153 bytes} [Tue Aug 2 10:26:06 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 82 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 02 10:26:06.379607 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17ce002d-cb8a-4171-b51c-9bb00d6b3fac tempest-VolumeQuotaClassesTest-440405426 tempest-VolumeQuotaClassesTest-440405426-project-admin] Authenticating user token {{(pid=80377) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:26:06.390153 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17ce002d-cb8a-4171-b51c-9bb00d6b3fac tempest-VolumeQuotaClassesTest-440405426 tempest-VolumeQuotaClassesTest-440405426-project-admin] Validating token access rules against request {{(pid=80377) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:26:06.398676 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a8ef4e5-24c7-4b2f-9d25-6ee132a28e94 tempest-VolumeQuotaClassesTest-440405426 tempest-VolumeQuotaClassesTest-440405426-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f521142f0604a5dad9294703b7e56cc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cfede307bc0941b38a30fdbdeb09f473', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80377) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:26:06.398676 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a8ef4e5-24c7-4b2f-9d25-6ee132a28e94 tempest-VolumeQuotaClassesTest-440405426 tempest-VolumeQuotaClassesTest-440405426-project-admin] REQUEST_METHOD: `DELETE` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:26:06.398676 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a8ef4e5-24c7-4b2f-9d25-6ee132a28e94 tempest-VolumeQuotaClassesTest-440405426 tempest-VolumeQuotaClassesTest-440405426-project-admin] SCRIPT_NAME: `/identity` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:26:06.399087 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a8ef4e5-24c7-4b2f-9d25-6ee132a28e94 tempest-VolumeQuotaClassesTest-440405426 tempest-VolumeQuotaClassesTest-440405426-project-admin] PATH_INFO: `/v3/projects/4b589f3e289447aeb6e30aa545f80e74` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:26:06.400353 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a8ef4e5-24c7-4b2f-9d25-6ee132a28e94 tempest-VolumeQuotaClassesTest-440405426 tempest-VolumeQuotaClassesTest-440405426-project-admin] RBAC: Authorizing `identity:delete_project(project_id=4b589f3e289447aeb6e30aa545f80e74)` {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:26:06.403422 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a8ef4e5-24c7-4b2f-9d25-6ee132a28e94 tempest-VolumeQuotaClassesTest-440405426 tempest-VolumeQuotaClassesTest-440405426-project-admin] RBAC: Authorization granted {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:26:06.420638 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG dbcounter [-] [80377] Writing DB stats keystone:SELECT=94,keystone:INSERT=6 {{(pid=80377) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 10:26:06.455635 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.notifications [None req-0a8ef4e5-24c7-4b2f-9d25-6ee132a28e94 tempest-VolumeQuotaClassesTest-440405426 tempest-VolumeQuotaClassesTest-440405426-project-admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '4b589f3e289447aeb6e30aa545f80e74', 'request_id': 'req-0a8ef4e5-24c7-4b2f-9d25-6ee132a28e94'} {{(pid=80377) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 10:26:06.461137 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.notifications [None req-0a8ef4e5-24c7-4b2f-9d25-6ee132a28e94 tempest-VolumeQuotaClassesTest-440405426 tempest-VolumeQuotaClassesTest-440405426-project-admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '4b589f3e289447aeb6e30aa545f80e74', 'request_id': 'req-0a8ef4e5-24c7-4b2f-9d25-6ee132a28e94'} {{(pid=80377) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 10:26:06.468069 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.notifications [None req-0a8ef4e5-24c7-4b2f-9d25-6ee132a28e94 tempest-VolumeQuotaClassesTest-440405426 tempest-VolumeQuotaClassesTest-440405426-project-admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '4b589f3e289447aeb6e30aa545f80e74', 'request_id': 'req-0a8ef4e5-24c7-4b2f-9d25-6ee132a28e94'} {{(pid=80377) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 10:26:06.483652 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.notifications [None req-0a8ef4e5-24c7-4b2f-9d25-6ee132a28e94 tempest-VolumeQuotaClassesTest-440405426 tempest-VolumeQuotaClassesTest-440405426-project-admin] The token cache is being invalidate because project 4b589f3e289447aeb6e30aa545f80e74 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80377) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 02 10:26:06.483950 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.notifications [None req-0a8ef4e5-24c7-4b2f-9d25-6ee132a28e94 tempest-VolumeQuotaClassesTest-440405426 tempest-VolumeQuotaClassesTest-440405426-project-admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80377) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 10:26:06.484610 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.notifications [None req-0a8ef4e5-24c7-4b2f-9d25-6ee132a28e94 tempest-VolumeQuotaClassesTest-440405426 tempest-VolumeQuotaClassesTest-440405426-project-admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80377) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 10:26:06.486193 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: [pid: 80377|app: 0|req: 1300/2585] 10.222.0.82 () {58 vars in 1233 bytes} [Tue Aug 2 10:26:06 2022] DELETE /identity/v3/projects/4b589f3e289447aeb6e30aa545f80e74 => generated 0 bytes in 107 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 02 10:26:06.690182 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2ce9990-f614-457b-ba1c-2b024d39a8ac tempest-VolumeQuotaClassesTest-440405426 tempest-VolumeQuotaClassesTest-440405426-project-admin] Authenticating user token {{(pid=80378) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:26:06.692006 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.fernet_utils [None req-f2ce9990-f614-457b-ba1c-2b024d39a8ac tempest-VolumeQuotaClassesTest-440405426 tempest-VolumeQuotaClassesTest-440405426-project-admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80378) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 10:26:06.761094 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2ce9990-f614-457b-ba1c-2b024d39a8ac tempest-VolumeQuotaClassesTest-440405426 tempest-VolumeQuotaClassesTest-440405426-project-admin] Validating token access rules against request {{(pid=80378) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:26:06.816549 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55ee38ce-8269-46b2-9723-7b30142c2c18 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '891005fa5ff74041b75747d16cc66798', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7047912c5acd4adeba5d87c3bf895e39', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80378) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:26:06.817195 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55ee38ce-8269-46b2-9723-7b30142c2c18 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:26:06.817513 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55ee38ce-8269-46b2-9723-7b30142c2c18 admin admin] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:26:06.818065 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55ee38ce-8269-46b2-9723-7b30142c2c18 admin admin] PATH_INFO: `/v3/users/fba7f9d4a7ed4dd5a02dd4cd23f57e9f` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:26:06.819255 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55ee38ce-8269-46b2-9723-7b30142c2c18 admin admin] RBAC: Authorizing `identity:delete_user(user_id=fba7f9d4a7ed4dd5a02dd4cd23f57e9f)` {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:26:06.821200 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55ee38ce-8269-46b2-9723-7b30142c2c18 admin admin] RBAC: Authorization granted {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:26:06.831034 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG dbcounter [-] [80378] Writing DB stats keystone:SELECT=100 {{(pid=80378) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 10:26:06.869292 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.notifications [None req-55ee38ce-8269-46b2-9723-7b30142c2c18 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'fba7f9d4a7ed4dd5a02dd4cd23f57e9f'} {{(pid=80378) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 10:26:06.877816 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.notifications [None req-55ee38ce-8269-46b2-9723-7b30142c2c18 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'fba7f9d4a7ed4dd5a02dd4cd23f57e9f'} {{(pid=80378) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 10:26:06.884656 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.notifications [None req-55ee38ce-8269-46b2-9723-7b30142c2c18 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'fba7f9d4a7ed4dd5a02dd4cd23f57e9f'} {{(pid=80378) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 10:26:06.885317 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.notifications [None req-55ee38ce-8269-46b2-9723-7b30142c2c18 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'fba7f9d4a7ed4dd5a02dd4cd23f57e9f'} {{(pid=80378) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 10:26:06.896848 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.notifications [None req-55ee38ce-8269-46b2-9723-7b30142c2c18 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'fba7f9d4a7ed4dd5a02dd4cd23f57e9f'} {{(pid=80378) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 10:26:06.903253 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 1286/2586] 10.222.0.82 () {58 vars in 1224 bytes} [Tue Aug 2 10:26:06 2022] DELETE /identity/v3/users/fba7f9d4a7ed4dd5a02dd4cd23f57e9f => generated 0 bytes in 214 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 02 10:26:06.907835 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a44581bb-4dbc-4d82-bfa0-1e94a7fd2492 None None] REQUEST_METHOD: `POST` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:26:06.908122 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a44581bb-4dbc-4d82-bfa0-1e94a7fd2492 None None] SCRIPT_NAME: `/identity` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:26:06.908400 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a44581bb-4dbc-4d82-bfa0-1e94a7fd2492 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:26:06.927583 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.auth.core [None req-a44581bb-4dbc-4d82-bfa0-1e94a7fd2492 None None] MFA Rules not processed for user `891005fa5ff74041b75747d16cc66798`. Rule list: `[]` (Enabled: `True`). {{(pid=80377) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 02 10:26:06.928784 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.fernet_utils [None req-a44581bb-4dbc-4d82-bfa0-1e94a7fd2492 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80377) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 10:26:06.986448 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: [pid: 80377|app: 0|req: 1301/2587] 10.222.0.82 () {58 vars in 958 bytes} [Tue Aug 2 10:26:06 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 80 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 02 10:26:06.996102 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55ee38ce-8269-46b2-9723-7b30142c2c18 admin admin] Authenticating user token {{(pid=80378) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:26:06.997502 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.fernet_utils [None req-55ee38ce-8269-46b2-9723-7b30142c2c18 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80378) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 10:26:07.055911 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55ee38ce-8269-46b2-9723-7b30142c2c18 admin admin] Validating token access rules against request {{(pid=80378) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:26:07.059905 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d0ff6ba-bb37-4de0-887f-52e0484343f8 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '33b87170f0634b67bce60750847b5cbe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f1b38ebb804c41c69772c998aae72c41', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80378) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:26:07.060537 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d0ff6ba-bb37-4de0-887f-52e0484343f8 service neutron] REQUEST_METHOD: `GET` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:26:07.060869 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d0ff6ba-bb37-4de0-887f-52e0484343f8 service neutron] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:26:07.061170 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d0ff6ba-bb37-4de0-887f-52e0484343f8 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:26:07.076243 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d0ff6ba-bb37-4de0-887f-52e0484343f8 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:26:07.077722 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d0ff6ba-bb37-4de0-887f-52e0484343f8 service neutron] RBAC: Authorization granted {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:26:07.088004 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 1287/2588] 10.222.0.82 () {60 vars in 1365 bytes} [Tue Aug 2 10:26:06 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 92 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 02 10:26:07.352400 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a44581bb-4dbc-4d82-bfa0-1e94a7fd2492 None None] Authenticating user token {{(pid=80377) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:26:07.354943 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.fernet_utils [None req-a44581bb-4dbc-4d82-bfa0-1e94a7fd2492 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80377) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 10:26:07.370150 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a44581bb-4dbc-4d82-bfa0-1e94a7fd2492 None None] Validating token access rules against request {{(pid=80377) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:26:07.376124 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1e022fa-d324-4faf-bb27-4241f6ed9b51 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '891005fa5ff74041b75747d16cc66798', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7047912c5acd4adeba5d87c3bf895e39', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80377) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:26:07.376527 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1e022fa-d324-4faf-bb27-4241f6ed9b51 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:26:07.376883 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1e022fa-d324-4faf-bb27-4241f6ed9b51 admin admin] SCRIPT_NAME: `/identity` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:26:07.377251 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1e022fa-d324-4faf-bb27-4241f6ed9b51 admin admin] PATH_INFO: `/v3/projects/fab5382451fe4e2db7711dcfc696c8d9` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:26:07.378644 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1e022fa-d324-4faf-bb27-4241f6ed9b51 admin admin] RBAC: Authorizing `identity:delete_project(project_id=fab5382451fe4e2db7711dcfc696c8d9)` {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:26:07.381111 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1e022fa-d324-4faf-bb27-4241f6ed9b51 admin admin] RBAC: Authorization granted {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:26:07.434657 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.notifications [None req-a1e022fa-d324-4faf-bb27-4241f6ed9b51 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'fab5382451fe4e2db7711dcfc696c8d9', 'request_id': 'req-a1e022fa-d324-4faf-bb27-4241f6ed9b51'} {{(pid=80377) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 10:26:07.440195 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.notifications [None req-a1e022fa-d324-4faf-bb27-4241f6ed9b51 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'fab5382451fe4e2db7711dcfc696c8d9', 'request_id': 'req-a1e022fa-d324-4faf-bb27-4241f6ed9b51'} {{(pid=80377) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 10:26:07.446658 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.notifications [None req-a1e022fa-d324-4faf-bb27-4241f6ed9b51 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'fab5382451fe4e2db7711dcfc696c8d9', 'request_id': 'req-a1e022fa-d324-4faf-bb27-4241f6ed9b51'} {{(pid=80377) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 10:26:07.459989 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.notifications [None req-a1e022fa-d324-4faf-bb27-4241f6ed9b51 admin admin] The token cache is being invalidate because project fab5382451fe4e2db7711dcfc696c8d9 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80377) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 02 10:26:07.460271 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.notifications [None req-a1e022fa-d324-4faf-bb27-4241f6ed9b51 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80377) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 10:26:07.461003 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.notifications [None req-a1e022fa-d324-4faf-bb27-4241f6ed9b51 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80377) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 10:26:07.463090 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: [pid: 80377|app: 0|req: 1302/2589] 10.222.0.82 () {58 vars in 1233 bytes} [Tue Aug 2 10:26:07 2022] DELETE /identity/v3/projects/fab5382451fe4e2db7711dcfc696c8d9 => generated 0 bytes in 111 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 02 10:26:07.467078 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d0ff6ba-bb37-4de0-887f-52e0484343f8 service neutron] Authenticating user token {{(pid=80378) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:26:07.476606 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d0ff6ba-bb37-4de0-887f-52e0484343f8 service neutron] Validating token access rules against request {{(pid=80378) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:26:07.531628 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f9ccf4f-e74d-46ca-8dc4-1676ec12f67d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '891005fa5ff74041b75747d16cc66798', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7047912c5acd4adeba5d87c3bf895e39', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80378) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:26:07.532269 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f9ccf4f-e74d-46ca-8dc4-1676ec12f67d admin admin] REQUEST_METHOD: `DELETE` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:26:07.532546 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f9ccf4f-e74d-46ca-8dc4-1676ec12f67d admin admin] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:26:07.532820 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f9ccf4f-e74d-46ca-8dc4-1676ec12f67d admin admin] PATH_INFO: `/v3/users/6f521142f0604a5dad9294703b7e56cc` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:26:07.534001 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f9ccf4f-e74d-46ca-8dc4-1676ec12f67d admin admin] RBAC: Authorizing `identity:delete_user(user_id=6f521142f0604a5dad9294703b7e56cc)` {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:26:07.535896 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f9ccf4f-e74d-46ca-8dc4-1676ec12f67d admin admin] RBAC: Authorization granted {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:26:07.570469 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG dbcounter [-] [80378] Writing DB stats keystone:DELETE=15,keystone:SELECT=84,keystone:INSERT=1 {{(pid=80378) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 10:26:07.585899 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.notifications [None req-3f9ccf4f-e74d-46ca-8dc4-1676ec12f67d admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '6f521142f0604a5dad9294703b7e56cc'} {{(pid=80378) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 10:26:07.594386 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.notifications [None req-3f9ccf4f-e74d-46ca-8dc4-1676ec12f67d admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '6f521142f0604a5dad9294703b7e56cc'} {{(pid=80378) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 10:26:07.601635 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.notifications [None req-3f9ccf4f-e74d-46ca-8dc4-1676ec12f67d admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '6f521142f0604a5dad9294703b7e56cc'} {{(pid=80378) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 10:26:07.602344 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.notifications [None req-3f9ccf4f-e74d-46ca-8dc4-1676ec12f67d admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '6f521142f0604a5dad9294703b7e56cc'} {{(pid=80378) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 10:26:07.614127 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.notifications [None req-3f9ccf4f-e74d-46ca-8dc4-1676ec12f67d admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '6f521142f0604a5dad9294703b7e56cc'} {{(pid=80378) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 10:26:07.620826 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 1288/2590] 10.222.0.82 () {58 vars in 1224 bytes} [Tue Aug 2 10:26:07 2022] DELETE /identity/v3/users/6f521142f0604a5dad9294703b7e56cc => generated 0 bytes in 154 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 02 10:26:07.902618 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1e022fa-d324-4faf-bb27-4241f6ed9b51 admin admin] Authenticating user token {{(pid=80377) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:26:07.905458 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.fernet_utils [None req-a1e022fa-d324-4faf-bb27-4241f6ed9b51 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80377) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 10:26:07.930553 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1e022fa-d324-4faf-bb27-4241f6ed9b51 admin admin] Validating token access rules against request {{(pid=80377) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:26:07.959217 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG dbcounter [-] [80377] Writing DB stats keystone:SELECT=84,keystone:DELETE=14,keystone:INSERT=2 {{(pid=80377) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 10:26:07.993196 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8583d5e3-6de6-4fc1-b1db-08debc06ab6e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '891005fa5ff74041b75747d16cc66798', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7047912c5acd4adeba5d87c3bf895e39', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80377) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:26:07.994071 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8583d5e3-6de6-4fc1-b1db-08debc06ab6e admin admin] REQUEST_METHOD: `DELETE` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:26:07.994458 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8583d5e3-6de6-4fc1-b1db-08debc06ab6e admin admin] SCRIPT_NAME: `/identity` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:26:07.994844 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8583d5e3-6de6-4fc1-b1db-08debc06ab6e admin admin] PATH_INFO: `/v3/projects/cfede307bc0941b38a30fdbdeb09f473` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:26:07.996159 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8583d5e3-6de6-4fc1-b1db-08debc06ab6e admin admin] RBAC: Authorizing `identity:delete_project(project_id=cfede307bc0941b38a30fdbdeb09f473)` {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:26:07.998222 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8583d5e3-6de6-4fc1-b1db-08debc06ab6e admin admin] RBAC: Authorization granted {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:26:08.047824 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.notifications [None req-8583d5e3-6de6-4fc1-b1db-08debc06ab6e admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'cfede307bc0941b38a30fdbdeb09f473', 'request_id': 'req-8583d5e3-6de6-4fc1-b1db-08debc06ab6e'} {{(pid=80377) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 10:26:08.053401 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.notifications [None req-8583d5e3-6de6-4fc1-b1db-08debc06ab6e admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'cfede307bc0941b38a30fdbdeb09f473', 'request_id': 'req-8583d5e3-6de6-4fc1-b1db-08debc06ab6e'} {{(pid=80377) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 10:26:08.060330 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.notifications [None req-8583d5e3-6de6-4fc1-b1db-08debc06ab6e admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'cfede307bc0941b38a30fdbdeb09f473', 'request_id': 'req-8583d5e3-6de6-4fc1-b1db-08debc06ab6e'} {{(pid=80377) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 10:26:08.075947 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.notifications [None req-8583d5e3-6de6-4fc1-b1db-08debc06ab6e admin admin] The token cache is being invalidate because project cfede307bc0941b38a30fdbdeb09f473 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80377) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 02 10:26:08.076248 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.notifications [None req-8583d5e3-6de6-4fc1-b1db-08debc06ab6e admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80377) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 10:26:08.076911 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.notifications [None req-8583d5e3-6de6-4fc1-b1db-08debc06ab6e admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80377) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 10:26:08.078484 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: [pid: 80377|app: 0|req: 1303/2591] 10.222.0.82 () {58 vars in 1233 bytes} [Tue Aug 2 10:26:07 2022] DELETE /identity/v3/projects/cfede307bc0941b38a30fdbdeb09f473 => generated 0 bytes in 177 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 02 10:26:08.083395 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b511dd54-0ce0-4a85-aad9-32e01c469b56 None None] REQUEST_METHOD: `POST` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:26:08.083689 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b511dd54-0ce0-4a85-aad9-32e01c469b56 None None] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:26:08.084021 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b511dd54-0ce0-4a85-aad9-32e01c469b56 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:26:08.104384 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.auth.core [None req-b511dd54-0ce0-4a85-aad9-32e01c469b56 None None] MFA Rules not processed for user `891005fa5ff74041b75747d16cc66798`. Rule list: `[]` (Enabled: `True`). {{(pid=80378) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 02 10:26:08.105512 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.fernet_utils [None req-b511dd54-0ce0-4a85-aad9-32e01c469b56 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80378) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 10:26:08.166567 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 1289/2592] 10.222.0.82 () {58 vars in 958 bytes} [Tue Aug 2 10:26:08 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 02 10:26:08.174363 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-add2afc9-8215-43c2-a5ea-ac32dbdb4937 None None] REQUEST_METHOD: `POST` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:26:08.174655 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-add2afc9-8215-43c2-a5ea-ac32dbdb4937 None None] SCRIPT_NAME: `/identity` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:26:08.174938 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-add2afc9-8215-43c2-a5ea-ac32dbdb4937 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:26:08.197000 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.auth.core [None req-add2afc9-8215-43c2-a5ea-ac32dbdb4937 None None] MFA Rules not processed for user `891005fa5ff74041b75747d16cc66798`. Rule list: `[]` (Enabled: `True`). {{(pid=80377) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 02 10:26:08.198392 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.fernet_utils [None req-add2afc9-8215-43c2-a5ea-ac32dbdb4937 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80377) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 10:26:08.207223 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: [pid: 80377|app: 0|req: 1304/2593] 10.222.0.82 () {58 vars in 958 bytes} [Tue Aug 2 10:26:08 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 34 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 02 10:26:08.211700 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b511dd54-0ce0-4a85-aad9-32e01c469b56 None None] Authenticating user token {{(pid=80378) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:26:08.222766 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b511dd54-0ce0-4a85-aad9-32e01c469b56 None None] Validating token access rules against request {{(pid=80378) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:26:08.227762 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89cdc4d0-01ab-49bf-bf5d-4e5a21502131 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '891005fa5ff74041b75747d16cc66798', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7047912c5acd4adeba5d87c3bf895e39', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80378) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:26:08.228416 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89cdc4d0-01ab-49bf-bf5d-4e5a21502131 admin admin] REQUEST_METHOD: `GET` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:26:08.228702 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89cdc4d0-01ab-49bf-bf5d-4e5a21502131 admin admin] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:26:08.228981 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89cdc4d0-01ab-49bf-bf5d-4e5a21502131 admin admin] PATH_INFO: `/v3/domains` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:26:08.229690 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89cdc4d0-01ab-49bf-bf5d-4e5a21502131 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80378) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 02 10:26:08.230040 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89cdc4d0-01ab-49bf-bf5d-4e5a21502131 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:26:08.231649 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89cdc4d0-01ab-49bf-bf5d-4e5a21502131 admin admin] RBAC: Authorization granted {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:26:08.250969 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-add2afc9-8215-43c2-a5ea-ac32dbdb4937 None None] Authenticating user token {{(pid=80377) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:26:08.256695 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 1290/2594] 10.222.0.82 () {58 vars in 1153 bytes} [Tue Aug 2 10:26:08 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 44 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 02 10:26:08.261496 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-add2afc9-8215-43c2-a5ea-ac32dbdb4937 None None] Validating token access rules against request {{(pid=80377) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:26:08.266455 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c049519a-869f-40bf-82d8-55e2f943eb03 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '891005fa5ff74041b75747d16cc66798', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7047912c5acd4adeba5d87c3bf895e39', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80377) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:26:08.267107 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c049519a-869f-40bf-82d8-55e2f943eb03 admin admin] REQUEST_METHOD: `POST` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:26:08.267456 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c049519a-869f-40bf-82d8-55e2f943eb03 admin admin] SCRIPT_NAME: `/identity` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:26:08.267740 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c049519a-869f-40bf-82d8-55e2f943eb03 admin admin] PATH_INFO: `/v3/projects` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:26:08.268451 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c049519a-869f-40bf-82d8-55e2f943eb03 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:26:08.270421 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c049519a-869f-40bf-82d8-55e2f943eb03 admin admin] RBAC: Authorization granted {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:26:08.302203 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: [pid: 80377|app: 0|req: 1305/2595] 10.222.0.82 () {60 vars in 1153 bytes} [Tue Aug 2 10:26:08 2022] POST /identity/v3/projects => generated 393 bytes in 52 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 02 10:26:08.306911 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89cdc4d0-01ab-49bf-bf5d-4e5a21502131 admin admin] Authenticating user token {{(pid=80378) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:26:08.317448 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89cdc4d0-01ab-49bf-bf5d-4e5a21502131 admin admin] Validating token access rules against request {{(pid=80378) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:26:08.378440 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50989578-c84f-423f-b046-a9069a699470 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '891005fa5ff74041b75747d16cc66798', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7047912c5acd4adeba5d87c3bf895e39', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80378) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:26:08.379122 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50989578-c84f-423f-b046-a9069a699470 admin admin] REQUEST_METHOD: `POST` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:26:08.379410 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50989578-c84f-423f-b046-a9069a699470 admin admin] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:26:08.379689 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50989578-c84f-423f-b046-a9069a699470 admin admin] PATH_INFO: `/v3/users` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:26:08.380372 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50989578-c84f-423f-b046-a9069a699470 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:26:08.382359 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50989578-c84f-423f-b046-a9069a699470 admin admin] RBAC: Authorization granted {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:26:08.407001 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 1291/2596] 10.222.0.82 () {60 vars in 1144 bytes} [Tue Aug 2 10:26:08 2022] POST /identity/v3/users => generated 297 bytes in 101 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 02 10:26:08.411186 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c049519a-869f-40bf-82d8-55e2f943eb03 admin admin] Authenticating user token {{(pid=80377) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:26:08.423053 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c049519a-869f-40bf-82d8-55e2f943eb03 admin admin] Validating token access rules against request {{(pid=80377) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:26:08.427762 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e355eda4-a771-4aaf-af02-317152d210f4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '891005fa5ff74041b75747d16cc66798', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7047912c5acd4adeba5d87c3bf895e39', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80377) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:26:08.428395 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e355eda4-a771-4aaf-af02-317152d210f4 admin admin] REQUEST_METHOD: `GET` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:26:08.428679 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e355eda4-a771-4aaf-af02-317152d210f4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:26:08.428956 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e355eda4-a771-4aaf-af02-317152d210f4 admin admin] PATH_INFO: `/v3/roles` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:26:08.429758 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e355eda4-a771-4aaf-af02-317152d210f4 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:26:08.431421 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e355eda4-a771-4aaf-af02-317152d210f4 admin admin] RBAC: Authorization granted {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:26:08.447341 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: [pid: 80377|app: 0|req: 1306/2597] 10.222.0.82 () {58 vars in 1122 bytes} [Tue Aug 2 10:26:08 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 02 10:26:08.449439 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50989578-c84f-423f-b046-a9069a699470 admin admin] Authenticating user token {{(pid=80378) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:26:08.460606 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50989578-c84f-423f-b046-a9069a699470 admin admin] Validating token access rules against request {{(pid=80378) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:26:08.466264 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32d14205-4049-4928-a0ab-95ee3e3137a1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '891005fa5ff74041b75747d16cc66798', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7047912c5acd4adeba5d87c3bf895e39', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80378) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:26:08.466849 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32d14205-4049-4928-a0ab-95ee3e3137a1 admin admin] REQUEST_METHOD: `GET` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:26:08.467152 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32d14205-4049-4928-a0ab-95ee3e3137a1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:26:08.467736 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32d14205-4049-4928-a0ab-95ee3e3137a1 admin admin] PATH_INFO: `/v3/roles` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:26:08.468121 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32d14205-4049-4928-a0ab-95ee3e3137a1 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:26:08.469715 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32d14205-4049-4928-a0ab-95ee3e3137a1 admin admin] RBAC: Authorization granted {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:26:08.484570 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 1292/2598] 10.222.0.82 () {58 vars in 1122 bytes} [Tue Aug 2 10:26:08 2022] GET /identity/v3/roles => generated 1420 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 02 10:26:08.488460 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e355eda4-a771-4aaf-af02-317152d210f4 admin admin] Authenticating user token {{(pid=80377) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:26:08.498963 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e355eda4-a771-4aaf-af02-317152d210f4 admin admin] Validating token access rules against request {{(pid=80377) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:26:08.504242 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b73c60b8-8627-4370-95a3-dfdd07809cb7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '891005fa5ff74041b75747d16cc66798', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7047912c5acd4adeba5d87c3bf895e39', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80377) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:26:08.505042 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b73c60b8-8627-4370-95a3-dfdd07809cb7 admin admin] REQUEST_METHOD: `PUT` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:26:08.505332 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b73c60b8-8627-4370-95a3-dfdd07809cb7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:26:08.505613 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b73c60b8-8627-4370-95a3-dfdd07809cb7 admin admin] PATH_INFO: `/v3/projects/b08856972e7447bbb79b521dbaa4fe42/users/924cb2120c8e448b8b992f6f62508b1f/roles/e1c04b19fc4543b8b54f58bbcffbc598` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:26:08.520715 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b73c60b8-8627-4370-95a3-dfdd07809cb7 admin admin] RBAC: Authorizing `identity:create_grant(project_id=b08856972e7447bbb79b521dbaa4fe42, user_id=924cb2120c8e448b8b992f6f62508b1f, role_id=e1c04b19fc4543b8b54f58bbcffbc598)` {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:26:08.523608 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b73c60b8-8627-4370-95a3-dfdd07809cb7 admin admin] RBAC: Authorization granted {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:26:08.537423 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: [pid: 80377|app: 0|req: 1307/2599] 10.222.0.82 () {60 vars in 1483 bytes} [Tue Aug 2 10:26:08 2022] PUT /identity/v3/projects/b08856972e7447bbb79b521dbaa4fe42/users/924cb2120c8e448b8b992f6f62508b1f/roles/e1c04b19fc4543b8b54f58bbcffbc598 => generated 0 bytes in 49 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 02 10:26:08.548753 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d6cf0ea-bde4-4d79-a217-20503bd8cfb7 None None] REQUEST_METHOD: `POST` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:26:08.549194 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d6cf0ea-bde4-4d79-a217-20503bd8cfb7 None None] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:26:08.549330 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d6cf0ea-bde4-4d79-a217-20503bd8cfb7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:26:08.578051 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.auth.core [None req-2d6cf0ea-bde4-4d79-a217-20503bd8cfb7 None None] MFA Rules not processed for user `924cb2120c8e448b8b992f6f62508b1f`. Rule list: `[]` (Enabled: `True`). {{(pid=80378) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 02 10:26:08.579296 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.fernet_utils [None req-2d6cf0ea-bde4-4d79-a217-20503bd8cfb7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80378) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 10:26:08.583510 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG dbcounter [-] [80378] Writing DB stats keystone:SELECT=91,keystone:DELETE=5,keystone:INSERT=4 {{(pid=80378) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 10:26:08.651496 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 1293/2600] 10.222.0.82 () {58 vars in 958 bytes} [Tue Aug 2 10:26:08 2022] POST /identity/v3/auth/tokens => generated 3270 bytes in 104 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 02 10:26:08.655536 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b73c60b8-8627-4370-95a3-dfdd07809cb7 admin admin] Authenticating user token {{(pid=80377) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:26:08.715327 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG dbcounter [-] [80377] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=80377) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 10:26:08.721854 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b73c60b8-8627-4370-95a3-dfdd07809cb7 admin admin] Validating token access rules against request {{(pid=80377) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:26:08.727990 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec8e5a9e-a36f-48d4-b04d-b18d62ee540a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '891005fa5ff74041b75747d16cc66798', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7047912c5acd4adeba5d87c3bf895e39', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80377) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:26:08.728774 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec8e5a9e-a36f-48d4-b04d-b18d62ee540a admin admin] REQUEST_METHOD: `POST` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:26:08.729165 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec8e5a9e-a36f-48d4-b04d-b18d62ee540a admin admin] SCRIPT_NAME: `/identity` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:26:08.729549 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec8e5a9e-a36f-48d4-b04d-b18d62ee540a admin admin] PATH_INFO: `/v3/projects` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:26:08.730361 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec8e5a9e-a36f-48d4-b04d-b18d62ee540a admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:26:08.732461 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec8e5a9e-a36f-48d4-b04d-b18d62ee540a admin admin] RBAC: Authorization granted {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:26:08.762850 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: [pid: 80377|app: 0|req: 1308/2601] 10.222.0.82 () {60 vars in 1153 bytes} [Tue Aug 2 10:26:08 2022] POST /identity/v3/projects => generated 391 bytes in 108 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 02 10:26:08.767234 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d6cf0ea-bde4-4d79-a217-20503bd8cfb7 None None] Authenticating user token {{(pid=80378) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:26:08.779431 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d6cf0ea-bde4-4d79-a217-20503bd8cfb7 None None] Validating token access rules against request {{(pid=80378) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:26:08.842344 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd7b21ab-276d-41a1-a41e-ddb6af12f64b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '891005fa5ff74041b75747d16cc66798', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7047912c5acd4adeba5d87c3bf895e39', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80378) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:26:08.843280 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd7b21ab-276d-41a1-a41e-ddb6af12f64b admin admin] REQUEST_METHOD: `POST` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:26:08.843628 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd7b21ab-276d-41a1-a41e-ddb6af12f64b admin admin] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:26:08.843906 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd7b21ab-276d-41a1-a41e-ddb6af12f64b admin admin] PATH_INFO: `/v3/users` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:26:08.844595 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd7b21ab-276d-41a1-a41e-ddb6af12f64b admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:26:08.846595 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd7b21ab-276d-41a1-a41e-ddb6af12f64b admin admin] RBAC: Authorization granted {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:26:08.868692 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 1294/2602] 10.222.0.82 () {60 vars in 1144 bytes} [Tue Aug 2 10:26:08 2022] POST /identity/v3/users => generated 302 bytes in 102 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 02 10:26:08.872799 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec8e5a9e-a36f-48d4-b04d-b18d62ee540a admin admin] Authenticating user token {{(pid=80377) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:26:08.883533 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec8e5a9e-a36f-48d4-b04d-b18d62ee540a admin admin] Validating token access rules against request {{(pid=80377) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:26:08.888830 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7e72f22-73c4-430e-a251-26fe5ed6c18a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '891005fa5ff74041b75747d16cc66798', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7047912c5acd4adeba5d87c3bf895e39', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80377) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:26:08.889455 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7e72f22-73c4-430e-a251-26fe5ed6c18a admin admin] REQUEST_METHOD: `GET` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:26:08.889762 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7e72f22-73c4-430e-a251-26fe5ed6c18a admin admin] SCRIPT_NAME: `/identity` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:26:08.890083 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7e72f22-73c4-430e-a251-26fe5ed6c18a admin admin] PATH_INFO: `/v3/roles` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:26:08.890788 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7e72f22-73c4-430e-a251-26fe5ed6c18a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:26:08.892377 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7e72f22-73c4-430e-a251-26fe5ed6c18a admin admin] RBAC: Authorization granted {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:26:08.904456 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: [pid: 80377|app: 0|req: 1309/2603] 10.222.0.82 () {58 vars in 1122 bytes} [Tue Aug 2 10:26:08 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 02 10:26:08.908989 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd7b21ab-276d-41a1-a41e-ddb6af12f64b admin admin] Authenticating user token {{(pid=80378) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:26:08.920378 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd7b21ab-276d-41a1-a41e-ddb6af12f64b admin admin] Validating token access rules against request {{(pid=80378) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:26:08.925433 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efb5eb4f-397b-46c2-bf72-22b210aadb86 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '891005fa5ff74041b75747d16cc66798', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7047912c5acd4adeba5d87c3bf895e39', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80378) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:26:08.926238 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efb5eb4f-397b-46c2-bf72-22b210aadb86 admin admin] REQUEST_METHOD: `PUT` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:26:08.926592 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efb5eb4f-397b-46c2-bf72-22b210aadb86 admin admin] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:26:08.926963 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efb5eb4f-397b-46c2-bf72-22b210aadb86 admin admin] PATH_INFO: `/v3/projects/4b66bab0c5754f7f877ad4a8c24c3468/users/bcbb127ff7894b0a83096e6edfe75523/roles/237966442f05411191105bf336bc14f3` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:26:08.950936 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-efb5eb4f-397b-46c2-bf72-22b210aadb86 admin admin] RBAC: Authorizing `identity:create_grant(project_id=4b66bab0c5754f7f877ad4a8c24c3468, user_id=bcbb127ff7894b0a83096e6edfe75523, role_id=237966442f05411191105bf336bc14f3)` {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:26:08.954551 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-efb5eb4f-397b-46c2-bf72-22b210aadb86 admin admin] RBAC: Authorization granted {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:26:08.974196 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 1295/2604] 10.222.0.82 () {60 vars in 1483 bytes} [Tue Aug 2 10:26:08 2022] PUT /identity/v3/projects/4b66bab0c5754f7f877ad4a8c24c3468/users/bcbb127ff7894b0a83096e6edfe75523/roles/237966442f05411191105bf336bc14f3 => generated 0 bytes in 65 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 02 10:26:08.978522 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7e72f22-73c4-430e-a251-26fe5ed6c18a admin admin] Authenticating user token {{(pid=80377) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:26:08.989852 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7e72f22-73c4-430e-a251-26fe5ed6c18a admin admin] Validating token access rules against request {{(pid=80377) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:26:09.051338 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-595548ee-bc1f-4ffe-a516-5803a68a82f3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '891005fa5ff74041b75747d16cc66798', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7047912c5acd4adeba5d87c3bf895e39', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80377) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:26:09.051967 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-595548ee-bc1f-4ffe-a516-5803a68a82f3 admin admin] REQUEST_METHOD: `GET` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:26:09.052251 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-595548ee-bc1f-4ffe-a516-5803a68a82f3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:26:09.052528 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-595548ee-bc1f-4ffe-a516-5803a68a82f3 admin admin] PATH_INFO: `/v3/roles` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:26:09.053236 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-595548ee-bc1f-4ffe-a516-5803a68a82f3 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:26:09.055003 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-595548ee-bc1f-4ffe-a516-5803a68a82f3 admin admin] RBAC: Authorization granted {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:26:09.070130 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: [pid: 80377|app: 0|req: 1310/2605] 10.222.0.82 () {58 vars in 1122 bytes} [Tue Aug 2 10:26:08 2022] GET /identity/v3/roles => generated 1420 bytes in 89 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 02 10:26:09.072094 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efb5eb4f-397b-46c2-bf72-22b210aadb86 admin admin] Authenticating user token {{(pid=80378) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:26:09.082928 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efb5eb4f-397b-46c2-bf72-22b210aadb86 admin admin] Validating token access rules against request {{(pid=80378) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:26:09.088025 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea1190e7-3b13-40e1-b0f4-44f5269425aa admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '891005fa5ff74041b75747d16cc66798', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7047912c5acd4adeba5d87c3bf895e39', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80378) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:26:09.088797 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea1190e7-3b13-40e1-b0f4-44f5269425aa admin admin] REQUEST_METHOD: `PUT` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:26:09.089156 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea1190e7-3b13-40e1-b0f4-44f5269425aa admin admin] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:26:09.089497 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea1190e7-3b13-40e1-b0f4-44f5269425aa admin admin] PATH_INFO: `/v3/projects/4b66bab0c5754f7f877ad4a8c24c3468/users/bcbb127ff7894b0a83096e6edfe75523/roles/237966442f05411191105bf336bc14f3` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:26:09.091585 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea1190e7-3b13-40e1-b0f4-44f5269425aa admin admin] RBAC: Authorizing `identity:create_grant(project_id=4b66bab0c5754f7f877ad4a8c24c3468, user_id=bcbb127ff7894b0a83096e6edfe75523, role_id=237966442f05411191105bf336bc14f3)` {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:26:09.094355 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea1190e7-3b13-40e1-b0f4-44f5269425aa admin admin] RBAC: Authorization granted {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:26:09.105704 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 1296/2606] 10.222.0.82 () {60 vars in 1483 bytes} [Tue Aug 2 10:26:09 2022] PUT /identity/v3/projects/4b66bab0c5754f7f877ad4a8c24c3468/users/bcbb127ff7894b0a83096e6edfe75523/roles/237966442f05411191105bf336bc14f3 => generated 0 bytes in 34 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 02 10:26:09.117099 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-514a1123-b92d-4d10-8db4-dfa3521eb7a9 None None] REQUEST_METHOD: `POST` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:26:09.117392 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-514a1123-b92d-4d10-8db4-dfa3521eb7a9 None None] SCRIPT_NAME: `/identity` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:26:09.117675 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-514a1123-b92d-4d10-8db4-dfa3521eb7a9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:26:09.150697 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.auth.core [None req-514a1123-b92d-4d10-8db4-dfa3521eb7a9 None None] MFA Rules not processed for user `bcbb127ff7894b0a83096e6edfe75523`. Rule list: `[]` (Enabled: `True`). {{(pid=80377) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 02 10:26:09.151926 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.fernet_utils [None req-514a1123-b92d-4d10-8db4-dfa3521eb7a9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80377) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 10:26:09.225976 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: [pid: 80377|app: 0|req: 1311/2607] 10.222.0.82 () {58 vars in 958 bytes} [Tue Aug 2 10:26:09 2022] POST /identity/v3/auth/tokens => generated 3335 bytes in 110 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 02 10:26:09.238247 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea1190e7-3b13-40e1-b0f4-44f5269425aa admin admin] Authenticating user token {{(pid=80378) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:26:09.240094 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.fernet_utils [None req-ea1190e7-3b13-40e1-b0f4-44f5269425aa admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80378) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 10:26:09.317223 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea1190e7-3b13-40e1-b0f4-44f5269425aa admin admin] Validating token access rules against request {{(pid=80378) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:26:09.322343 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-715343de-6809-43ac-a7a7-4adedbc8f251 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1023de281ae74e9d8a6d6c312c1c97a7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f1b38ebb804c41c69772c998aae72c41', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80378) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:26:09.323175 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-715343de-6809-43ac-a7a7-4adedbc8f251 service cinder] REQUEST_METHOD: `GET` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:26:09.323510 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-715343de-6809-43ac-a7a7-4adedbc8f251 service cinder] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:26:09.323886 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-715343de-6809-43ac-a7a7-4adedbc8f251 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:26:09.335931 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-715343de-6809-43ac-a7a7-4adedbc8f251 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:26:09.337923 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-715343de-6809-43ac-a7a7-4adedbc8f251 service cinder] RBAC: Authorization granted {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:26:09.344798 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG dbcounter [-] [80378] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80378) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 10:26:09.352310 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 1297/2608] 10.222.0.82 () {60 vars in 1365 bytes} [Tue Aug 2 10:26:09 2022] GET /identity/v3/auth/tokens => generated 3335 bytes in 115 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 02 10:26:09.409023 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-514a1123-b92d-4d10-8db4-dfa3521eb7a9 None None] Authenticating user token {{(pid=80377) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:26:09.411888 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.fernet_utils [None req-514a1123-b92d-4d10-8db4-dfa3521eb7a9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80377) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 10:26:09.425822 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-514a1123-b92d-4d10-8db4-dfa3521eb7a9 None None] Validating token access rules against request {{(pid=80377) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:26:09.431875 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7f8e289-0e8d-49d6-ab75-99455511e1b0 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1023de281ae74e9d8a6d6c312c1c97a7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f1b38ebb804c41c69772c998aae72c41', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80377) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:26:09.431875 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7f8e289-0e8d-49d6-ab75-99455511e1b0 service cinder] REQUEST_METHOD: `GET` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:26:09.431875 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7f8e289-0e8d-49d6-ab75-99455511e1b0 service cinder] SCRIPT_NAME: `/identity` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:26:09.431875 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7f8e289-0e8d-49d6-ab75-99455511e1b0 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:26:09.487631 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG dbcounter [-] [80377] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80377) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 10:26:09.511830 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7f8e289-0e8d-49d6-ab75-99455511e1b0 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:26:09.513704 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7f8e289-0e8d-49d6-ab75-99455511e1b0 service cinder] RBAC: Authorization granted {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:26:09.525243 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: [pid: 80377|app: 0|req: 1312/2609] 10.222.0.82 () {60 vars in 1365 bytes} [Tue Aug 2 10:26:09 2022] GET /identity/v3/auth/tokens => generated 3270 bytes in 117 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 02 10:26:12.873007 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-715343de-6809-43ac-a7a7-4adedbc8f251 service cinder] Authenticating user token {{(pid=80378) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:26:12.934598 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-715343de-6809-43ac-a7a7-4adedbc8f251 service cinder] Validating token access rules against request {{(pid=80378) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:26:12.939322 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0de0ab6f-92f9-487f-b240-d3d4ad71fc46 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '891005fa5ff74041b75747d16cc66798', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7047912c5acd4adeba5d87c3bf895e39', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80378) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:26:12.939996 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0de0ab6f-92f9-487f-b240-d3d4ad71fc46 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:26:12.940279 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0de0ab6f-92f9-487f-b240-d3d4ad71fc46 admin admin] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:26:12.940558 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0de0ab6f-92f9-487f-b240-d3d4ad71fc46 admin admin] PATH_INFO: `/v3/users/924cb2120c8e448b8b992f6f62508b1f` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:26:12.941733 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0de0ab6f-92f9-487f-b240-d3d4ad71fc46 admin admin] RBAC: Authorizing `identity:delete_user(user_id=924cb2120c8e448b8b992f6f62508b1f)` {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:26:12.944013 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0de0ab6f-92f9-487f-b240-d3d4ad71fc46 admin admin] RBAC: Authorization granted {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:26:12.988345 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.notifications [None req-0de0ab6f-92f9-487f-b240-d3d4ad71fc46 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '924cb2120c8e448b8b992f6f62508b1f'} {{(pid=80378) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 10:26:12.995865 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.notifications [None req-0de0ab6f-92f9-487f-b240-d3d4ad71fc46 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '924cb2120c8e448b8b992f6f62508b1f'} {{(pid=80378) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 10:26:13.002666 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.notifications [None req-0de0ab6f-92f9-487f-b240-d3d4ad71fc46 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '924cb2120c8e448b8b992f6f62508b1f'} {{(pid=80378) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 10:26:13.003295 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.notifications [None req-0de0ab6f-92f9-487f-b240-d3d4ad71fc46 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '924cb2120c8e448b8b992f6f62508b1f'} {{(pid=80378) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 10:26:13.014980 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.notifications [None req-0de0ab6f-92f9-487f-b240-d3d4ad71fc46 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '924cb2120c8e448b8b992f6f62508b1f'} {{(pid=80378) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 10:26:13.020733 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 1298/2610] 10.222.0.82 () {58 vars in 1224 bytes} [Tue Aug 2 10:26:12 2022] DELETE /identity/v3/users/924cb2120c8e448b8b992f6f62508b1f => generated 0 bytes in 148 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 02 10:26:13.025891 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-782dd530-fdda-46c7-81e6-39364dd37199 None None] REQUEST_METHOD: `POST` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:26:13.026379 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-782dd530-fdda-46c7-81e6-39364dd37199 None None] SCRIPT_NAME: `/identity` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:26:13.026482 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-782dd530-fdda-46c7-81e6-39364dd37199 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:26:13.045042 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.auth.core [None req-782dd530-fdda-46c7-81e6-39364dd37199 None None] MFA Rules not processed for user `891005fa5ff74041b75747d16cc66798`. Rule list: `[]` (Enabled: `True`). {{(pid=80377) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 02 10:26:13.046274 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.fernet_utils [None req-782dd530-fdda-46c7-81e6-39364dd37199 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80377) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 10:26:13.105117 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: [pid: 80377|app: 0|req: 1313/2611] 10.222.0.82 () {58 vars in 958 bytes} [Tue Aug 2 10:26:13 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 81 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 02 10:26:13.114804 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0de0ab6f-92f9-487f-b240-d3d4ad71fc46 admin admin] Authenticating user token {{(pid=80378) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:26:13.116241 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.fernet_utils [None req-0de0ab6f-92f9-487f-b240-d3d4ad71fc46 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80378) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 10:26:13.173312 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0de0ab6f-92f9-487f-b240-d3d4ad71fc46 admin admin] Validating token access rules against request {{(pid=80378) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:26:13.176974 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18aaf0d3-3372-497e-9656-8931320cca10 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '33b87170f0634b67bce60750847b5cbe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f1b38ebb804c41c69772c998aae72c41', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80378) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:26:13.177608 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18aaf0d3-3372-497e-9656-8931320cca10 service neutron] REQUEST_METHOD: `GET` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:26:13.177919 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18aaf0d3-3372-497e-9656-8931320cca10 service neutron] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:26:13.178190 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18aaf0d3-3372-497e-9656-8931320cca10 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:26:13.187610 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18aaf0d3-3372-497e-9656-8931320cca10 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:26:13.189052 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18aaf0d3-3372-497e-9656-8931320cca10 service neutron] RBAC: Authorization granted {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:26:13.198651 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 1299/2612] 10.222.0.82 () {60 vars in 1365 bytes} [Tue Aug 2 10:26:13 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 84 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 02 10:26:13.464127 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-782dd530-fdda-46c7-81e6-39364dd37199 None None] Authenticating user token {{(pid=80377) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:26:13.465644 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.fernet_utils [None req-782dd530-fdda-46c7-81e6-39364dd37199 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80377) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 10:26:13.478008 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-782dd530-fdda-46c7-81e6-39364dd37199 None None] Validating token access rules against request {{(pid=80377) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:26:13.482502 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6878a04-9fd7-4ae5-aa66-2232b7ec0b4a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '891005fa5ff74041b75747d16cc66798', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7047912c5acd4adeba5d87c3bf895e39', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80377) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:26:13.483161 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6878a04-9fd7-4ae5-aa66-2232b7ec0b4a admin admin] REQUEST_METHOD: `DELETE` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:26:13.483439 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6878a04-9fd7-4ae5-aa66-2232b7ec0b4a admin admin] SCRIPT_NAME: `/identity` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:26:13.483711 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6878a04-9fd7-4ae5-aa66-2232b7ec0b4a admin admin] PATH_INFO: `/v3/projects/b08856972e7447bbb79b521dbaa4fe42` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:26:13.484870 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6878a04-9fd7-4ae5-aa66-2232b7ec0b4a admin admin] RBAC: Authorizing `identity:delete_project(project_id=b08856972e7447bbb79b521dbaa4fe42)` {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:26:13.486871 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6878a04-9fd7-4ae5-aa66-2232b7ec0b4a admin admin] RBAC: Authorization granted {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:26:13.533313 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.notifications [None req-d6878a04-9fd7-4ae5-aa66-2232b7ec0b4a admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'b08856972e7447bbb79b521dbaa4fe42', 'request_id': 'req-d6878a04-9fd7-4ae5-aa66-2232b7ec0b4a'} {{(pid=80377) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 10:26:13.538828 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.notifications [None req-d6878a04-9fd7-4ae5-aa66-2232b7ec0b4a admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'b08856972e7447bbb79b521dbaa4fe42', 'request_id': 'req-d6878a04-9fd7-4ae5-aa66-2232b7ec0b4a'} {{(pid=80377) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 10:26:13.545316 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.notifications [None req-d6878a04-9fd7-4ae5-aa66-2232b7ec0b4a admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'b08856972e7447bbb79b521dbaa4fe42', 'request_id': 'req-d6878a04-9fd7-4ae5-aa66-2232b7ec0b4a'} {{(pid=80377) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 10:26:13.557399 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.notifications [None req-d6878a04-9fd7-4ae5-aa66-2232b7ec0b4a admin admin] The token cache is being invalidate because project b08856972e7447bbb79b521dbaa4fe42 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80377) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 02 10:26:13.557848 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.notifications [None req-d6878a04-9fd7-4ae5-aa66-2232b7ec0b4a admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80377) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 10:26:13.558393 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.notifications [None req-d6878a04-9fd7-4ae5-aa66-2232b7ec0b4a admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80377) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 10:26:13.559929 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: [pid: 80377|app: 0|req: 1314/2613] 10.222.0.82 () {58 vars in 1233 bytes} [Tue Aug 2 10:26:13 2022] DELETE /identity/v3/projects/b08856972e7447bbb79b521dbaa4fe42 => generated 0 bytes in 96 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 02 10:26:13.564077 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18aaf0d3-3372-497e-9656-8931320cca10 service neutron] Authenticating user token {{(pid=80378) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:26:13.577475 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18aaf0d3-3372-497e-9656-8931320cca10 service neutron] Validating token access rules against request {{(pid=80378) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:26:13.595854 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG dbcounter [-] [80378] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80378) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 10:26:13.636302 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad19287e-8cca-4a25-9330-400f0cf23f37 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '891005fa5ff74041b75747d16cc66798', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7047912c5acd4adeba5d87c3bf895e39', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80378) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:26:13.636942 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad19287e-8cca-4a25-9330-400f0cf23f37 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:26:13.637228 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad19287e-8cca-4a25-9330-400f0cf23f37 admin admin] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:26:13.637507 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad19287e-8cca-4a25-9330-400f0cf23f37 admin admin] PATH_INFO: `/v3/users/bcbb127ff7894b0a83096e6edfe75523` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:26:13.638787 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad19287e-8cca-4a25-9330-400f0cf23f37 admin admin] RBAC: Authorizing `identity:delete_user(user_id=bcbb127ff7894b0a83096e6edfe75523)` {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:26:13.640703 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad19287e-8cca-4a25-9330-400f0cf23f37 admin admin] RBAC: Authorization granted {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:26:13.686192 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.notifications [None req-ad19287e-8cca-4a25-9330-400f0cf23f37 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'bcbb127ff7894b0a83096e6edfe75523'} {{(pid=80378) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 10:26:13.693484 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.notifications [None req-ad19287e-8cca-4a25-9330-400f0cf23f37 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'bcbb127ff7894b0a83096e6edfe75523'} {{(pid=80378) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 10:26:13.701065 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.notifications [None req-ad19287e-8cca-4a25-9330-400f0cf23f37 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'bcbb127ff7894b0a83096e6edfe75523'} {{(pid=80378) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 10:26:13.701720 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.notifications [None req-ad19287e-8cca-4a25-9330-400f0cf23f37 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'bcbb127ff7894b0a83096e6edfe75523'} {{(pid=80378) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 10:26:13.713544 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.notifications [None req-ad19287e-8cca-4a25-9330-400f0cf23f37 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'bcbb127ff7894b0a83096e6edfe75523'} {{(pid=80378) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 10:26:13.719347 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 1300/2614] 10.222.0.82 () {58 vars in 1224 bytes} [Tue Aug 2 10:26:13 2022] DELETE /identity/v3/users/bcbb127ff7894b0a83096e6edfe75523 => generated 0 bytes in 156 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 02 10:26:13.990471 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6878a04-9fd7-4ae5-aa66-2232b7ec0b4a admin admin] Authenticating user token {{(pid=80377) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:26:13.991941 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.fernet_utils [None req-d6878a04-9fd7-4ae5-aa66-2232b7ec0b4a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80377) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 10:26:14.007124 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6878a04-9fd7-4ae5-aa66-2232b7ec0b4a admin admin] Validating token access rules against request {{(pid=80377) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:26:14.066153 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-472ea52d-39bd-4a77-9f7e-5a839fdf3cd0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '891005fa5ff74041b75747d16cc66798', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7047912c5acd4adeba5d87c3bf895e39', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80377) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:26:14.066856 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-472ea52d-39bd-4a77-9f7e-5a839fdf3cd0 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:26:14.067072 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-472ea52d-39bd-4a77-9f7e-5a839fdf3cd0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:26:14.067349 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-472ea52d-39bd-4a77-9f7e-5a839fdf3cd0 admin admin] PATH_INFO: `/v3/projects/4b66bab0c5754f7f877ad4a8c24c3468` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:26:14.068836 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-472ea52d-39bd-4a77-9f7e-5a839fdf3cd0 admin admin] RBAC: Authorizing `identity:delete_project(project_id=4b66bab0c5754f7f877ad4a8c24c3468)` {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:26:14.070741 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-472ea52d-39bd-4a77-9f7e-5a839fdf3cd0 admin admin] RBAC: Authorization granted {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:26:14.086157 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG dbcounter [-] [80377] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80377) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 10:26:14.117975 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.notifications [None req-472ea52d-39bd-4a77-9f7e-5a839fdf3cd0 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '4b66bab0c5754f7f877ad4a8c24c3468', 'request_id': 'req-472ea52d-39bd-4a77-9f7e-5a839fdf3cd0'} {{(pid=80377) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 10:26:14.123435 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.notifications [None req-472ea52d-39bd-4a77-9f7e-5a839fdf3cd0 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '4b66bab0c5754f7f877ad4a8c24c3468', 'request_id': 'req-472ea52d-39bd-4a77-9f7e-5a839fdf3cd0'} {{(pid=80377) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 10:26:14.129459 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.notifications [None req-472ea52d-39bd-4a77-9f7e-5a839fdf3cd0 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '4b66bab0c5754f7f877ad4a8c24c3468', 'request_id': 'req-472ea52d-39bd-4a77-9f7e-5a839fdf3cd0'} {{(pid=80377) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 10:26:14.143120 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.notifications [None req-472ea52d-39bd-4a77-9f7e-5a839fdf3cd0 admin admin] The token cache is being invalidate because project 4b66bab0c5754f7f877ad4a8c24c3468 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80377) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 02 10:26:14.143410 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.notifications [None req-472ea52d-39bd-4a77-9f7e-5a839fdf3cd0 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80377) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 10:26:14.144034 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.notifications [None req-472ea52d-39bd-4a77-9f7e-5a839fdf3cd0 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80377) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 10:26:14.145533 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: [pid: 80377|app: 0|req: 1315/2615] 10.222.0.82 () {58 vars in 1233 bytes} [Tue Aug 2 10:26:13 2022] DELETE /identity/v3/projects/4b66bab0c5754f7f877ad4a8c24c3468 => generated 0 bytes in 156 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 02 10:26:14.152473 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9872754b-36d7-414a-bd4d-782699881888 None None] REQUEST_METHOD: `POST` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:26:14.152963 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9872754b-36d7-414a-bd4d-782699881888 None None] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:26:14.153069 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9872754b-36d7-414a-bd4d-782699881888 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:26:14.172285 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.auth.core [None req-9872754b-36d7-414a-bd4d-782699881888 None None] MFA Rules not processed for user `891005fa5ff74041b75747d16cc66798`. Rule list: `[]` (Enabled: `True`). {{(pid=80378) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 02 10:26:14.173484 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.fernet_utils [None req-9872754b-36d7-414a-bd4d-782699881888 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80378) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 10:26:14.233052 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 1301/2616] 10.222.0.82 () {58 vars in 958 bytes} [Tue Aug 2 10:26:14 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 82 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 02 10:26:14.241002 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cb6962c-bcb1-46a6-83f3-275babc04f8d None None] REQUEST_METHOD: `POST` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:26:14.241323 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cb6962c-bcb1-46a6-83f3-275babc04f8d None None] SCRIPT_NAME: `/identity` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:26:14.241604 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cb6962c-bcb1-46a6-83f3-275babc04f8d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:26:14.262602 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.auth.core [None req-5cb6962c-bcb1-46a6-83f3-275babc04f8d None None] MFA Rules not processed for user `891005fa5ff74041b75747d16cc66798`. Rule list: `[]` (Enabled: `True`). {{(pid=80377) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 02 10:26:14.263829 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.fernet_utils [None req-5cb6962c-bcb1-46a6-83f3-275babc04f8d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80377) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 10:26:14.271823 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: [pid: 80377|app: 0|req: 1316/2617] 10.222.0.82 () {58 vars in 958 bytes} [Tue Aug 2 10:26:14 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 02 10:26:14.276019 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9872754b-36d7-414a-bd4d-782699881888 None None] Authenticating user token {{(pid=80378) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:26:14.285583 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9872754b-36d7-414a-bd4d-782699881888 None None] Validating token access rules against request {{(pid=80378) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:26:14.290179 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0939da80-b71a-4fbf-9fa5-4fcf08fa6052 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '891005fa5ff74041b75747d16cc66798', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7047912c5acd4adeba5d87c3bf895e39', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80378) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:26:14.290788 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0939da80-b71a-4fbf-9fa5-4fcf08fa6052 admin admin] REQUEST_METHOD: `GET` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:26:14.291075 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0939da80-b71a-4fbf-9fa5-4fcf08fa6052 admin admin] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:26:14.291353 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0939da80-b71a-4fbf-9fa5-4fcf08fa6052 admin admin] PATH_INFO: `/v3/domains` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:26:14.292072 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0939da80-b71a-4fbf-9fa5-4fcf08fa6052 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80378) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 02 10:26:14.292377 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0939da80-b71a-4fbf-9fa5-4fcf08fa6052 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:26:14.294073 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0939da80-b71a-4fbf-9fa5-4fcf08fa6052 admin admin] RBAC: Authorization granted {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:26:14.305376 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 1302/2618] 10.222.0.82 () {58 vars in 1153 bytes} [Tue Aug 2 10:26:14 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 02 10:26:14.310044 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5cb6962c-bcb1-46a6-83f3-275babc04f8d None None] Authenticating user token {{(pid=80377) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:26:14.319408 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5cb6962c-bcb1-46a6-83f3-275babc04f8d None None] Validating token access rules against request {{(pid=80377) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:26:14.324913 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43dbad5f-14e6-49ef-b6f3-eb7ef5f7899c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '891005fa5ff74041b75747d16cc66798', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7047912c5acd4adeba5d87c3bf895e39', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80377) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:26:14.325523 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43dbad5f-14e6-49ef-b6f3-eb7ef5f7899c admin admin] REQUEST_METHOD: `POST` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:26:14.325892 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43dbad5f-14e6-49ef-b6f3-eb7ef5f7899c admin admin] SCRIPT_NAME: `/identity` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:26:14.326175 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43dbad5f-14e6-49ef-b6f3-eb7ef5f7899c admin admin] PATH_INFO: `/v3/projects` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:26:14.326850 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43dbad5f-14e6-49ef-b6f3-eb7ef5f7899c admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:26:14.328740 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43dbad5f-14e6-49ef-b6f3-eb7ef5f7899c admin admin] RBAC: Authorization granted {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:26:14.356787 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: [pid: 80377|app: 0|req: 1317/2619] 10.222.0.82 () {60 vars in 1153 bytes} [Tue Aug 2 10:26:14 2022] POST /identity/v3/projects => generated 373 bytes in 47 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 02 10:26:14.360976 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0939da80-b71a-4fbf-9fa5-4fcf08fa6052 admin admin] Authenticating user token {{(pid=80378) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:26:14.371655 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0939da80-b71a-4fbf-9fa5-4fcf08fa6052 admin admin] Validating token access rules against request {{(pid=80378) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:26:14.397226 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG dbcounter [-] [80378] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80378) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 10:26:14.429282 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c2690c1-2fa8-42ae-83ad-e95963634ddb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '891005fa5ff74041b75747d16cc66798', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7047912c5acd4adeba5d87c3bf895e39', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80378) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:26:14.429935 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c2690c1-2fa8-42ae-83ad-e95963634ddb admin admin] REQUEST_METHOD: `POST` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:26:14.430219 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c2690c1-2fa8-42ae-83ad-e95963634ddb admin admin] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:26:14.430991 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c2690c1-2fa8-42ae-83ad-e95963634ddb admin admin] PATH_INFO: `/v3/users` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:26:14.431694 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c2690c1-2fa8-42ae-83ad-e95963634ddb admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:26:14.433606 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c2690c1-2fa8-42ae-83ad-e95963634ddb admin admin] RBAC: Authorization granted {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:26:14.455427 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 1303/2620] 10.222.0.82 () {60 vars in 1144 bytes} [Tue Aug 2 10:26:14 2022] POST /identity/v3/users => generated 287 bytes in 95 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 02 10:26:14.459831 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43dbad5f-14e6-49ef-b6f3-eb7ef5f7899c admin admin] Authenticating user token {{(pid=80377) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:26:14.470539 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43dbad5f-14e6-49ef-b6f3-eb7ef5f7899c admin admin] Validating token access rules against request {{(pid=80377) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:26:14.475355 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e6dc6d6-eb90-4a22-aedf-33f4e571755a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '891005fa5ff74041b75747d16cc66798', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7047912c5acd4adeba5d87c3bf895e39', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80377) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:26:14.476051 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e6dc6d6-eb90-4a22-aedf-33f4e571755a admin admin] REQUEST_METHOD: `GET` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:26:14.476328 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e6dc6d6-eb90-4a22-aedf-33f4e571755a admin admin] SCRIPT_NAME: `/identity` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:26:14.476601 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e6dc6d6-eb90-4a22-aedf-33f4e571755a admin admin] PATH_INFO: `/v3/roles` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:26:14.477319 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e6dc6d6-eb90-4a22-aedf-33f4e571755a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:26:14.478962 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e6dc6d6-eb90-4a22-aedf-33f4e571755a admin admin] RBAC: Authorization granted {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:26:14.490142 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: [pid: 80377|app: 0|req: 1318/2621] 10.222.0.82 () {58 vars in 1122 bytes} [Tue Aug 2 10:26:14 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 02 10:26:14.494724 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c2690c1-2fa8-42ae-83ad-e95963634ddb admin admin] Authenticating user token {{(pid=80378) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:26:14.505156 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c2690c1-2fa8-42ae-83ad-e95963634ddb admin admin] Validating token access rules against request {{(pid=80378) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:26:14.510794 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4a4245b-d0fb-419f-9d47-32b75202b187 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '891005fa5ff74041b75747d16cc66798', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7047912c5acd4adeba5d87c3bf895e39', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80378) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:26:14.511413 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4a4245b-d0fb-419f-9d47-32b75202b187 admin admin] REQUEST_METHOD: `GET` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:26:14.511698 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4a4245b-d0fb-419f-9d47-32b75202b187 admin admin] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:26:14.511974 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4a4245b-d0fb-419f-9d47-32b75202b187 admin admin] PATH_INFO: `/v3/roles` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:26:14.512683 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4a4245b-d0fb-419f-9d47-32b75202b187 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:26:14.514394 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4a4245b-d0fb-419f-9d47-32b75202b187 admin admin] RBAC: Authorization granted {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:26:14.525359 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 1304/2622] 10.222.0.82 () {58 vars in 1122 bytes} [Tue Aug 2 10:26:14 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 02 10:26:14.529633 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e6dc6d6-eb90-4a22-aedf-33f4e571755a admin admin] Authenticating user token {{(pid=80377) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:26:14.543781 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e6dc6d6-eb90-4a22-aedf-33f4e571755a admin admin] Validating token access rules against request {{(pid=80377) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:26:14.551269 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05240948-dcd9-4805-b849-f1ce38b0b38c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '891005fa5ff74041b75747d16cc66798', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7047912c5acd4adeba5d87c3bf895e39', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80377) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:26:14.551987 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05240948-dcd9-4805-b849-f1ce38b0b38c admin admin] REQUEST_METHOD: `PUT` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:26:14.552837 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05240948-dcd9-4805-b849-f1ce38b0b38c admin admin] SCRIPT_NAME: `/identity` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:26:14.552837 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05240948-dcd9-4805-b849-f1ce38b0b38c admin admin] PATH_INFO: `/v3/projects/c3211253f9154419a7511b40e0d1e291/users/5a01fa66107e4f989712cd883265a296/roles/e1c04b19fc4543b8b54f58bbcffbc598` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:26:14.569246 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05240948-dcd9-4805-b849-f1ce38b0b38c admin admin] RBAC: Authorizing `identity:create_grant(project_id=c3211253f9154419a7511b40e0d1e291, user_id=5a01fa66107e4f989712cd883265a296, role_id=e1c04b19fc4543b8b54f58bbcffbc598)` {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:26:14.572588 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05240948-dcd9-4805-b849-f1ce38b0b38c admin admin] RBAC: Authorization granted {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:26:14.586321 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: [pid: 80377|app: 0|req: 1319/2623] 10.222.0.82 () {60 vars in 1483 bytes} [Tue Aug 2 10:26:14 2022] PUT /identity/v3/projects/c3211253f9154419a7511b40e0d1e291/users/5a01fa66107e4f989712cd883265a296/roles/e1c04b19fc4543b8b54f58bbcffbc598 => generated 0 bytes in 57 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 02 10:26:14.596952 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5dbf6ac9-f5e2-4ecc-9c37-28be42397f71 None None] REQUEST_METHOD: `POST` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:26:14.597244 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5dbf6ac9-f5e2-4ecc-9c37-28be42397f71 None None] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:26:14.597528 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5dbf6ac9-f5e2-4ecc-9c37-28be42397f71 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:26:14.633092 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.auth.core [None req-5dbf6ac9-f5e2-4ecc-9c37-28be42397f71 None None] MFA Rules not processed for user `5a01fa66107e4f989712cd883265a296`. Rule list: `[]` (Enabled: `True`). {{(pid=80378) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 02 10:26:14.634388 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.fernet_utils [None req-5dbf6ac9-f5e2-4ecc-9c37-28be42397f71 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80378) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 10:26:14.703030 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 1305/2624] 10.222.0.82 () {58 vars in 958 bytes} [Tue Aug 2 10:26:14 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 107 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 02 10:26:14.714171 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05240948-dcd9-4805-b849-f1ce38b0b38c admin admin] Authenticating user token {{(pid=80377) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:26:14.715915 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.fernet_utils [None req-05240948-dcd9-4805-b849-f1ce38b0b38c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80377) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 10:26:14.775672 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05240948-dcd9-4805-b849-f1ce38b0b38c admin admin] Validating token access rules against request {{(pid=80377) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:26:14.779575 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39a68066-4831-4e97-a0a7-817a359897cc service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1023de281ae74e9d8a6d6c312c1c97a7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f1b38ebb804c41c69772c998aae72c41', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80377) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:26:14.780206 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39a68066-4831-4e97-a0a7-817a359897cc service cinder] REQUEST_METHOD: `GET` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:26:14.780502 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39a68066-4831-4e97-a0a7-817a359897cc service cinder] SCRIPT_NAME: `/identity` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:26:14.780763 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39a68066-4831-4e97-a0a7-817a359897cc service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:26:14.789608 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39a68066-4831-4e97-a0a7-817a359897cc service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:26:14.791602 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39a68066-4831-4e97-a0a7-817a359897cc service cinder] RBAC: Authorization granted {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:26:14.800764 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: [pid: 80377|app: 0|req: 1320/2625] 10.222.0.82 () {60 vars in 1365 bytes} [Tue Aug 2 10:26:14 2022] GET /identity/v3/auth/tokens => generated 3250 bytes in 87 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 02 10:26:16.160824 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72a04644-0325-489e-85a7-00f2d3e39fca None None] REQUEST_METHOD: `POST` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:26:16.161369 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72a04644-0325-489e-85a7-00f2d3e39fca None None] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:26:16.161369 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72a04644-0325-489e-85a7-00f2d3e39fca None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:26:16.182366 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.auth.core [None req-72a04644-0325-489e-85a7-00f2d3e39fca None None] MFA Rules not processed for user `5a01fa66107e4f989712cd883265a296`. Rule list: `[]` (Enabled: `True`). {{(pid=80378) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 02 10:26:16.184094 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.fernet_utils [None req-72a04644-0325-489e-85a7-00f2d3e39fca None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80378) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 10:26:16.190828 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 1306/2626] 10.222.0.82 () {58 vars in 958 bytes} [Tue Aug 2 10:26:16 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 02 10:26:19.463451 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a034deed-2380-4c8f-928b-1d1ace485102 None None] REQUEST_METHOD: `POST` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:26:19.463929 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a034deed-2380-4c8f-928b-1d1ace485102 None None] SCRIPT_NAME: `/identity` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:26:19.464016 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a034deed-2380-4c8f-928b-1d1ace485102 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:26:19.484376 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.auth.core [None req-a034deed-2380-4c8f-928b-1d1ace485102 None None] MFA Rules not processed for user `e08bc25f122445f592652671790cc658`. Rule list: `[]` (Enabled: `True`). {{(pid=80377) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 02 10:26:19.485584 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.fernet_utils [None req-a034deed-2380-4c8f-928b-1d1ace485102 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80377) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 10:26:19.529976 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG dbcounter [-] [80377] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=80377) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 10:26:19.546069 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: [pid: 80377|app: 0|req: 1321/2627] 10.222.0.82 () {58 vars in 958 bytes} [Tue Aug 2 10:26:19 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 02 10:26:26.176463 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG dbcounter [-] [80378] Writing DB stats keystone:SELECT=56,keystone:INSERT=3 {{(pid=80378) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 10:26:29.544133 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG dbcounter [-] [80377] Writing DB stats keystone:SELECT=6 {{(pid=80377) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 10:26:36.596282 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d5640a3-c7ad-41f9-8be6-7389256533f5 None None] REQUEST_METHOD: `POST` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:26:36.596749 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d5640a3-c7ad-41f9-8be6-7389256533f5 None None] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:26:36.596868 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d5640a3-c7ad-41f9-8be6-7389256533f5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:26:36.651923 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.auth.core [None req-7d5640a3-c7ad-41f9-8be6-7389256533f5 None None] MFA Rules not processed for user `bd8dd72086db48b8aaac2ed0058872df`. Rule list: `[]` (Enabled: `True`). {{(pid=80378) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 02 10:26:36.653160 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.fernet_utils [None req-7d5640a3-c7ad-41f9-8be6-7389256533f5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80378) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 10:26:36.719957 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 1307/2628] 10.222.0.82 () {58 vars in 1009 bytes} [Tue Aug 2 10:26:36 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 125 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 02 10:26:36.767308 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a034deed-2380-4c8f-928b-1d1ace485102 None None] Authenticating user token {{(pid=80377) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:26:36.768721 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.fernet_utils [None req-a034deed-2380-4c8f-928b-1d1ace485102 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80377) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 10:26:36.866320 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a034deed-2380-4c8f-928b-1d1ace485102 None None] Validating token access rules against request {{(pid=80377) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:26:36.872029 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9cfa12df-791d-4770-b55b-16b945780ed3 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bef1b2e714cb45c4a51eb6d3e15aad6d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f1b38ebb804c41c69772c998aae72c41', 'project_domain_id': 'default', 'roles': ['admin', 'service', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80377) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:26:36.872656 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cfa12df-791d-4770-b55b-16b945780ed3 service swift] REQUEST_METHOD: `GET` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:26:36.872946 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cfa12df-791d-4770-b55b-16b945780ed3 service swift] SCRIPT_NAME: `/identity` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:26:36.873225 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cfa12df-791d-4770-b55b-16b945780ed3 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:26:36.881652 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9cfa12df-791d-4770-b55b-16b945780ed3 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:26:36.883286 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9cfa12df-791d-4770-b55b-16b945780ed3 service swift] RBAC: Authorization granted {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:26:36.892284 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: [pid: 80377|app: 0|req: 1322/2629] 10.222.0.82 () {60 vars in 1384 bytes} [Tue Aug 2 10:26:36 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 126 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Aug 02 10:26:37.404336 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92344963-7a89-403a-986b-5e00c62378eb None None] REQUEST_METHOD: `POST` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:26:37.404632 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92344963-7a89-403a-986b-5e00c62378eb None None] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:26:37.404911 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92344963-7a89-403a-986b-5e00c62378eb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:26:37.424732 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.auth.core [None req-92344963-7a89-403a-986b-5e00c62378eb None None] MFA Rules not processed for user `bd8dd72086db48b8aaac2ed0058872df`. Rule list: `[]` (Enabled: `True`). {{(pid=80378) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 02 10:26:37.426031 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.fernet_utils [None req-92344963-7a89-403a-986b-5e00c62378eb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80378) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 10:26:37.434338 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 1308/2630] 10.222.0.82 () {58 vars in 967 bytes} [Tue Aug 2 10:26:37 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 02 10:26:37.451744 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9cfa12df-791d-4770-b55b-16b945780ed3 service swift] Authenticating user token {{(pid=80377) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:26:37.471901 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9cfa12df-791d-4770-b55b-16b945780ed3 service swift] Validating token access rules against request {{(pid=80377) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:26:37.478677 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b37539ba-a0e0-49d3-83ab-a0e0efd1b027 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bef1b2e714cb45c4a51eb6d3e15aad6d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f1b38ebb804c41c69772c998aae72c41', 'project_domain_id': 'default', 'roles': ['admin', 'service', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80377) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:26:37.479484 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b37539ba-a0e0-49d3-83ab-a0e0efd1b027 service swift] REQUEST_METHOD: `GET` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:26:37.480014 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b37539ba-a0e0-49d3-83ab-a0e0efd1b027 service swift] SCRIPT_NAME: `/identity` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:26:37.480574 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b37539ba-a0e0-49d3-83ab-a0e0efd1b027 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:26:37.490461 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b37539ba-a0e0-49d3-83ab-a0e0efd1b027 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:26:37.492524 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b37539ba-a0e0-49d3-83ab-a0e0efd1b027 service swift] RBAC: Authorization granted {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:26:37.505850 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: [pid: 80377|app: 0|req: 1323/2631] 10.222.0.82 () {60 vars in 1384 bytes} [Tue Aug 2 10:26:37 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 56 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Aug 02 10:26:37.791710 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b21deb9-b753-4c9d-9bb0-037965c7dc18 None None] REQUEST_METHOD: `POST` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:26:37.792241 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b21deb9-b753-4c9d-9bb0-037965c7dc18 None None] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:26:37.792345 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b21deb9-b753-4c9d-9bb0-037965c7dc18 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:26:37.812252 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.auth.core [None req-8b21deb9-b753-4c9d-9bb0-037965c7dc18 None None] MFA Rules not processed for user `bd8dd72086db48b8aaac2ed0058872df`. Rule list: `[]` (Enabled: `True`). {{(pid=80378) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 02 10:26:37.813537 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.fernet_utils [None req-8b21deb9-b753-4c9d-9bb0-037965c7dc18 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80378) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 10:26:37.821191 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 1309/2632] 10.222.0.82 () {58 vars in 1009 bytes} [Tue Aug 2 10:26:37 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 02 10:26:37.833765 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b37539ba-a0e0-49d3-83ab-a0e0efd1b027 service swift] Authenticating user token {{(pid=80377) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:26:37.845073 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b37539ba-a0e0-49d3-83ab-a0e0efd1b027 service swift] Validating token access rules against request {{(pid=80377) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:26:37.853646 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4af96e53-6fb6-4f8e-84de-c93a7c25455b service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bef1b2e714cb45c4a51eb6d3e15aad6d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f1b38ebb804c41c69772c998aae72c41', 'project_domain_id': 'default', 'roles': ['admin', 'service', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80377) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:26:37.854408 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4af96e53-6fb6-4f8e-84de-c93a7c25455b service swift] REQUEST_METHOD: `GET` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:26:37.854692 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4af96e53-6fb6-4f8e-84de-c93a7c25455b service swift] SCRIPT_NAME: `/identity` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:26:37.854965 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4af96e53-6fb6-4f8e-84de-c93a7c25455b service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:26:37.863814 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4af96e53-6fb6-4f8e-84de-c93a7c25455b service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:26:37.865549 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4af96e53-6fb6-4f8e-84de-c93a7c25455b service swift] RBAC: Authorization granted {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:26:37.880088 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: [pid: 80377|app: 0|req: 1324/2633] 10.222.0.82 () {60 vars in 1384 bytes} [Tue Aug 2 10:26:37 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 41 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Aug 02 10:26:38.223926 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-909e2745-4383-4546-b597-aa908887cc29 None None] REQUEST_METHOD: `POST` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:26:38.224293 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-909e2745-4383-4546-b597-aa908887cc29 None None] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:26:38.224574 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-909e2745-4383-4546-b597-aa908887cc29 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:26:38.244548 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.auth.core [None req-909e2745-4383-4546-b597-aa908887cc29 None None] MFA Rules not processed for user `bd8dd72086db48b8aaac2ed0058872df`. Rule list: `[]` (Enabled: `True`). {{(pid=80378) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 02 10:26:38.245744 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.fernet_utils [None req-909e2745-4383-4546-b597-aa908887cc29 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80378) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 10:26:38.252735 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 1310/2634] 10.222.0.82 () {58 vars in 967 bytes} [Tue Aug 2 10:26:38 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 02 10:26:38.265039 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4af96e53-6fb6-4f8e-84de-c93a7c25455b service swift] Authenticating user token {{(pid=80377) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:26:38.276663 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4af96e53-6fb6-4f8e-84de-c93a7c25455b service swift] Validating token access rules against request {{(pid=80377) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:26:38.281692 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7e531b2-4723-49c6-afd0-dbd06f75eaf5 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bef1b2e714cb45c4a51eb6d3e15aad6d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f1b38ebb804c41c69772c998aae72c41', 'project_domain_id': 'default', 'roles': ['admin', 'service', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80377) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:26:38.282371 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7e531b2-4723-49c6-afd0-dbd06f75eaf5 service swift] REQUEST_METHOD: `GET` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:26:38.282680 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7e531b2-4723-49c6-afd0-dbd06f75eaf5 service swift] SCRIPT_NAME: `/identity` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:26:38.282958 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7e531b2-4723-49c6-afd0-dbd06f75eaf5 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:26:38.291955 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7e531b2-4723-49c6-afd0-dbd06f75eaf5 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:26:38.293468 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7e531b2-4723-49c6-afd0-dbd06f75eaf5 service swift] RBAC: Authorization granted {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:26:38.301403 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: [pid: 80377|app: 0|req: 1325/2635] 10.222.0.82 () {60 vars in 1384 bytes} [Tue Aug 2 10:26:38 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 37 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Aug 02 10:26:40.425478 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-909e2745-4383-4546-b597-aa908887cc29 None None] Authenticating user token {{(pid=80378) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:26:40.470330 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7e531b2-4723-49c6-afd0-dbd06f75eaf5 service swift] Authenticating user token {{(pid=80377) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:26:40.471863 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.fernet_utils [None req-c7e531b2-4723-49c6-afd0-dbd06f75eaf5 service swift] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80377) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 10:26:40.500515 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-909e2745-4383-4546-b597-aa908887cc29 None None] Validating token access rules against request {{(pid=80378) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:26:40.505463 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9101833a-8196-49e4-b52d-350b1af0b9c8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '891005fa5ff74041b75747d16cc66798', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7047912c5acd4adeba5d87c3bf895e39', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80378) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:26:40.506056 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9101833a-8196-49e4-b52d-350b1af0b9c8 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:26:40.506342 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9101833a-8196-49e4-b52d-350b1af0b9c8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:26:40.506621 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9101833a-8196-49e4-b52d-350b1af0b9c8 admin admin] PATH_INFO: `/v3/users/5a01fa66107e4f989712cd883265a296` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:26:40.507841 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9101833a-8196-49e4-b52d-350b1af0b9c8 admin admin] RBAC: Authorizing `identity:delete_user(user_id=5a01fa66107e4f989712cd883265a296)` {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:26:40.510435 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9101833a-8196-49e4-b52d-350b1af0b9c8 admin admin] RBAC: Authorization granted {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:26:40.529871 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7e531b2-4723-49c6-afd0-dbd06f75eaf5 service swift] Validating token access rules against request {{(pid=80377) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:26:40.533495 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef8d4a01-a71c-4f37-8544-230e1e7ceffa service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '33b87170f0634b67bce60750847b5cbe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f1b38ebb804c41c69772c998aae72c41', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80377) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:26:40.534224 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef8d4a01-a71c-4f37-8544-230e1e7ceffa service neutron] REQUEST_METHOD: `GET` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:26:40.534568 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef8d4a01-a71c-4f37-8544-230e1e7ceffa service neutron] SCRIPT_NAME: `/identity` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:26:40.534854 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef8d4a01-a71c-4f37-8544-230e1e7ceffa service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:26:40.536986 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.fernet_utils [None req-ef8d4a01-a71c-4f37-8544-230e1e7ceffa service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80377) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 10:26:40.550022 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef8d4a01-a71c-4f37-8544-230e1e7ceffa service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:26:40.551562 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef8d4a01-a71c-4f37-8544-230e1e7ceffa service neutron] RBAC: Authorization granted {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:26:40.558772 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.notifications [None req-9101833a-8196-49e4-b52d-350b1af0b9c8 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '5a01fa66107e4f989712cd883265a296'} {{(pid=80378) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 10:26:40.562169 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: [pid: 80377|app: 0|req: 1326/2636] 10.222.0.82 () {60 vars in 1365 bytes} [Tue Aug 2 10:26:40 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 93 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 02 10:26:40.580903 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.notifications [None req-9101833a-8196-49e4-b52d-350b1af0b9c8 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '5a01fa66107e4f989712cd883265a296'} {{(pid=80378) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 10:26:40.582597 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG dbcounter [-] [80378] Writing DB stats keystone:SELECT=91,keystone:DELETE=8,keystone:INSERT=1 {{(pid=80378) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 10:26:40.590497 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.notifications [None req-9101833a-8196-49e4-b52d-350b1af0b9c8 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '5a01fa66107e4f989712cd883265a296'} {{(pid=80378) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 10:26:40.591586 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.notifications [None req-9101833a-8196-49e4-b52d-350b1af0b9c8 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '5a01fa66107e4f989712cd883265a296'} {{(pid=80378) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 10:26:40.604243 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.notifications [None req-9101833a-8196-49e4-b52d-350b1af0b9c8 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '5a01fa66107e4f989712cd883265a296'} {{(pid=80378) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 10:26:40.610426 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 1311/2637] 10.222.0.82 () {58 vars in 1224 bytes} [Tue Aug 2 10:26:40 2022] DELETE /identity/v3/users/5a01fa66107e4f989712cd883265a296 => generated 0 bytes in 189 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 02 10:26:40.615107 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03837ce7-a90f-490f-9038-1288a4c9394d None None] REQUEST_METHOD: `POST` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:26:40.615488 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03837ce7-a90f-490f-9038-1288a4c9394d None None] SCRIPT_NAME: `/identity` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:26:40.615767 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03837ce7-a90f-490f-9038-1288a4c9394d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:26:40.636974 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.auth.core [None req-03837ce7-a90f-490f-9038-1288a4c9394d None None] MFA Rules not processed for user `891005fa5ff74041b75747d16cc66798`. Rule list: `[]` (Enabled: `True`). {{(pid=80377) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 02 10:26:40.638152 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.fernet_utils [None req-03837ce7-a90f-490f-9038-1288a4c9394d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80377) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 10:26:40.668659 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG dbcounter [-] [80377] Writing DB stats keystone:SELECT=100 {{(pid=80377) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 10:26:40.701987 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: [pid: 80377|app: 0|req: 1327/2638] 10.222.0.82 () {58 vars in 958 bytes} [Tue Aug 2 10:26:40 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 02 10:26:40.712025 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9101833a-8196-49e4-b52d-350b1af0b9c8 admin admin] Authenticating user token {{(pid=80378) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:26:40.713775 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.fernet_utils [None req-9101833a-8196-49e4-b52d-350b1af0b9c8 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80378) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 10:26:40.782603 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9101833a-8196-49e4-b52d-350b1af0b9c8 admin admin] Validating token access rules against request {{(pid=80378) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:26:40.786276 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe0699c1-dc57-464b-8e12-92068fd8ff98 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '33b87170f0634b67bce60750847b5cbe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f1b38ebb804c41c69772c998aae72c41', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80378) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:26:40.786912 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe0699c1-dc57-464b-8e12-92068fd8ff98 service neutron] REQUEST_METHOD: `GET` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:26:40.787198 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe0699c1-dc57-464b-8e12-92068fd8ff98 service neutron] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:26:40.787505 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe0699c1-dc57-464b-8e12-92068fd8ff98 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:26:40.798123 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe0699c1-dc57-464b-8e12-92068fd8ff98 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:26:40.799574 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe0699c1-dc57-464b-8e12-92068fd8ff98 service neutron] RBAC: Authorization granted {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:26:40.810377 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 1312/2639] 10.222.0.82 () {60 vars in 1365 bytes} [Tue Aug 2 10:26:40 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 99 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 02 10:26:40.989466 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03837ce7-a90f-490f-9038-1288a4c9394d None None] Authenticating user token {{(pid=80377) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:26:40.991085 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.fernet_utils [None req-03837ce7-a90f-490f-9038-1288a4c9394d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80377) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 10:26:41.010375 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03837ce7-a90f-490f-9038-1288a4c9394d None None] Validating token access rules against request {{(pid=80377) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:26:41.015100 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bb5dbf6-aaa5-4a1b-9bfc-bfea9197562b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '891005fa5ff74041b75747d16cc66798', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7047912c5acd4adeba5d87c3bf895e39', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80377) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:26:41.015767 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bb5dbf6-aaa5-4a1b-9bfc-bfea9197562b admin admin] REQUEST_METHOD: `DELETE` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:26:41.016051 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bb5dbf6-aaa5-4a1b-9bfc-bfea9197562b admin admin] SCRIPT_NAME: `/identity` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:26:41.016335 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bb5dbf6-aaa5-4a1b-9bfc-bfea9197562b admin admin] PATH_INFO: `/v3/projects/c3211253f9154419a7511b40e0d1e291` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:26:41.017523 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4bb5dbf6-aaa5-4a1b-9bfc-bfea9197562b admin admin] RBAC: Authorizing `identity:delete_project(project_id=c3211253f9154419a7511b40e0d1e291)` {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:26:41.019506 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4bb5dbf6-aaa5-4a1b-9bfc-bfea9197562b admin admin] RBAC: Authorization granted {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:26:41.080644 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.notifications [None req-4bb5dbf6-aaa5-4a1b-9bfc-bfea9197562b admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'c3211253f9154419a7511b40e0d1e291', 'request_id': 'req-4bb5dbf6-aaa5-4a1b-9bfc-bfea9197562b'} {{(pid=80377) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 10:26:41.088249 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.notifications [None req-4bb5dbf6-aaa5-4a1b-9bfc-bfea9197562b admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'c3211253f9154419a7511b40e0d1e291', 'request_id': 'req-4bb5dbf6-aaa5-4a1b-9bfc-bfea9197562b'} {{(pid=80377) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 10:26:41.095852 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.notifications [None req-4bb5dbf6-aaa5-4a1b-9bfc-bfea9197562b admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'c3211253f9154419a7511b40e0d1e291', 'request_id': 'req-4bb5dbf6-aaa5-4a1b-9bfc-bfea9197562b'} {{(pid=80377) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 10:26:41.111062 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.notifications [None req-4bb5dbf6-aaa5-4a1b-9bfc-bfea9197562b admin admin] The token cache is being invalidate because project c3211253f9154419a7511b40e0d1e291 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80377) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 02 10:26:41.111641 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.notifications [None req-4bb5dbf6-aaa5-4a1b-9bfc-bfea9197562b admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80377) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 10:26:41.112383 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.notifications [None req-4bb5dbf6-aaa5-4a1b-9bfc-bfea9197562b admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80377) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 10:26:41.114005 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: [pid: 80377|app: 0|req: 1328/2640] 10.222.0.82 () {58 vars in 1233 bytes} [Tue Aug 2 10:26:40 2022] DELETE /identity/v3/projects/c3211253f9154419a7511b40e0d1e291 => generated 0 bytes in 125 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 02 10:26:41.119422 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7ae4005-2bdd-4117-8cce-18b3c25268d6 None None] REQUEST_METHOD: `POST` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:26:41.119717 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7ae4005-2bdd-4117-8cce-18b3c25268d6 None None] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:26:41.119999 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7ae4005-2bdd-4117-8cce-18b3c25268d6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:26:41.139612 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.auth.core [None req-b7ae4005-2bdd-4117-8cce-18b3c25268d6 None None] MFA Rules not processed for user `891005fa5ff74041b75747d16cc66798`. Rule list: `[]` (Enabled: `True`). {{(pid=80378) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 02 10:26:41.140816 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.fernet_utils [None req-b7ae4005-2bdd-4117-8cce-18b3c25268d6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80378) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 10:26:41.208837 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 1313/2641] 10.222.0.82 () {58 vars in 958 bytes} [Tue Aug 2 10:26:41 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 91 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 02 10:26:41.217734 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e8bc5b7-1de4-4b42-ba9f-027341827404 None None] REQUEST_METHOD: `POST` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:26:41.218121 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e8bc5b7-1de4-4b42-ba9f-027341827404 None None] SCRIPT_NAME: `/identity` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:26:41.218402 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e8bc5b7-1de4-4b42-ba9f-027341827404 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:26:41.241450 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.auth.core [None req-3e8bc5b7-1de4-4b42-ba9f-027341827404 None None] MFA Rules not processed for user `891005fa5ff74041b75747d16cc66798`. Rule list: `[]` (Enabled: `True`). {{(pid=80377) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 02 10:26:41.242730 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.fernet_utils [None req-3e8bc5b7-1de4-4b42-ba9f-027341827404 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80377) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 10:26:41.250863 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: [pid: 80377|app: 0|req: 1329/2642] 10.222.0.82 () {58 vars in 958 bytes} [Tue Aug 2 10:26:41 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 34 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 02 10:26:41.255332 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7ae4005-2bdd-4117-8cce-18b3c25268d6 None None] Authenticating user token {{(pid=80378) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:26:41.266273 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7ae4005-2bdd-4117-8cce-18b3c25268d6 None None] Validating token access rules against request {{(pid=80378) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:26:41.270982 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be853ca3-5a62-4dbe-8d94-175f6761159f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '891005fa5ff74041b75747d16cc66798', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7047912c5acd4adeba5d87c3bf895e39', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80378) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:26:41.271682 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be853ca3-5a62-4dbe-8d94-175f6761159f admin admin] REQUEST_METHOD: `GET` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:26:41.272032 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be853ca3-5a62-4dbe-8d94-175f6761159f admin admin] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:26:41.272374 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be853ca3-5a62-4dbe-8d94-175f6761159f admin admin] PATH_INFO: `/v3/domains` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:26:41.273164 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be853ca3-5a62-4dbe-8d94-175f6761159f admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80378) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 02 10:26:41.273546 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be853ca3-5a62-4dbe-8d94-175f6761159f admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:26:41.275747 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be853ca3-5a62-4dbe-8d94-175f6761159f admin admin] RBAC: Authorization granted {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:26:41.288221 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 1314/2643] 10.222.0.82 () {58 vars in 1153 bytes} [Tue Aug 2 10:26:41 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 02 10:26:41.293278 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e8bc5b7-1de4-4b42-ba9f-027341827404 None None] Authenticating user token {{(pid=80377) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:26:41.302610 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e8bc5b7-1de4-4b42-ba9f-027341827404 None None] Validating token access rules against request {{(pid=80377) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:26:41.308561 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f730ec34-fdfd-41f8-b927-47ab4c132f6d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '891005fa5ff74041b75747d16cc66798', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7047912c5acd4adeba5d87c3bf895e39', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80377) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:26:41.309282 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f730ec34-fdfd-41f8-b927-47ab4c132f6d admin admin] REQUEST_METHOD: `POST` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:26:41.310296 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f730ec34-fdfd-41f8-b927-47ab4c132f6d admin admin] SCRIPT_NAME: `/identity` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:26:41.310296 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f730ec34-fdfd-41f8-b927-47ab4c132f6d admin admin] PATH_INFO: `/v3/projects` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:26:41.310700 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f730ec34-fdfd-41f8-b927-47ab4c132f6d admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:26:41.312644 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f730ec34-fdfd-41f8-b927-47ab4c132f6d admin admin] RBAC: Authorization granted {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:26:41.341889 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: [pid: 80377|app: 0|req: 1330/2644] 10.222.0.82 () {60 vars in 1153 bytes} [Tue Aug 2 10:26:41 2022] POST /identity/v3/projects => generated 399 bytes in 49 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 02 10:26:41.346195 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be853ca3-5a62-4dbe-8d94-175f6761159f admin admin] Authenticating user token {{(pid=80378) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:26:41.356972 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be853ca3-5a62-4dbe-8d94-175f6761159f admin admin] Validating token access rules against request {{(pid=80378) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:26:41.422813 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45e5cb07-f4e6-4815-8596-67f1607de570 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '891005fa5ff74041b75747d16cc66798', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7047912c5acd4adeba5d87c3bf895e39', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80378) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:26:41.423444 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45e5cb07-f4e6-4815-8596-67f1607de570 admin admin] REQUEST_METHOD: `POST` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:26:41.423727 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45e5cb07-f4e6-4815-8596-67f1607de570 admin admin] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:26:41.424002 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45e5cb07-f4e6-4815-8596-67f1607de570 admin admin] PATH_INFO: `/v3/users` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:26:41.424675 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45e5cb07-f4e6-4815-8596-67f1607de570 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:26:41.426623 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45e5cb07-f4e6-4815-8596-67f1607de570 admin admin] RBAC: Authorization granted {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:26:41.435614 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG dbcounter [-] [80378] Writing DB stats keystone:SELECT=97,keystone:DELETE=2,keystone:INSERT=1 {{(pid=80378) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 10:26:41.461343 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 1315/2645] 10.222.0.82 () {60 vars in 1144 bytes} [Tue Aug 2 10:26:41 2022] POST /identity/v3/users => generated 300 bytes in 113 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 02 10:26:41.463159 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f730ec34-fdfd-41f8-b927-47ab4c132f6d admin admin] Authenticating user token {{(pid=80377) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:26:41.474191 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f730ec34-fdfd-41f8-b927-47ab4c132f6d admin admin] Validating token access rules against request {{(pid=80377) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:26:41.479681 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ff41184-5159-4baa-96d0-28c143b06771 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '891005fa5ff74041b75747d16cc66798', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7047912c5acd4adeba5d87c3bf895e39', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80377) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:26:41.480290 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ff41184-5159-4baa-96d0-28c143b06771 admin admin] REQUEST_METHOD: `GET` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:26:41.480577 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ff41184-5159-4baa-96d0-28c143b06771 admin admin] SCRIPT_NAME: `/identity` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:26:41.480985 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ff41184-5159-4baa-96d0-28c143b06771 admin admin] PATH_INFO: `/v3/roles` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:26:41.481697 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ff41184-5159-4baa-96d0-28c143b06771 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:26:41.483342 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ff41184-5159-4baa-96d0-28c143b06771 admin admin] RBAC: Authorization granted {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:26:41.494135 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: [pid: 80377|app: 0|req: 1331/2646] 10.222.0.82 () {58 vars in 1122 bytes} [Tue Aug 2 10:26:41 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 02 10:26:41.498215 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45e5cb07-f4e6-4815-8596-67f1607de570 admin admin] Authenticating user token {{(pid=80378) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:26:41.508271 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45e5cb07-f4e6-4815-8596-67f1607de570 admin admin] Validating token access rules against request {{(pid=80378) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:26:41.513033 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b1a2538-82d4-41c6-b76c-ea79a513125e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '891005fa5ff74041b75747d16cc66798', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7047912c5acd4adeba5d87c3bf895e39', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80378) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:26:41.513648 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b1a2538-82d4-41c6-b76c-ea79a513125e admin admin] REQUEST_METHOD: `GET` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:26:41.513973 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b1a2538-82d4-41c6-b76c-ea79a513125e admin admin] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:26:41.514258 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b1a2538-82d4-41c6-b76c-ea79a513125e admin admin] PATH_INFO: `/v3/roles` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:26:41.514982 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b1a2538-82d4-41c6-b76c-ea79a513125e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:26:41.516579 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b1a2538-82d4-41c6-b76c-ea79a513125e admin admin] RBAC: Authorization granted {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:26:41.527870 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 1316/2647] 10.222.0.82 () {58 vars in 1122 bytes} [Tue Aug 2 10:26:41 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 02 10:26:41.532474 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ff41184-5159-4baa-96d0-28c143b06771 admin admin] Authenticating user token {{(pid=80377) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:26:41.543222 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ff41184-5159-4baa-96d0-28c143b06771 admin admin] Validating token access rules against request {{(pid=80377) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:26:41.547925 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42a581b2-7c76-4dc9-8030-1254f018e2c2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '891005fa5ff74041b75747d16cc66798', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7047912c5acd4adeba5d87c3bf895e39', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80377) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:26:41.548593 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42a581b2-7c76-4dc9-8030-1254f018e2c2 admin admin] REQUEST_METHOD: `PUT` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:26:41.548876 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42a581b2-7c76-4dc9-8030-1254f018e2c2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:26:41.549179 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42a581b2-7c76-4dc9-8030-1254f018e2c2 admin admin] PATH_INFO: `/v3/projects/4a3a0cebeb5f44ca8373430c42cc7747/users/e3a7eea489b64161983b08aa489bd33c/roles/e1c04b19fc4543b8b54f58bbcffbc598` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:26:41.567586 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42a581b2-7c76-4dc9-8030-1254f018e2c2 admin admin] RBAC: Authorizing `identity:create_grant(project_id=4a3a0cebeb5f44ca8373430c42cc7747, user_id=e3a7eea489b64161983b08aa489bd33c, role_id=e1c04b19fc4543b8b54f58bbcffbc598)` {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:26:41.570300 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42a581b2-7c76-4dc9-8030-1254f018e2c2 admin admin] RBAC: Authorization granted {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:26:41.584849 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: [pid: 80377|app: 0|req: 1332/2648] 10.222.0.82 () {60 vars in 1483 bytes} [Tue Aug 2 10:26:41 2022] PUT /identity/v3/projects/4a3a0cebeb5f44ca8373430c42cc7747/users/e3a7eea489b64161983b08aa489bd33c/roles/e1c04b19fc4543b8b54f58bbcffbc598 => generated 0 bytes in 53 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 02 10:26:41.652742 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73249fc9-d019-4b71-a762-cf958b983f38 None None] REQUEST_METHOD: `POST` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:26:41.653026 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73249fc9-d019-4b71-a762-cf958b983f38 None None] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:26:41.653310 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73249fc9-d019-4b71-a762-cf958b983f38 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:26:41.686633 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.auth.core [None req-73249fc9-d019-4b71-a762-cf958b983f38 None None] MFA Rules not processed for user `e3a7eea489b64161983b08aa489bd33c`. Rule list: `[]` (Enabled: `True`). {{(pid=80378) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 02 10:26:41.688001 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.fernet_utils [None req-73249fc9-d019-4b71-a762-cf958b983f38 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80378) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 10:26:41.761608 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 1317/2649] 10.222.0.82 () {58 vars in 958 bytes} [Tue Aug 2 10:26:41 2022] POST /identity/v3/auth/tokens => generated 3276 bytes in 110 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 02 10:26:41.773472 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42a581b2-7c76-4dc9-8030-1254f018e2c2 admin admin] Authenticating user token {{(pid=80377) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:26:41.776029 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.fernet_utils [None req-42a581b2-7c76-4dc9-8030-1254f018e2c2 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80377) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 10:26:41.833760 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42a581b2-7c76-4dc9-8030-1254f018e2c2 admin admin] Validating token access rules against request {{(pid=80377) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:26:41.838093 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c1ee515-06db-46c1-b7c7-b691f86baa06 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1023de281ae74e9d8a6d6c312c1c97a7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f1b38ebb804c41c69772c998aae72c41', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80377) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:26:41.838720 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c1ee515-06db-46c1-b7c7-b691f86baa06 service cinder] REQUEST_METHOD: `GET` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:26:41.839003 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c1ee515-06db-46c1-b7c7-b691f86baa06 service cinder] SCRIPT_NAME: `/identity` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:26:41.839279 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c1ee515-06db-46c1-b7c7-b691f86baa06 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:26:41.845591 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG dbcounter [-] [80377] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=80377) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 10:26:41.853466 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c1ee515-06db-46c1-b7c7-b691f86baa06 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:26:41.855857 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c1ee515-06db-46c1-b7c7-b691f86baa06 service cinder] RBAC: Authorization granted {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:26:41.868871 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: [pid: 80377|app: 0|req: 1333/2650] 10.222.0.82 () {60 vars in 1365 bytes} [Tue Aug 2 10:26:41 2022] GET /identity/v3/auth/tokens => generated 3276 bytes in 96 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 02 10:26:45.527732 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73249fc9-d019-4b71-a762-cf958b983f38 None None] Authenticating user token {{(pid=80378) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:26:45.529720 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.fernet_utils [None req-73249fc9-d019-4b71-a762-cf958b983f38 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80378) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 10:26:45.603038 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73249fc9-d019-4b71-a762-cf958b983f38 None None] Validating token access rules against request {{(pid=80378) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:26:45.607793 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8872b01d-4002-49ce-8b84-ceddf61d4cab admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '891005fa5ff74041b75747d16cc66798', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7047912c5acd4adeba5d87c3bf895e39', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80378) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:26:45.608449 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8872b01d-4002-49ce-8b84-ceddf61d4cab admin admin] REQUEST_METHOD: `DELETE` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:26:45.608730 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8872b01d-4002-49ce-8b84-ceddf61d4cab admin admin] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:26:45.609012 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8872b01d-4002-49ce-8b84-ceddf61d4cab admin admin] PATH_INFO: `/v3/users/b6890a8127e04092920cb1d31ab56d91` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:26:45.624009 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8872b01d-4002-49ce-8b84-ceddf61d4cab admin admin] RBAC: Authorizing `identity:delete_user(user_id=b6890a8127e04092920cb1d31ab56d91)` {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:26:45.626039 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8872b01d-4002-49ce-8b84-ceddf61d4cab admin admin] RBAC: Authorization granted {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:26:45.674867 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.notifications [None req-8872b01d-4002-49ce-8b84-ceddf61d4cab admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'b6890a8127e04092920cb1d31ab56d91'} {{(pid=80378) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 10:26:45.683680 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.notifications [None req-8872b01d-4002-49ce-8b84-ceddf61d4cab admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'b6890a8127e04092920cb1d31ab56d91'} {{(pid=80378) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 10:26:45.690214 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG dbcounter [-] [80378] Writing DB stats keystone:INSERT=3,keystone:SELECT=89,keystone:DELETE=8 {{(pid=80378) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 10:26:45.692628 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.notifications [None req-8872b01d-4002-49ce-8b84-ceddf61d4cab admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'b6890a8127e04092920cb1d31ab56d91'} {{(pid=80378) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 10:26:45.693445 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.notifications [None req-8872b01d-4002-49ce-8b84-ceddf61d4cab admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'b6890a8127e04092920cb1d31ab56d91'} {{(pid=80378) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 10:26:45.708276 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.notifications [None req-8872b01d-4002-49ce-8b84-ceddf61d4cab admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'b6890a8127e04092920cb1d31ab56d91'} {{(pid=80378) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 10:26:45.715385 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 1318/2651] 10.222.0.82 () {58 vars in 1224 bytes} [Tue Aug 2 10:26:45 2022] DELETE /identity/v3/users/b6890a8127e04092920cb1d31ab56d91 => generated 0 bytes in 188 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 02 10:26:45.898412 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c1ee515-06db-46c1-b7c7-b691f86baa06 service cinder] Authenticating user token {{(pid=80377) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:26:45.911152 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c1ee515-06db-46c1-b7c7-b691f86baa06 service cinder] Validating token access rules against request {{(pid=80377) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:26:45.982942 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edcde53b-c3a8-48cb-8246-70fc2aebe323 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '891005fa5ff74041b75747d16cc66798', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7047912c5acd4adeba5d87c3bf895e39', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80377) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:26:45.983603 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edcde53b-c3a8-48cb-8246-70fc2aebe323 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:26:45.983885 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edcde53b-c3a8-48cb-8246-70fc2aebe323 admin admin] SCRIPT_NAME: `/identity` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:26:45.984162 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edcde53b-c3a8-48cb-8246-70fc2aebe323 admin admin] PATH_INFO: `/v3/projects/23714720faaa4562b694d7e3fbda2c33` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:26:45.995464 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-edcde53b-c3a8-48cb-8246-70fc2aebe323 admin admin] RBAC: Authorizing `identity:delete_project(project_id=23714720faaa4562b694d7e3fbda2c33)` {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:26:45.997310 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-edcde53b-c3a8-48cb-8246-70fc2aebe323 admin admin] RBAC: Authorization granted {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:26:46.047108 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.notifications [None req-edcde53b-c3a8-48cb-8246-70fc2aebe323 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '23714720faaa4562b694d7e3fbda2c33', 'request_id': 'req-edcde53b-c3a8-48cb-8246-70fc2aebe323'} {{(pid=80377) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 10:26:46.052916 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.notifications [None req-edcde53b-c3a8-48cb-8246-70fc2aebe323 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '23714720faaa4562b694d7e3fbda2c33', 'request_id': 'req-edcde53b-c3a8-48cb-8246-70fc2aebe323'} {{(pid=80377) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 10:26:46.059830 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.notifications [None req-edcde53b-c3a8-48cb-8246-70fc2aebe323 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '23714720faaa4562b694d7e3fbda2c33', 'request_id': 'req-edcde53b-c3a8-48cb-8246-70fc2aebe323'} {{(pid=80377) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 10:26:46.073807 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.notifications [None req-edcde53b-c3a8-48cb-8246-70fc2aebe323 admin admin] The token cache is being invalidate because project 23714720faaa4562b694d7e3fbda2c33 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80377) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 02 10:26:46.074120 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.notifications [None req-edcde53b-c3a8-48cb-8246-70fc2aebe323 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80377) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 10:26:46.074752 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.notifications [None req-edcde53b-c3a8-48cb-8246-70fc2aebe323 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80377) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 10:26:46.076428 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: [pid: 80377|app: 0|req: 1334/2652] 10.222.0.82 () {58 vars in 1233 bytes} [Tue Aug 2 10:26:45 2022] DELETE /identity/v3/projects/23714720faaa4562b694d7e3fbda2c33 => generated 0 bytes in 179 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 02 10:26:46.081428 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63d8f739-d5a7-4624-a828-baaa1ac03ac6 None None] REQUEST_METHOD: `POST` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:26:46.081834 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63d8f739-d5a7-4624-a828-baaa1ac03ac6 None None] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:26:46.082138 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63d8f739-d5a7-4624-a828-baaa1ac03ac6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:26:46.102321 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.auth.core [None req-63d8f739-d5a7-4624-a828-baaa1ac03ac6 None None] MFA Rules not processed for user `891005fa5ff74041b75747d16cc66798`. Rule list: `[]` (Enabled: `True`). {{(pid=80378) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 02 10:26:46.103674 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.fernet_utils [None req-63d8f739-d5a7-4624-a828-baaa1ac03ac6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80378) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 10:26:46.166294 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 1319/2653] 10.222.0.82 () {58 vars in 958 bytes} [Tue Aug 2 10:26:46 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 02 10:26:46.174383 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5edba2b-228a-4feb-8c14-4baffac42349 None None] REQUEST_METHOD: `POST` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:26:46.174673 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5edba2b-228a-4feb-8c14-4baffac42349 None None] SCRIPT_NAME: `/identity` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:26:46.174951 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5edba2b-228a-4feb-8c14-4baffac42349 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:26:46.194828 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.auth.core [None req-b5edba2b-228a-4feb-8c14-4baffac42349 None None] MFA Rules not processed for user `891005fa5ff74041b75747d16cc66798`. Rule list: `[]` (Enabled: `True`). {{(pid=80377) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 02 10:26:46.196047 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.fernet_utils [None req-b5edba2b-228a-4feb-8c14-4baffac42349 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80377) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 10:26:46.208964 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: [pid: 80377|app: 0|req: 1335/2654] 10.222.0.82 () {58 vars in 958 bytes} [Tue Aug 2 10:26:46 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 36 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 02 10:26:46.214006 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63d8f739-d5a7-4624-a828-baaa1ac03ac6 None None] Authenticating user token {{(pid=80378) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:26:46.224070 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63d8f739-d5a7-4624-a828-baaa1ac03ac6 None None] Validating token access rules against request {{(pid=80378) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:26:46.228851 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8fb944d-9d84-41d8-9044-d9118177c210 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '891005fa5ff74041b75747d16cc66798', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7047912c5acd4adeba5d87c3bf895e39', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80378) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:26:46.229460 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8fb944d-9d84-41d8-9044-d9118177c210 admin admin] REQUEST_METHOD: `GET` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:26:46.229743 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8fb944d-9d84-41d8-9044-d9118177c210 admin admin] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:26:46.230060 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8fb944d-9d84-41d8-9044-d9118177c210 admin admin] PATH_INFO: `/v3/domains` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:26:46.230754 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8fb944d-9d84-41d8-9044-d9118177c210 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80378) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 02 10:26:46.231063 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8fb944d-9d84-41d8-9044-d9118177c210 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:26:46.232664 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8fb944d-9d84-41d8-9044-d9118177c210 admin admin] RBAC: Authorization granted {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:26:46.243988 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 1320/2655] 10.222.0.82 () {58 vars in 1153 bytes} [Tue Aug 2 10:26:46 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 02 10:26:46.248574 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5edba2b-228a-4feb-8c14-4baffac42349 None None] Authenticating user token {{(pid=80377) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:26:46.258138 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5edba2b-228a-4feb-8c14-4baffac42349 None None] Validating token access rules against request {{(pid=80377) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:26:46.263381 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-670f3935-f7a0-4277-9726-0ca218ad5f62 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '891005fa5ff74041b75747d16cc66798', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7047912c5acd4adeba5d87c3bf895e39', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80377) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:26:46.263988 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-670f3935-f7a0-4277-9726-0ca218ad5f62 admin admin] REQUEST_METHOD: `POST` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:26:46.264274 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-670f3935-f7a0-4277-9726-0ca218ad5f62 admin admin] SCRIPT_NAME: `/identity` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:26:46.264550 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-670f3935-f7a0-4277-9726-0ca218ad5f62 admin admin] PATH_INFO: `/v3/projects` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:26:46.265219 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-670f3935-f7a0-4277-9726-0ca218ad5f62 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:26:46.267151 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-670f3935-f7a0-4277-9726-0ca218ad5f62 admin admin] RBAC: Authorization granted {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:26:46.298000 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: [pid: 80377|app: 0|req: 1336/2656] 10.222.0.82 () {60 vars in 1153 bytes} [Tue Aug 2 10:26:46 2022] POST /identity/v3/projects => generated 367 bytes in 50 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 02 10:26:46.302364 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8fb944d-9d84-41d8-9044-d9118177c210 admin admin] Authenticating user token {{(pid=80378) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:26:46.312250 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8fb944d-9d84-41d8-9044-d9118177c210 admin admin] Validating token access rules against request {{(pid=80378) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:26:46.367536 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0d93eb5-3b83-4a6d-bf30-d28292a2a99c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '891005fa5ff74041b75747d16cc66798', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7047912c5acd4adeba5d87c3bf895e39', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80378) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:26:46.368178 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0d93eb5-3b83-4a6d-bf30-d28292a2a99c admin admin] REQUEST_METHOD: `POST` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:26:46.368458 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0d93eb5-3b83-4a6d-bf30-d28292a2a99c admin admin] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:26:46.368727 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0d93eb5-3b83-4a6d-bf30-d28292a2a99c admin admin] PATH_INFO: `/v3/users` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:26:46.369388 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0d93eb5-3b83-4a6d-bf30-d28292a2a99c admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:26:46.371329 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0d93eb5-3b83-4a6d-bf30-d28292a2a99c admin admin] RBAC: Authorization granted {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:26:46.392739 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 1321/2657] 10.222.0.82 () {60 vars in 1144 bytes} [Tue Aug 2 10:26:46 2022] POST /identity/v3/users => generated 284 bytes in 91 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 02 10:26:46.396924 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-670f3935-f7a0-4277-9726-0ca218ad5f62 admin admin] Authenticating user token {{(pid=80377) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:26:46.408002 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-670f3935-f7a0-4277-9726-0ca218ad5f62 admin admin] Validating token access rules against request {{(pid=80377) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:26:46.413062 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a576b8f-53fa-4ada-8b79-ae1056637bc9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '891005fa5ff74041b75747d16cc66798', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7047912c5acd4adeba5d87c3bf895e39', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80377) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:26:46.413695 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a576b8f-53fa-4ada-8b79-ae1056637bc9 admin admin] REQUEST_METHOD: `GET` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:26:46.414079 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a576b8f-53fa-4ada-8b79-ae1056637bc9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:26:46.414272 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a576b8f-53fa-4ada-8b79-ae1056637bc9 admin admin] PATH_INFO: `/v3/roles` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:26:46.415002 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a576b8f-53fa-4ada-8b79-ae1056637bc9 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:26:46.416635 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a576b8f-53fa-4ada-8b79-ae1056637bc9 admin admin] RBAC: Authorization granted {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:26:46.426613 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: [pid: 80377|app: 0|req: 1337/2658] 10.222.0.82 () {58 vars in 1122 bytes} [Tue Aug 2 10:26:46 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 02 10:26:46.431024 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0d93eb5-3b83-4a6d-bf30-d28292a2a99c admin admin] Authenticating user token {{(pid=80378) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:26:46.440892 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0d93eb5-3b83-4a6d-bf30-d28292a2a99c admin admin] Validating token access rules against request {{(pid=80378) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:26:46.446876 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e516d4c-064c-4ce4-a3a4-f8f001bdd025 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '891005fa5ff74041b75747d16cc66798', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7047912c5acd4adeba5d87c3bf895e39', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80378) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:26:46.447488 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e516d4c-064c-4ce4-a3a4-f8f001bdd025 admin admin] REQUEST_METHOD: `GET` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:26:46.447777 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e516d4c-064c-4ce4-a3a4-f8f001bdd025 admin admin] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:26:46.448050 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e516d4c-064c-4ce4-a3a4-f8f001bdd025 admin admin] PATH_INFO: `/v3/roles` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:26:46.448741 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e516d4c-064c-4ce4-a3a4-f8f001bdd025 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:26:46.450374 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e516d4c-064c-4ce4-a3a4-f8f001bdd025 admin admin] RBAC: Authorization granted {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:26:46.461066 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 1322/2659] 10.222.0.82 () {58 vars in 1122 bytes} [Tue Aug 2 10:26:46 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 02 10:26:46.465050 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a576b8f-53fa-4ada-8b79-ae1056637bc9 admin admin] Authenticating user token {{(pid=80377) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:26:46.474813 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a576b8f-53fa-4ada-8b79-ae1056637bc9 admin admin] Validating token access rules against request {{(pid=80377) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:26:46.479494 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25724ad6-eba6-4e69-bccf-e69f28343c79 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '891005fa5ff74041b75747d16cc66798', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7047912c5acd4adeba5d87c3bf895e39', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80377) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:26:46.480146 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25724ad6-eba6-4e69-bccf-e69f28343c79 admin admin] REQUEST_METHOD: `PUT` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:26:46.480428 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25724ad6-eba6-4e69-bccf-e69f28343c79 admin admin] SCRIPT_NAME: `/identity` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:26:46.480707 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25724ad6-eba6-4e69-bccf-e69f28343c79 admin admin] PATH_INFO: `/v3/projects/824d9789b78841aea0d8ba2503d6b072/users/0a81d8125561413592f3eef9aa6f9abd/roles/e1c04b19fc4543b8b54f58bbcffbc598` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:26:46.495637 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25724ad6-eba6-4e69-bccf-e69f28343c79 admin admin] RBAC: Authorizing `identity:create_grant(project_id=824d9789b78841aea0d8ba2503d6b072, user_id=0a81d8125561413592f3eef9aa6f9abd, role_id=e1c04b19fc4543b8b54f58bbcffbc598)` {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:26:46.498336 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25724ad6-eba6-4e69-bccf-e69f28343c79 admin admin] RBAC: Authorization granted {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:26:46.512570 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: [pid: 80377|app: 0|req: 1338/2660] 10.222.0.82 () {60 vars in 1483 bytes} [Tue Aug 2 10:26:46 2022] PUT /identity/v3/projects/824d9789b78841aea0d8ba2503d6b072/users/0a81d8125561413592f3eef9aa6f9abd/roles/e1c04b19fc4543b8b54f58bbcffbc598 => generated 0 bytes in 48 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 02 10:26:46.522889 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97c7e7c0-56f8-4944-a162-b89ee57baf06 None None] REQUEST_METHOD: `POST` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:26:46.523178 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97c7e7c0-56f8-4944-a162-b89ee57baf06 None None] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:26:46.523483 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97c7e7c0-56f8-4944-a162-b89ee57baf06 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:26:46.550487 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.auth.core [None req-97c7e7c0-56f8-4944-a162-b89ee57baf06 None None] MFA Rules not processed for user `0a81d8125561413592f3eef9aa6f9abd`. Rule list: `[]` (Enabled: `True`). {{(pid=80378) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 02 10:26:46.551692 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.fernet_utils [None req-97c7e7c0-56f8-4944-a162-b89ee57baf06 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80378) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 10:26:46.587771 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG dbcounter [-] [80378] Writing DB stats keystone:SELECT=95,keystone:DELETE=2,keystone:INSERT=3 {{(pid=80378) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 10:26:46.624273 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 1323/2661] 10.222.0.82 () {58 vars in 958 bytes} [Tue Aug 2 10:26:46 2022] POST /identity/v3/auth/tokens => generated 3244 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 02 10:26:46.628721 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25724ad6-eba6-4e69-bccf-e69f28343c79 admin admin] Authenticating user token {{(pid=80377) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:26:46.642651 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG dbcounter [-] [80377] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=80377) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 10:26:46.695097 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25724ad6-eba6-4e69-bccf-e69f28343c79 admin admin] Validating token access rules against request {{(pid=80377) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:26:46.699713 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba14128d-5a17-4ae1-97ca-f195d344762f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '891005fa5ff74041b75747d16cc66798', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7047912c5acd4adeba5d87c3bf895e39', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80377) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:26:46.700328 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba14128d-5a17-4ae1-97ca-f195d344762f admin admin] REQUEST_METHOD: `POST` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:26:46.700609 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba14128d-5a17-4ae1-97ca-f195d344762f admin admin] SCRIPT_NAME: `/identity` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:26:46.700882 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba14128d-5a17-4ae1-97ca-f195d344762f admin admin] PATH_INFO: `/v3/projects` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:26:46.701553 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba14128d-5a17-4ae1-97ca-f195d344762f admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:26:46.703484 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba14128d-5a17-4ae1-97ca-f195d344762f admin admin] RBAC: Authorization granted {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:26:46.730592 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: [pid: 80377|app: 0|req: 1339/2662] 10.222.0.82 () {60 vars in 1153 bytes} [Tue Aug 2 10:26:46 2022] POST /identity/v3/projects => generated 367 bytes in 102 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 02 10:26:46.734642 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97c7e7c0-56f8-4944-a162-b89ee57baf06 None None] Authenticating user token {{(pid=80378) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:26:46.745437 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97c7e7c0-56f8-4944-a162-b89ee57baf06 None None] Validating token access rules against request {{(pid=80378) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:26:46.925233 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1e2871b-8398-4501-9066-568f6d430313 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '891005fa5ff74041b75747d16cc66798', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7047912c5acd4adeba5d87c3bf895e39', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80378) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:26:46.925866 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1e2871b-8398-4501-9066-568f6d430313 admin admin] REQUEST_METHOD: `POST` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:26:46.926146 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1e2871b-8398-4501-9066-568f6d430313 admin admin] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:26:46.926414 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1e2871b-8398-4501-9066-568f6d430313 admin admin] PATH_INFO: `/v3/users` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:26:46.927075 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1e2871b-8398-4501-9066-568f6d430313 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:26:46.929077 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1e2871b-8398-4501-9066-568f6d430313 admin admin] RBAC: Authorization granted {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:26:46.950240 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 1324/2663] 10.222.0.82 () {60 vars in 1144 bytes} [Tue Aug 2 10:26:46 2022] POST /identity/v3/users => generated 290 bytes in 216 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 02 10:26:46.954180 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba14128d-5a17-4ae1-97ca-f195d344762f admin admin] Authenticating user token {{(pid=80377) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:26:46.964505 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba14128d-5a17-4ae1-97ca-f195d344762f admin admin] Validating token access rules against request {{(pid=80377) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:26:46.969173 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9293c70e-de04-40dd-9553-dd253a49da67 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '891005fa5ff74041b75747d16cc66798', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7047912c5acd4adeba5d87c3bf895e39', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80377) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:26:46.969804 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9293c70e-de04-40dd-9553-dd253a49da67 admin admin] REQUEST_METHOD: `GET` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:26:46.970094 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9293c70e-de04-40dd-9553-dd253a49da67 admin admin] SCRIPT_NAME: `/identity` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:26:46.970368 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9293c70e-de04-40dd-9553-dd253a49da67 admin admin] PATH_INFO: `/v3/roles` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:26:46.971066 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9293c70e-de04-40dd-9553-dd253a49da67 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:26:46.972662 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9293c70e-de04-40dd-9553-dd253a49da67 admin admin] RBAC: Authorization granted {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:26:46.984828 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: [pid: 80377|app: 0|req: 1340/2664] 10.222.0.82 () {58 vars in 1122 bytes} [Tue Aug 2 10:26:46 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 02 10:26:46.988905 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1e2871b-8398-4501-9066-568f6d430313 admin admin] Authenticating user token {{(pid=80378) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:26:46.999388 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1e2871b-8398-4501-9066-568f6d430313 admin admin] Validating token access rules against request {{(pid=80378) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:26:47.004074 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59fbc86f-fc0f-48cc-bcd5-4722f0f41404 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '891005fa5ff74041b75747d16cc66798', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7047912c5acd4adeba5d87c3bf895e39', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80378) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:26:47.004717 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59fbc86f-fc0f-48cc-bcd5-4722f0f41404 admin admin] REQUEST_METHOD: `PUT` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:26:47.004998 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59fbc86f-fc0f-48cc-bcd5-4722f0f41404 admin admin] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:26:47.005266 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59fbc86f-fc0f-48cc-bcd5-4722f0f41404 admin admin] PATH_INFO: `/v3/projects/a880731c4a0240fd9b33b9663581ee34/users/c7c91d8905f44c4d960efd0d329ecc2b/roles/237966442f05411191105bf336bc14f3` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:26:47.020197 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59fbc86f-fc0f-48cc-bcd5-4722f0f41404 admin admin] RBAC: Authorizing `identity:create_grant(project_id=a880731c4a0240fd9b33b9663581ee34, user_id=c7c91d8905f44c4d960efd0d329ecc2b, role_id=237966442f05411191105bf336bc14f3)` {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:26:47.022984 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59fbc86f-fc0f-48cc-bcd5-4722f0f41404 admin admin] RBAC: Authorization granted {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:26:47.036024 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 1325/2665] 10.222.0.82 () {60 vars in 1483 bytes} [Tue Aug 2 10:26:46 2022] PUT /identity/v3/projects/a880731c4a0240fd9b33b9663581ee34/users/c7c91d8905f44c4d960efd0d329ecc2b/roles/237966442f05411191105bf336bc14f3 => generated 0 bytes in 48 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 02 10:26:47.040684 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9293c70e-de04-40dd-9553-dd253a49da67 admin admin] Authenticating user token {{(pid=80377) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:26:47.050557 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9293c70e-de04-40dd-9553-dd253a49da67 admin admin] Validating token access rules against request {{(pid=80377) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:26:47.105235 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f0c0ba5-a643-4caa-8ce4-67b1a501c50f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '891005fa5ff74041b75747d16cc66798', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7047912c5acd4adeba5d87c3bf895e39', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80377) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:26:47.105802 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f0c0ba5-a643-4caa-8ce4-67b1a501c50f admin admin] REQUEST_METHOD: `GET` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:26:47.106098 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f0c0ba5-a643-4caa-8ce4-67b1a501c50f admin admin] SCRIPT_NAME: `/identity` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:26:47.106375 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f0c0ba5-a643-4caa-8ce4-67b1a501c50f admin admin] PATH_INFO: `/v3/roles` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:26:47.107063 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f0c0ba5-a643-4caa-8ce4-67b1a501c50f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:26:47.108643 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f0c0ba5-a643-4caa-8ce4-67b1a501c50f admin admin] RBAC: Authorization granted {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:26:47.119056 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: [pid: 80377|app: 0|req: 1341/2666] 10.222.0.82 () {58 vars in 1122 bytes} [Tue Aug 2 10:26:47 2022] GET /identity/v3/roles => generated 1420 bytes in 79 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 02 10:26:47.123176 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59fbc86f-fc0f-48cc-bcd5-4722f0f41404 admin admin] Authenticating user token {{(pid=80378) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:26:47.133125 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59fbc86f-fc0f-48cc-bcd5-4722f0f41404 admin admin] Validating token access rules against request {{(pid=80378) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:26:47.137649 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fffe3767-b23e-4e34-892e-63200f894947 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '891005fa5ff74041b75747d16cc66798', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7047912c5acd4adeba5d87c3bf895e39', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80378) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:26:47.138330 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fffe3767-b23e-4e34-892e-63200f894947 admin admin] REQUEST_METHOD: `PUT` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:26:47.138609 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fffe3767-b23e-4e34-892e-63200f894947 admin admin] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:26:47.138879 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fffe3767-b23e-4e34-892e-63200f894947 admin admin] PATH_INFO: `/v3/projects/a880731c4a0240fd9b33b9663581ee34/users/c7c91d8905f44c4d960efd0d329ecc2b/roles/237966442f05411191105bf336bc14f3` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:26:47.140811 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fffe3767-b23e-4e34-892e-63200f894947 admin admin] RBAC: Authorizing `identity:create_grant(project_id=a880731c4a0240fd9b33b9663581ee34, user_id=c7c91d8905f44c4d960efd0d329ecc2b, role_id=237966442f05411191105bf336bc14f3)` {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:26:47.143460 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fffe3767-b23e-4e34-892e-63200f894947 admin admin] RBAC: Authorization granted {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:26:47.153560 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 1326/2667] 10.222.0.82 () {60 vars in 1483 bytes} [Tue Aug 2 10:26:47 2022] PUT /identity/v3/projects/a880731c4a0240fd9b33b9663581ee34/users/c7c91d8905f44c4d960efd0d329ecc2b/roles/237966442f05411191105bf336bc14f3 => generated 0 bytes in 31 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 02 10:26:47.164687 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1bf3329-222b-43ac-ad26-41df94fc93c3 None None] REQUEST_METHOD: `POST` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:26:47.165486 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1bf3329-222b-43ac-ad26-41df94fc93c3 None None] SCRIPT_NAME: `/identity` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:26:47.165767 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1bf3329-222b-43ac-ad26-41df94fc93c3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:26:47.193632 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.auth.core [None req-e1bf3329-222b-43ac-ad26-41df94fc93c3 None None] MFA Rules not processed for user `c7c91d8905f44c4d960efd0d329ecc2b`. Rule list: `[]` (Enabled: `True`). {{(pid=80377) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 02 10:26:47.194847 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.fernet_utils [None req-e1bf3329-222b-43ac-ad26-41df94fc93c3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80377) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 10:26:47.261549 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: [pid: 80377|app: 0|req: 1342/2668] 10.222.0.82 () {58 vars in 958 bytes} [Tue Aug 2 10:26:47 2022] POST /identity/v3/auth/tokens => generated 3311 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 02 10:26:47.272205 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fffe3767-b23e-4e34-892e-63200f894947 admin admin] Authenticating user token {{(pid=80378) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:26:47.273955 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.fernet_utils [None req-fffe3767-b23e-4e34-892e-63200f894947 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80378) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 10:26:47.335266 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fffe3767-b23e-4e34-892e-63200f894947 admin admin] Validating token access rules against request {{(pid=80378) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:26:47.339657 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1e5d2e0-4adb-420f-b262-e6e220708f9d service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1023de281ae74e9d8a6d6c312c1c97a7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f1b38ebb804c41c69772c998aae72c41', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80378) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:26:47.340498 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1e5d2e0-4adb-420f-b262-e6e220708f9d service cinder] REQUEST_METHOD: `GET` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:26:47.340783 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1e5d2e0-4adb-420f-b262-e6e220708f9d service cinder] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:26:47.341060 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1e5d2e0-4adb-420f-b262-e6e220708f9d service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:26:47.350376 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1e5d2e0-4adb-420f-b262-e6e220708f9d service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:26:47.351808 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1e5d2e0-4adb-420f-b262-e6e220708f9d service cinder] RBAC: Authorization granted {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:26:47.363256 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 1327/2669] 10.222.0.82 () {60 vars in 1365 bytes} [Tue Aug 2 10:26:47 2022] GET /identity/v3/auth/tokens => generated 3311 bytes in 91 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 02 10:26:48.745155 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1bf3329-222b-43ac-ad26-41df94fc93c3 None None] Authenticating user token {{(pid=80377) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:26:48.757105 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG dbcounter [-] [80377] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80377) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 10:26:48.811320 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1bf3329-222b-43ac-ad26-41df94fc93c3 None None] Validating token access rules against request {{(pid=80377) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:26:48.815837 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79123cbf-d8d5-45d0-a8cb-3e12259f4ccc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '891005fa5ff74041b75747d16cc66798', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7047912c5acd4adeba5d87c3bf895e39', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80377) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:26:48.816486 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79123cbf-d8d5-45d0-a8cb-3e12259f4ccc admin admin] REQUEST_METHOD: `DELETE` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:26:48.817026 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79123cbf-d8d5-45d0-a8cb-3e12259f4ccc admin admin] SCRIPT_NAME: `/identity` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:26:48.817306 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79123cbf-d8d5-45d0-a8cb-3e12259f4ccc admin admin] PATH_INFO: `/v3/users/0a81d8125561413592f3eef9aa6f9abd` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:26:48.818488 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79123cbf-d8d5-45d0-a8cb-3e12259f4ccc admin admin] RBAC: Authorizing `identity:delete_user(user_id=0a81d8125561413592f3eef9aa6f9abd)` {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:26:48.820636 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79123cbf-d8d5-45d0-a8cb-3e12259f4ccc admin admin] RBAC: Authorization granted {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:26:48.875961 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.notifications [None req-79123cbf-d8d5-45d0-a8cb-3e12259f4ccc admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '0a81d8125561413592f3eef9aa6f9abd'} {{(pid=80377) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 10:26:48.884460 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.notifications [None req-79123cbf-d8d5-45d0-a8cb-3e12259f4ccc admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '0a81d8125561413592f3eef9aa6f9abd'} {{(pid=80377) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 10:26:48.892024 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.notifications [None req-79123cbf-d8d5-45d0-a8cb-3e12259f4ccc admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '0a81d8125561413592f3eef9aa6f9abd'} {{(pid=80377) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 10:26:48.892970 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.notifications [None req-79123cbf-d8d5-45d0-a8cb-3e12259f4ccc admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '0a81d8125561413592f3eef9aa6f9abd'} {{(pid=80377) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 10:26:48.905585 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.notifications [None req-79123cbf-d8d5-45d0-a8cb-3e12259f4ccc admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '0a81d8125561413592f3eef9aa6f9abd'} {{(pid=80377) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 10:26:48.911600 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: [pid: 80377|app: 0|req: 1343/2670] 10.222.0.82 () {58 vars in 1224 bytes} [Tue Aug 2 10:26:48 2022] DELETE /identity/v3/users/0a81d8125561413592f3eef9aa6f9abd => generated 0 bytes in 167 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 02 10:26:48.916556 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-500a3e91-a4d9-4954-9042-ed1b65030066 None None] REQUEST_METHOD: `POST` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:26:48.917006 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-500a3e91-a4d9-4954-9042-ed1b65030066 None None] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:26:48.917116 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-500a3e91-a4d9-4954-9042-ed1b65030066 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:26:48.935189 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.auth.core [None req-500a3e91-a4d9-4954-9042-ed1b65030066 None None] MFA Rules not processed for user `891005fa5ff74041b75747d16cc66798`. Rule list: `[]` (Enabled: `True`). {{(pid=80378) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 02 10:26:48.936375 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.fernet_utils [None req-500a3e91-a4d9-4954-9042-ed1b65030066 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80378) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 10:26:48.959238 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG dbcounter [-] [80378] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80378) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 10:26:48.994849 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 1328/2671] 10.222.0.82 () {58 vars in 958 bytes} [Tue Aug 2 10:26:48 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 80 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 02 10:26:49.004419 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79123cbf-d8d5-45d0-a8cb-3e12259f4ccc admin admin] Authenticating user token {{(pid=80377) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:26:49.005880 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.fernet_utils [None req-79123cbf-d8d5-45d0-a8cb-3e12259f4ccc admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80377) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 10:26:49.064638 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79123cbf-d8d5-45d0-a8cb-3e12259f4ccc admin admin] Validating token access rules against request {{(pid=80377) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:26:49.068532 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e25479c5-3a47-4b44-954f-f8109f4f7fc7 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '33b87170f0634b67bce60750847b5cbe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f1b38ebb804c41c69772c998aae72c41', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80377) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:26:49.069154 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e25479c5-3a47-4b44-954f-f8109f4f7fc7 service neutron] REQUEST_METHOD: `GET` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:26:49.069438 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e25479c5-3a47-4b44-954f-f8109f4f7fc7 service neutron] SCRIPT_NAME: `/identity` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:26:49.069711 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e25479c5-3a47-4b44-954f-f8109f4f7fc7 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:26:49.079030 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e25479c5-3a47-4b44-954f-f8109f4f7fc7 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:26:49.080517 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e25479c5-3a47-4b44-954f-f8109f4f7fc7 service neutron] RBAC: Authorization granted {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:26:49.090392 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: [pid: 80377|app: 0|req: 1344/2672] 10.222.0.82 () {60 vars in 1365 bytes} [Tue Aug 2 10:26:49 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 86 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 02 10:26:49.357751 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-500a3e91-a4d9-4954-9042-ed1b65030066 None None] Authenticating user token {{(pid=80378) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:26:49.359385 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.fernet_utils [None req-500a3e91-a4d9-4954-9042-ed1b65030066 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80378) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 10:26:49.372024 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-500a3e91-a4d9-4954-9042-ed1b65030066 None None] Validating token access rules against request {{(pid=80378) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:26:49.376699 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e878214-581c-431b-a4e5-2e6c4d3a1559 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '891005fa5ff74041b75747d16cc66798', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7047912c5acd4adeba5d87c3bf895e39', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80378) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:26:49.378640 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e878214-581c-431b-a4e5-2e6c4d3a1559 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:26:49.378928 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e878214-581c-431b-a4e5-2e6c4d3a1559 admin admin] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:26:49.379202 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e878214-581c-431b-a4e5-2e6c4d3a1559 admin admin] PATH_INFO: `/v3/projects/824d9789b78841aea0d8ba2503d6b072` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:26:49.380481 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e878214-581c-431b-a4e5-2e6c4d3a1559 admin admin] RBAC: Authorizing `identity:delete_project(project_id=824d9789b78841aea0d8ba2503d6b072)` {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:26:49.383361 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e878214-581c-431b-a4e5-2e6c4d3a1559 admin admin] RBAC: Authorization granted {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:26:49.433440 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.notifications [None req-9e878214-581c-431b-a4e5-2e6c4d3a1559 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '824d9789b78841aea0d8ba2503d6b072', 'request_id': 'req-9e878214-581c-431b-a4e5-2e6c4d3a1559'} {{(pid=80378) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 10:26:49.439459 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.notifications [None req-9e878214-581c-431b-a4e5-2e6c4d3a1559 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '824d9789b78841aea0d8ba2503d6b072', 'request_id': 'req-9e878214-581c-431b-a4e5-2e6c4d3a1559'} {{(pid=80378) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 10:26:49.445684 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.notifications [None req-9e878214-581c-431b-a4e5-2e6c4d3a1559 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '824d9789b78841aea0d8ba2503d6b072', 'request_id': 'req-9e878214-581c-431b-a4e5-2e6c4d3a1559'} {{(pid=80378) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 10:26:49.457911 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.notifications [None req-9e878214-581c-431b-a4e5-2e6c4d3a1559 admin admin] The token cache is being invalidate because project 824d9789b78841aea0d8ba2503d6b072 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80378) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 02 10:26:49.458204 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.notifications [None req-9e878214-581c-431b-a4e5-2e6c4d3a1559 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80378) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 10:26:49.458827 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.notifications [None req-9e878214-581c-431b-a4e5-2e6c4d3a1559 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80378) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 10:26:49.460350 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 1329/2673] 10.222.0.82 () {58 vars in 1233 bytes} [Tue Aug 2 10:26:49 2022] DELETE /identity/v3/projects/824d9789b78841aea0d8ba2503d6b072 => generated 0 bytes in 103 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 02 10:26:49.465033 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e25479c5-3a47-4b44-954f-f8109f4f7fc7 service neutron] Authenticating user token {{(pid=80377) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:26:49.474643 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e25479c5-3a47-4b44-954f-f8109f4f7fc7 service neutron] Validating token access rules against request {{(pid=80377) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:26:49.503076 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG dbcounter [-] [80377] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80377) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 10:26:49.535105 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e061f0a5-1d60-443e-afcd-e70fae85bddf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '891005fa5ff74041b75747d16cc66798', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7047912c5acd4adeba5d87c3bf895e39', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80377) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:26:49.535750 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e061f0a5-1d60-443e-afcd-e70fae85bddf admin admin] REQUEST_METHOD: `DELETE` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:26:49.536030 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e061f0a5-1d60-443e-afcd-e70fae85bddf admin admin] SCRIPT_NAME: `/identity` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:26:49.536304 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e061f0a5-1d60-443e-afcd-e70fae85bddf admin admin] PATH_INFO: `/v3/users/c7c91d8905f44c4d960efd0d329ecc2b` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:26:49.537465 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e061f0a5-1d60-443e-afcd-e70fae85bddf admin admin] RBAC: Authorizing `identity:delete_user(user_id=c7c91d8905f44c4d960efd0d329ecc2b)` {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:26:49.539419 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e061f0a5-1d60-443e-afcd-e70fae85bddf admin admin] RBAC: Authorization granted {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:26:49.584318 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.notifications [None req-e061f0a5-1d60-443e-afcd-e70fae85bddf admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'c7c91d8905f44c4d960efd0d329ecc2b'} {{(pid=80377) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 10:26:49.592353 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.notifications [None req-e061f0a5-1d60-443e-afcd-e70fae85bddf admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'c7c91d8905f44c4d960efd0d329ecc2b'} {{(pid=80377) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 10:26:49.599588 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.notifications [None req-e061f0a5-1d60-443e-afcd-e70fae85bddf admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'c7c91d8905f44c4d960efd0d329ecc2b'} {{(pid=80377) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 10:26:49.600224 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.notifications [None req-e061f0a5-1d60-443e-afcd-e70fae85bddf admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'c7c91d8905f44c4d960efd0d329ecc2b'} {{(pid=80377) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 10:26:49.612659 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.notifications [None req-e061f0a5-1d60-443e-afcd-e70fae85bddf admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'c7c91d8905f44c4d960efd0d329ecc2b'} {{(pid=80377) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 10:26:49.618510 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: [pid: 80377|app: 0|req: 1345/2674] 10.222.0.82 () {58 vars in 1224 bytes} [Tue Aug 2 10:26:49 2022] DELETE /identity/v3/users/c7c91d8905f44c4d960efd0d329ecc2b => generated 0 bytes in 154 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 02 10:26:49.883479 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e878214-581c-431b-a4e5-2e6c4d3a1559 admin admin] Authenticating user token {{(pid=80378) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:26:49.885479 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.fernet_utils [None req-9e878214-581c-431b-a4e5-2e6c4d3a1559 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80378) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 10:26:49.909356 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e878214-581c-431b-a4e5-2e6c4d3a1559 admin admin] Validating token access rules against request {{(pid=80378) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:26:49.969442 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aaccddca-61c1-4e46-9e83-20546be9250c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '891005fa5ff74041b75747d16cc66798', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7047912c5acd4adeba5d87c3bf895e39', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80378) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:26:49.970129 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aaccddca-61c1-4e46-9e83-20546be9250c admin admin] REQUEST_METHOD: `DELETE` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:26:49.970618 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aaccddca-61c1-4e46-9e83-20546be9250c admin admin] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:26:49.970892 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aaccddca-61c1-4e46-9e83-20546be9250c admin admin] PATH_INFO: `/v3/projects/a880731c4a0240fd9b33b9663581ee34` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:26:49.972063 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aaccddca-61c1-4e46-9e83-20546be9250c admin admin] RBAC: Authorizing `identity:delete_project(project_id=a880731c4a0240fd9b33b9663581ee34)` {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:26:49.974005 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aaccddca-61c1-4e46-9e83-20546be9250c admin admin] RBAC: Authorization granted {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:26:50.037433 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.notifications [None req-aaccddca-61c1-4e46-9e83-20546be9250c admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'a880731c4a0240fd9b33b9663581ee34', 'request_id': 'req-aaccddca-61c1-4e46-9e83-20546be9250c'} {{(pid=80378) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 10:26:50.043586 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.notifications [None req-aaccddca-61c1-4e46-9e83-20546be9250c admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'a880731c4a0240fd9b33b9663581ee34', 'request_id': 'req-aaccddca-61c1-4e46-9e83-20546be9250c'} {{(pid=80378) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 10:26:50.044802 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG dbcounter [-] [80378] Writing DB stats keystone:SELECT=88,keystone:DELETE=11,keystone:INSERT=1 {{(pid=80378) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 10:26:50.050248 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.notifications [None req-aaccddca-61c1-4e46-9e83-20546be9250c admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'a880731c4a0240fd9b33b9663581ee34', 'request_id': 'req-aaccddca-61c1-4e46-9e83-20546be9250c'} {{(pid=80378) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 10:26:50.063186 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.notifications [None req-aaccddca-61c1-4e46-9e83-20546be9250c admin admin] The token cache is being invalidate because project a880731c4a0240fd9b33b9663581ee34 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80378) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 02 10:26:50.063508 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.notifications [None req-aaccddca-61c1-4e46-9e83-20546be9250c admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80378) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 10:26:50.064138 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.notifications [None req-aaccddca-61c1-4e46-9e83-20546be9250c admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80378) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 10:26:50.065625 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 1330/2675] 10.222.0.82 () {58 vars in 1233 bytes} [Tue Aug 2 10:26:49 2022] DELETE /identity/v3/projects/a880731c4a0240fd9b33b9663581ee34 => generated 0 bytes in 183 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 02 10:26:50.070855 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fc63387-96bb-457e-9372-5a837b8a6776 None None] REQUEST_METHOD: `POST` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:26:50.071835 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fc63387-96bb-457e-9372-5a837b8a6776 None None] SCRIPT_NAME: `/identity` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:26:50.071936 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fc63387-96bb-457e-9372-5a837b8a6776 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:26:50.091778 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.auth.core [None req-4fc63387-96bb-457e-9372-5a837b8a6776 None None] MFA Rules not processed for user `891005fa5ff74041b75747d16cc66798`. Rule list: `[]` (Enabled: `True`). {{(pid=80377) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 02 10:26:50.093363 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.fernet_utils [None req-4fc63387-96bb-457e-9372-5a837b8a6776 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80377) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 10:26:50.151759 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: [pid: 80377|app: 0|req: 1346/2676] 10.222.0.82 () {58 vars in 958 bytes} [Tue Aug 2 10:26:50 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 82 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 02 10:26:50.159539 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ccc5eba-1e77-4965-ad0d-8708263b9ec4 None None] REQUEST_METHOD: `POST` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:26:50.159830 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ccc5eba-1e77-4965-ad0d-8708263b9ec4 None None] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:26:50.160107 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ccc5eba-1e77-4965-ad0d-8708263b9ec4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:26:50.179332 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.auth.core [None req-9ccc5eba-1e77-4965-ad0d-8708263b9ec4 None None] MFA Rules not processed for user `891005fa5ff74041b75747d16cc66798`. Rule list: `[]` (Enabled: `True`). {{(pid=80378) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 02 10:26:50.180516 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.fernet_utils [None req-9ccc5eba-1e77-4965-ad0d-8708263b9ec4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80378) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 10:26:50.188947 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 1331/2677] 10.222.0.82 () {58 vars in 958 bytes} [Tue Aug 2 10:26:50 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 02 10:26:50.193561 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fc63387-96bb-457e-9372-5a837b8a6776 None None] Authenticating user token {{(pid=80377) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:26:50.203775 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fc63387-96bb-457e-9372-5a837b8a6776 None None] Validating token access rules against request {{(pid=80377) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:26:50.208337 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32ea8008-daaa-400a-9b0d-912b822e540f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '891005fa5ff74041b75747d16cc66798', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7047912c5acd4adeba5d87c3bf895e39', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80377) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:26:50.208946 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32ea8008-daaa-400a-9b0d-912b822e540f admin admin] REQUEST_METHOD: `GET` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:26:50.209224 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32ea8008-daaa-400a-9b0d-912b822e540f admin admin] SCRIPT_NAME: `/identity` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:26:50.209492 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32ea8008-daaa-400a-9b0d-912b822e540f admin admin] PATH_INFO: `/v3/domains` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:26:50.210210 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32ea8008-daaa-400a-9b0d-912b822e540f admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80377) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 02 10:26:50.210511 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32ea8008-daaa-400a-9b0d-912b822e540f admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:26:50.212195 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32ea8008-daaa-400a-9b0d-912b822e540f admin admin] RBAC: Authorization granted {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:26:50.223830 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: [pid: 80377|app: 0|req: 1347/2678] 10.222.0.82 () {58 vars in 1153 bytes} [Tue Aug 2 10:26:50 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 02 10:26:50.228115 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ccc5eba-1e77-4965-ad0d-8708263b9ec4 None None] Authenticating user token {{(pid=80378) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:26:50.240022 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ccc5eba-1e77-4965-ad0d-8708263b9ec4 None None] Validating token access rules against request {{(pid=80378) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:26:50.245186 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-169047de-ac0b-4f47-8242-5c6cfd3d00d5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '891005fa5ff74041b75747d16cc66798', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7047912c5acd4adeba5d87c3bf895e39', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80378) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:26:50.245833 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-169047de-ac0b-4f47-8242-5c6cfd3d00d5 admin admin] REQUEST_METHOD: `POST` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:26:50.246330 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-169047de-ac0b-4f47-8242-5c6cfd3d00d5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:26:50.246744 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-169047de-ac0b-4f47-8242-5c6cfd3d00d5 admin admin] PATH_INFO: `/v3/projects` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:26:50.247412 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-169047de-ac0b-4f47-8242-5c6cfd3d00d5 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:26:50.249417 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-169047de-ac0b-4f47-8242-5c6cfd3d00d5 admin admin] RBAC: Authorization granted {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:26:50.279072 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 1332/2679] 10.222.0.82 () {60 vars in 1153 bytes} [Tue Aug 2 10:26:50 2022] POST /identity/v3/projects => generated 361 bytes in 51 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 02 10:26:50.283501 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32ea8008-daaa-400a-9b0d-912b822e540f admin admin] Authenticating user token {{(pid=80377) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:26:50.293649 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32ea8008-daaa-400a-9b0d-912b822e540f admin admin] Validating token access rules against request {{(pid=80377) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:26:50.327934 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG dbcounter [-] [80377] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80377) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 10:26:50.358214 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80eed83d-bab1-4974-b4d8-bbab4729f7df admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '891005fa5ff74041b75747d16cc66798', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7047912c5acd4adeba5d87c3bf895e39', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80377) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:26:50.358933 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80eed83d-bab1-4974-b4d8-bbab4729f7df admin admin] REQUEST_METHOD: `POST` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:26:50.359216 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80eed83d-bab1-4974-b4d8-bbab4729f7df admin admin] SCRIPT_NAME: `/identity` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:26:50.359495 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80eed83d-bab1-4974-b4d8-bbab4729f7df admin admin] PATH_INFO: `/v3/users` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:26:50.360183 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80eed83d-bab1-4974-b4d8-bbab4729f7df admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:26:50.362143 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80eed83d-bab1-4974-b4d8-bbab4729f7df admin admin] RBAC: Authorization granted {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:26:50.383293 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: [pid: 80377|app: 0|req: 1348/2680] 10.222.0.82 () {60 vars in 1144 bytes} [Tue Aug 2 10:26:50 2022] POST /identity/v3/users => generated 281 bytes in 100 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 02 10:26:50.388016 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-169047de-ac0b-4f47-8242-5c6cfd3d00d5 admin admin] Authenticating user token {{(pid=80378) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:26:50.398999 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-169047de-ac0b-4f47-8242-5c6cfd3d00d5 admin admin] Validating token access rules against request {{(pid=80378) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:26:50.403837 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0888e5cd-4cc2-40bf-a981-844ae5160a8e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '891005fa5ff74041b75747d16cc66798', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7047912c5acd4adeba5d87c3bf895e39', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80378) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:26:50.404462 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0888e5cd-4cc2-40bf-a981-844ae5160a8e admin admin] REQUEST_METHOD: `GET` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:26:50.404774 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0888e5cd-4cc2-40bf-a981-844ae5160a8e admin admin] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:26:50.405046 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0888e5cd-4cc2-40bf-a981-844ae5160a8e admin admin] PATH_INFO: `/v3/roles` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:26:50.405745 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0888e5cd-4cc2-40bf-a981-844ae5160a8e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:26:50.407360 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0888e5cd-4cc2-40bf-a981-844ae5160a8e admin admin] RBAC: Authorization granted {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:26:50.417702 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 1333/2681] 10.222.0.82 () {58 vars in 1122 bytes} [Tue Aug 2 10:26:50 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 02 10:26:50.421666 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80eed83d-bab1-4974-b4d8-bbab4729f7df admin admin] Authenticating user token {{(pid=80377) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:26:50.432000 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80eed83d-bab1-4974-b4d8-bbab4729f7df admin admin] Validating token access rules against request {{(pid=80377) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:26:50.436583 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90f22538-bf43-4870-8da9-9ade22cae020 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '891005fa5ff74041b75747d16cc66798', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7047912c5acd4adeba5d87c3bf895e39', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80377) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:26:50.437184 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90f22538-bf43-4870-8da9-9ade22cae020 admin admin] REQUEST_METHOD: `GET` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:26:50.437491 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90f22538-bf43-4870-8da9-9ade22cae020 admin admin] SCRIPT_NAME: `/identity` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:26:50.437767 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90f22538-bf43-4870-8da9-9ade22cae020 admin admin] PATH_INFO: `/v3/roles` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:26:50.438484 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90f22538-bf43-4870-8da9-9ade22cae020 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:26:50.440059 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90f22538-bf43-4870-8da9-9ade22cae020 admin admin] RBAC: Authorization granted {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:26:50.452389 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: [pid: 80377|app: 0|req: 1349/2682] 10.222.0.82 () {58 vars in 1122 bytes} [Tue Aug 2 10:26:50 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 02 10:26:50.456216 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0888e5cd-4cc2-40bf-a981-844ae5160a8e admin admin] Authenticating user token {{(pid=80378) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:26:50.466209 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0888e5cd-4cc2-40bf-a981-844ae5160a8e admin admin] Validating token access rules against request {{(pid=80378) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:26:50.470840 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af16a40d-3ca9-48c4-94c0-16327a73b26e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '891005fa5ff74041b75747d16cc66798', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7047912c5acd4adeba5d87c3bf895e39', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80378) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:26:50.471490 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af16a40d-3ca9-48c4-94c0-16327a73b26e admin admin] REQUEST_METHOD: `PUT` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:26:50.471771 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af16a40d-3ca9-48c4-94c0-16327a73b26e admin admin] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:26:50.472070 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af16a40d-3ca9-48c4-94c0-16327a73b26e admin admin] PATH_INFO: `/v3/projects/b21cf8fe4b7c465eb056afa63d0c1b19/users/b97e15c946754d2597ed072988319879/roles/e1c04b19fc4543b8b54f58bbcffbc598` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:26:50.486319 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af16a40d-3ca9-48c4-94c0-16327a73b26e admin admin] RBAC: Authorizing `identity:create_grant(project_id=b21cf8fe4b7c465eb056afa63d0c1b19, user_id=b97e15c946754d2597ed072988319879, role_id=e1c04b19fc4543b8b54f58bbcffbc598)` {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:26:50.488867 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af16a40d-3ca9-48c4-94c0-16327a73b26e admin admin] RBAC: Authorization granted {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:26:50.502717 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 1334/2683] 10.222.0.82 () {60 vars in 1483 bytes} [Tue Aug 2 10:26:50 2022] PUT /identity/v3/projects/b21cf8fe4b7c465eb056afa63d0c1b19/users/b97e15c946754d2597ed072988319879/roles/e1c04b19fc4543b8b54f58bbcffbc598 => generated 0 bytes in 47 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 02 10:26:50.514980 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a00b8637-f2ff-4080-bd67-610a3324f695 None None] REQUEST_METHOD: `POST` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:26:50.515273 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a00b8637-f2ff-4080-bd67-610a3324f695 None None] SCRIPT_NAME: `/identity` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:26:50.515560 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a00b8637-f2ff-4080-bd67-610a3324f695 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:26:50.543643 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.auth.core [None req-a00b8637-f2ff-4080-bd67-610a3324f695 None None] MFA Rules not processed for user `b97e15c946754d2597ed072988319879`. Rule list: `[]` (Enabled: `True`). {{(pid=80377) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 02 10:26:50.544873 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.fernet_utils [None req-a00b8637-f2ff-4080-bd67-610a3324f695 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80377) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 10:26:50.608695 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: [pid: 80377|app: 0|req: 1350/2684] 10.222.0.82 () {58 vars in 958 bytes} [Tue Aug 2 10:26:50 2022] POST /identity/v3/auth/tokens => generated 3238 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 02 10:26:50.641412 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af16a40d-3ca9-48c4-94c0-16327a73b26e admin admin] Authenticating user token {{(pid=80378) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:26:50.643132 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.fernet_utils [None req-af16a40d-3ca9-48c4-94c0-16327a73b26e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80378) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 10:26:50.699375 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af16a40d-3ca9-48c4-94c0-16327a73b26e admin admin] Validating token access rules against request {{(pid=80378) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:26:50.703003 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ce02825-5798-4e9f-bd4e-00d7efdd89b4 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1023de281ae74e9d8a6d6c312c1c97a7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f1b38ebb804c41c69772c998aae72c41', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80378) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:26:50.703620 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ce02825-5798-4e9f-bd4e-00d7efdd89b4 service cinder] REQUEST_METHOD: `GET` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:26:50.703902 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ce02825-5798-4e9f-bd4e-00d7efdd89b4 service cinder] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:26:50.704175 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ce02825-5798-4e9f-bd4e-00d7efdd89b4 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:26:50.712388 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ce02825-5798-4e9f-bd4e-00d7efdd89b4 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:26:50.713808 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ce02825-5798-4e9f-bd4e-00d7efdd89b4 service cinder] RBAC: Authorization granted {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:26:50.722949 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 1335/2685] 10.222.0.82 () {60 vars in 1365 bytes} [Tue Aug 2 10:26:50 2022] GET /identity/v3/auth/tokens => generated 3238 bytes in 82 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 02 10:26:50.738601 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a00b8637-f2ff-4080-bd67-610a3324f695 None None] Authenticating user token {{(pid=80377) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:26:50.800890 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a00b8637-f2ff-4080-bd67-610a3324f695 None None] Validating token access rules against request {{(pid=80377) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:26:50.805487 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-338da923-6e8e-44d8-a7da-6b90c4d8d482 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '891005fa5ff74041b75747d16cc66798', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7047912c5acd4adeba5d87c3bf895e39', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80377) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:26:50.806779 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-338da923-6e8e-44d8-a7da-6b90c4d8d482 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:26:50.807059 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-338da923-6e8e-44d8-a7da-6b90c4d8d482 admin admin] SCRIPT_NAME: `/identity` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:26:50.807335 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-338da923-6e8e-44d8-a7da-6b90c4d8d482 admin admin] PATH_INFO: `/v3/users/b97e15c946754d2597ed072988319879` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:26:50.808508 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-338da923-6e8e-44d8-a7da-6b90c4d8d482 admin admin] RBAC: Authorizing `identity:delete_user(user_id=b97e15c946754d2597ed072988319879)` {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:26:50.810475 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-338da923-6e8e-44d8-a7da-6b90c4d8d482 admin admin] RBAC: Authorization granted {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:26:50.856073 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.notifications [None req-338da923-6e8e-44d8-a7da-6b90c4d8d482 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'b97e15c946754d2597ed072988319879'} {{(pid=80377) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 10:26:50.857701 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG dbcounter [-] [80377] Writing DB stats keystone:SELECT=90,keystone:INSERT=3,keystone:DELETE=7 {{(pid=80377) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 10:26:50.863862 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.notifications [None req-338da923-6e8e-44d8-a7da-6b90c4d8d482 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'b97e15c946754d2597ed072988319879'} {{(pid=80377) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 10:26:50.872498 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.notifications [None req-338da923-6e8e-44d8-a7da-6b90c4d8d482 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'b97e15c946754d2597ed072988319879'} {{(pid=80377) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 10:26:50.873163 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.notifications [None req-338da923-6e8e-44d8-a7da-6b90c4d8d482 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'b97e15c946754d2597ed072988319879'} {{(pid=80377) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 10:26:50.885251 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.notifications [None req-338da923-6e8e-44d8-a7da-6b90c4d8d482 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'b97e15c946754d2597ed072988319879'} {{(pid=80377) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 10:26:50.891211 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: [pid: 80377|app: 0|req: 1351/2686] 10.222.0.82 () {58 vars in 1224 bytes} [Tue Aug 2 10:26:50 2022] DELETE /identity/v3/users/b97e15c946754d2597ed072988319879 => generated 0 bytes in 153 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 02 10:26:50.895721 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53c0977e-d894-4cb1-92cd-5fcd3d77311d None None] REQUEST_METHOD: `POST` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:26:50.896013 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53c0977e-d894-4cb1-92cd-5fcd3d77311d None None] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:26:50.896296 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53c0977e-d894-4cb1-92cd-5fcd3d77311d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:26:50.916503 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.auth.core [None req-53c0977e-d894-4cb1-92cd-5fcd3d77311d None None] MFA Rules not processed for user `891005fa5ff74041b75747d16cc66798`. Rule list: `[]` (Enabled: `True`). {{(pid=80378) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 02 10:26:50.917698 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.fernet_utils [None req-53c0977e-d894-4cb1-92cd-5fcd3d77311d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80378) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 10:26:50.974604 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 1336/2687] 10.222.0.82 () {58 vars in 958 bytes} [Tue Aug 2 10:26:50 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 80 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 02 10:26:50.984723 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-338da923-6e8e-44d8-a7da-6b90c4d8d482 admin admin] Authenticating user token {{(pid=80377) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:26:50.986162 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.fernet_utils [None req-338da923-6e8e-44d8-a7da-6b90c4d8d482 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80377) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 10:26:51.045088 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53c0977e-d894-4cb1-92cd-5fcd3d77311d None None] Authenticating user token {{(pid=80378) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:26:51.046611 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.fernet_utils [None req-53c0977e-d894-4cb1-92cd-5fcd3d77311d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80378) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 10:26:51.049148 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-338da923-6e8e-44d8-a7da-6b90c4d8d482 admin admin] Validating token access rules against request {{(pid=80377) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:26:51.052838 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-779a9e1c-e9df-4eb5-a161-0b19b32a3b8e service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '33b87170f0634b67bce60750847b5cbe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f1b38ebb804c41c69772c998aae72c41', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80377) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:26:51.053560 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-779a9e1c-e9df-4eb5-a161-0b19b32a3b8e service neutron] REQUEST_METHOD: `GET` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:26:51.053863 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-779a9e1c-e9df-4eb5-a161-0b19b32a3b8e service neutron] SCRIPT_NAME: `/identity` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:26:51.054134 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-779a9e1c-e9df-4eb5-a161-0b19b32a3b8e service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:26:51.058531 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53c0977e-d894-4cb1-92cd-5fcd3d77311d None None] Validating token access rules against request {{(pid=80378) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:26:51.063282 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9222f827-065e-4adf-8855-45c71913f44a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '891005fa5ff74041b75747d16cc66798', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7047912c5acd4adeba5d87c3bf895e39', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80378) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:26:51.063719 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-779a9e1c-e9df-4eb5-a161-0b19b32a3b8e service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:26:51.063932 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9222f827-065e-4adf-8855-45c71913f44a admin admin] REQUEST_METHOD: `DELETE` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:26:51.064216 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9222f827-065e-4adf-8855-45c71913f44a admin admin] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:26:51.064492 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9222f827-065e-4adf-8855-45c71913f44a admin admin] PATH_INFO: `/v3/users/e3a7eea489b64161983b08aa489bd33c` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:26:51.065357 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-779a9e1c-e9df-4eb5-a161-0b19b32a3b8e service neutron] RBAC: Authorization granted {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:26:51.065669 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9222f827-065e-4adf-8855-45c71913f44a admin admin] RBAC: Authorizing `identity:delete_user(user_id=e3a7eea489b64161983b08aa489bd33c)` {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:26:51.067878 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9222f827-065e-4adf-8855-45c71913f44a admin admin] RBAC: Authorization granted {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:26:51.075348 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: [pid: 80377|app: 0|req: 1352/2688] 10.222.0.82 () {60 vars in 1365 bytes} [Tue Aug 2 10:26:50 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 91 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 02 10:26:51.089633 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG dbcounter [-] [80378] Writing DB stats keystone:DELETE=4,keystone:SELECT=93,keystone:INSERT=3 {{(pid=80378) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 10:26:51.125519 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.notifications [None req-9222f827-065e-4adf-8855-45c71913f44a admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'e3a7eea489b64161983b08aa489bd33c'} {{(pid=80378) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 10:26:51.133166 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.notifications [None req-9222f827-065e-4adf-8855-45c71913f44a admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'e3a7eea489b64161983b08aa489bd33c'} {{(pid=80378) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 10:26:51.142068 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.notifications [None req-9222f827-065e-4adf-8855-45c71913f44a admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'e3a7eea489b64161983b08aa489bd33c'} {{(pid=80378) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 10:26:51.143111 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.notifications [None req-9222f827-065e-4adf-8855-45c71913f44a admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'e3a7eea489b64161983b08aa489bd33c'} {{(pid=80378) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 10:26:51.156798 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.notifications [None req-9222f827-065e-4adf-8855-45c71913f44a admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'e3a7eea489b64161983b08aa489bd33c'} {{(pid=80378) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 10:26:51.163185 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 1337/2689] 10.222.0.82 () {58 vars in 1224 bytes} [Tue Aug 2 10:26:51 2022] DELETE /identity/v3/users/e3a7eea489b64161983b08aa489bd33c => generated 0 bytes in 118 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 02 10:26:51.168163 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34173db6-ebb4-41d1-92a8-0c5684c182ee None None] REQUEST_METHOD: `POST` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:26:51.168456 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34173db6-ebb4-41d1-92a8-0c5684c182ee None None] SCRIPT_NAME: `/identity` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:26:51.169192 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34173db6-ebb4-41d1-92a8-0c5684c182ee None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:26:51.188313 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.auth.core [None req-34173db6-ebb4-41d1-92a8-0c5684c182ee None None] MFA Rules not processed for user `891005fa5ff74041b75747d16cc66798`. Rule list: `[]` (Enabled: `True`). {{(pid=80377) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 02 10:26:51.189517 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.fernet_utils [None req-34173db6-ebb4-41d1-92a8-0c5684c182ee None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80377) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 10:26:51.253702 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: [pid: 80377|app: 0|req: 1353/2690] 10.222.0.82 () {58 vars in 958 bytes} [Tue Aug 2 10:26:51 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 02 10:26:51.263672 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9222f827-065e-4adf-8855-45c71913f44a admin admin] Authenticating user token {{(pid=80378) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:26:51.265250 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.fernet_utils [None req-9222f827-065e-4adf-8855-45c71913f44a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80378) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 10:26:51.326355 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9222f827-065e-4adf-8855-45c71913f44a admin admin] Validating token access rules against request {{(pid=80378) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:26:51.330043 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c61370f2-6edc-412d-90a0-2778878fc1e1 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '33b87170f0634b67bce60750847b5cbe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f1b38ebb804c41c69772c998aae72c41', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80378) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:26:51.330666 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c61370f2-6edc-412d-90a0-2778878fc1e1 service neutron] REQUEST_METHOD: `GET` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:26:51.330951 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c61370f2-6edc-412d-90a0-2778878fc1e1 service neutron] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:26:51.331231 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c61370f2-6edc-412d-90a0-2778878fc1e1 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:26:51.341117 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c61370f2-6edc-412d-90a0-2778878fc1e1 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:26:51.342669 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c61370f2-6edc-412d-90a0-2778878fc1e1 service neutron] RBAC: Authorization granted {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:26:51.353096 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 1338/2691] 10.222.0.82 () {60 vars in 1365 bytes} [Tue Aug 2 10:26:51 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 90 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 02 10:26:51.363408 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34173db6-ebb4-41d1-92a8-0c5684c182ee None None] Authenticating user token {{(pid=80377) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:26:51.365093 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.fernet_utils [None req-34173db6-ebb4-41d1-92a8-0c5684c182ee None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80377) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 10:26:51.378757 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34173db6-ebb4-41d1-92a8-0c5684c182ee None None] Validating token access rules against request {{(pid=80377) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:26:51.383673 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d21a4b90-6768-4191-9a13-bf4603d8c943 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '891005fa5ff74041b75747d16cc66798', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7047912c5acd4adeba5d87c3bf895e39', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80377) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:26:51.385064 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d21a4b90-6768-4191-9a13-bf4603d8c943 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:26:51.385344 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d21a4b90-6768-4191-9a13-bf4603d8c943 admin admin] SCRIPT_NAME: `/identity` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:26:51.385675 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d21a4b90-6768-4191-9a13-bf4603d8c943 admin admin] PATH_INFO: `/v3/projects/b21cf8fe4b7c465eb056afa63d0c1b19` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:26:51.386921 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d21a4b90-6768-4191-9a13-bf4603d8c943 admin admin] RBAC: Authorizing `identity:delete_project(project_id=b21cf8fe4b7c465eb056afa63d0c1b19)` {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:26:51.389334 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d21a4b90-6768-4191-9a13-bf4603d8c943 admin admin] RBAC: Authorization granted {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:26:51.440983 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.notifications [None req-d21a4b90-6768-4191-9a13-bf4603d8c943 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'b21cf8fe4b7c465eb056afa63d0c1b19', 'request_id': 'req-d21a4b90-6768-4191-9a13-bf4603d8c943'} {{(pid=80377) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 10:26:51.447831 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.notifications [None req-d21a4b90-6768-4191-9a13-bf4603d8c943 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'b21cf8fe4b7c465eb056afa63d0c1b19', 'request_id': 'req-d21a4b90-6768-4191-9a13-bf4603d8c943'} {{(pid=80377) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 10:26:51.454097 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.notifications [None req-d21a4b90-6768-4191-9a13-bf4603d8c943 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'b21cf8fe4b7c465eb056afa63d0c1b19', 'request_id': 'req-d21a4b90-6768-4191-9a13-bf4603d8c943'} {{(pid=80377) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 10:26:51.455986 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG dbcounter [-] [80377] Writing DB stats keystone:INSERT=1,keystone:DELETE=9,keystone:SELECT=90 {{(pid=80377) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 10:26:51.470191 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.notifications [None req-d21a4b90-6768-4191-9a13-bf4603d8c943 admin admin] The token cache is being invalidate because project b21cf8fe4b7c465eb056afa63d0c1b19 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80377) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 02 10:26:51.470753 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.notifications [None req-d21a4b90-6768-4191-9a13-bf4603d8c943 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80377) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 10:26:51.471790 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.notifications [None req-d21a4b90-6768-4191-9a13-bf4603d8c943 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80377) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 10:26:51.473863 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: [pid: 80377|app: 0|req: 1354/2692] 10.222.0.82 () {58 vars in 1233 bytes} [Tue Aug 2 10:26:51 2022] DELETE /identity/v3/projects/b21cf8fe4b7c465eb056afa63d0c1b19 => generated 0 bytes in 111 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 02 10:26:51.478760 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3fcf88e-1318-44de-8797-52755cd3bce9 None None] REQUEST_METHOD: `POST` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:26:51.479047 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3fcf88e-1318-44de-8797-52755cd3bce9 None None] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:26:51.479320 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3fcf88e-1318-44de-8797-52755cd3bce9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:26:51.498705 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.auth.core [None req-b3fcf88e-1318-44de-8797-52755cd3bce9 None None] MFA Rules not processed for user `891005fa5ff74041b75747d16cc66798`. Rule list: `[]` (Enabled: `True`). {{(pid=80378) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 02 10:26:51.500109 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.fernet_utils [None req-b3fcf88e-1318-44de-8797-52755cd3bce9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80378) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 10:26:51.564363 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 1339/2693] 10.222.0.82 () {58 vars in 958 bytes} [Tue Aug 2 10:26:51 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 02 10:26:51.572658 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e8d5876-8a91-45eb-b93e-7a25de4797c8 None None] REQUEST_METHOD: `POST` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:26:51.573140 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e8d5876-8a91-45eb-b93e-7a25de4797c8 None None] SCRIPT_NAME: `/identity` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:26:51.573492 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e8d5876-8a91-45eb-b93e-7a25de4797c8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:26:51.595501 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.auth.core [None req-5e8d5876-8a91-45eb-b93e-7a25de4797c8 None None] MFA Rules not processed for user `891005fa5ff74041b75747d16cc66798`. Rule list: `[]` (Enabled: `True`). {{(pid=80377) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 02 10:26:51.597349 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.fernet_utils [None req-5e8d5876-8a91-45eb-b93e-7a25de4797c8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80377) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 10:26:51.605714 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: [pid: 80377|app: 0|req: 1355/2694] 10.222.0.82 () {58 vars in 958 bytes} [Tue Aug 2 10:26:51 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 34 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 02 10:26:51.610088 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3fcf88e-1318-44de-8797-52755cd3bce9 None None] Authenticating user token {{(pid=80378) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:26:51.619572 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3fcf88e-1318-44de-8797-52755cd3bce9 None None] Validating token access rules against request {{(pid=80378) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:26:51.625165 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eae29c04-9aa5-48c7-a626-72f322fd10bb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '891005fa5ff74041b75747d16cc66798', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7047912c5acd4adeba5d87c3bf895e39', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80378) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:26:51.625874 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eae29c04-9aa5-48c7-a626-72f322fd10bb admin admin] REQUEST_METHOD: `GET` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:26:51.626156 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eae29c04-9aa5-48c7-a626-72f322fd10bb admin admin] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:26:51.626478 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eae29c04-9aa5-48c7-a626-72f322fd10bb admin admin] PATH_INFO: `/v3/domains` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:26:51.627165 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eae29c04-9aa5-48c7-a626-72f322fd10bb admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80378) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 02 10:26:51.627451 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eae29c04-9aa5-48c7-a626-72f322fd10bb admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:26:51.629040 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eae29c04-9aa5-48c7-a626-72f322fd10bb admin admin] RBAC: Authorization granted {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:26:51.642308 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e8d5876-8a91-45eb-b93e-7a25de4797c8 None None] Authenticating user token {{(pid=80377) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:26:51.643841 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.fernet_utils [None req-5e8d5876-8a91-45eb-b93e-7a25de4797c8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80377) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 10:26:51.644109 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 1340/2695] 10.222.0.82 () {58 vars in 1153 bytes} [Tue Aug 2 10:26:51 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 02 10:26:51.648484 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eae29c04-9aa5-48c7-a626-72f322fd10bb admin admin] Authenticating user token {{(pid=80378) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:26:51.657355 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e8d5876-8a91-45eb-b93e-7a25de4797c8 None None] Validating token access rules against request {{(pid=80377) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:26:51.660309 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eae29c04-9aa5-48c7-a626-72f322fd10bb admin admin] Validating token access rules against request {{(pid=80378) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:26:51.662707 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6dea9c8-5390-4165-8954-62a20ba74df1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '891005fa5ff74041b75747d16cc66798', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7047912c5acd4adeba5d87c3bf895e39', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80377) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:26:51.663434 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6dea9c8-5390-4165-8954-62a20ba74df1 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:26:51.663855 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6dea9c8-5390-4165-8954-62a20ba74df1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:26:51.664094 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6dea9c8-5390-4165-8954-62a20ba74df1 admin admin] PATH_INFO: `/v3/projects/4a3a0cebeb5f44ca8373430c42cc7747` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:26:51.665408 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6dea9c8-5390-4165-8954-62a20ba74df1 admin admin] RBAC: Authorizing `identity:delete_project(project_id=4a3a0cebeb5f44ca8373430c42cc7747)` {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:26:51.665586 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2109f43-7f94-40a9-bcbb-885cd283ce24 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '891005fa5ff74041b75747d16cc66798', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7047912c5acd4adeba5d87c3bf895e39', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80378) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:26:51.666226 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2109f43-7f94-40a9-bcbb-885cd283ce24 admin admin] REQUEST_METHOD: `POST` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:26:51.666512 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2109f43-7f94-40a9-bcbb-885cd283ce24 admin admin] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:26:51.666791 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2109f43-7f94-40a9-bcbb-885cd283ce24 admin admin] PATH_INFO: `/v3/projects` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:26:51.667463 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2109f43-7f94-40a9-bcbb-885cd283ce24 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:26:51.667653 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6dea9c8-5390-4165-8954-62a20ba74df1 admin admin] RBAC: Authorization granted {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:26:51.669488 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2109f43-7f94-40a9-bcbb-885cd283ce24 admin admin] RBAC: Authorization granted {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:26:51.699715 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 1341/2696] 10.222.0.82 () {60 vars in 1153 bytes} [Tue Aug 2 10:26:51 2022] POST /identity/v3/projects => generated 373 bytes in 52 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 02 10:26:51.703947 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2109f43-7f94-40a9-bcbb-885cd283ce24 admin admin] Authenticating user token {{(pid=80378) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:26:51.708894 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG dbcounter [-] [80378] Writing DB stats keystone:DELETE=9,keystone:SELECT=89,keystone:INSERT=2 {{(pid=80378) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 10:26:51.721048 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.notifications [None req-b6dea9c8-5390-4165-8954-62a20ba74df1 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '4a3a0cebeb5f44ca8373430c42cc7747', 'request_id': 'req-b6dea9c8-5390-4165-8954-62a20ba74df1'} {{(pid=80377) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 10:26:51.727266 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.notifications [None req-b6dea9c8-5390-4165-8954-62a20ba74df1 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '4a3a0cebeb5f44ca8373430c42cc7747', 'request_id': 'req-b6dea9c8-5390-4165-8954-62a20ba74df1'} {{(pid=80377) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 10:26:51.733693 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.notifications [None req-b6dea9c8-5390-4165-8954-62a20ba74df1 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '4a3a0cebeb5f44ca8373430c42cc7747', 'request_id': 'req-b6dea9c8-5390-4165-8954-62a20ba74df1'} {{(pid=80377) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 10:26:51.747881 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.notifications [None req-b6dea9c8-5390-4165-8954-62a20ba74df1 admin admin] The token cache is being invalidate because project 4a3a0cebeb5f44ca8373430c42cc7747 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80377) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 02 10:26:51.748242 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.notifications [None req-b6dea9c8-5390-4165-8954-62a20ba74df1 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80377) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 10:26:51.748975 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.notifications [None req-b6dea9c8-5390-4165-8954-62a20ba74df1 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80377) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 10:26:51.750824 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: [pid: 80377|app: 0|req: 1356/2697] 10.222.0.82 () {58 vars in 1233 bytes} [Tue Aug 2 10:26:51 2022] DELETE /identity/v3/projects/4a3a0cebeb5f44ca8373430c42cc7747 => generated 0 bytes in 109 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 02 10:26:51.775682 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2109f43-7f94-40a9-bcbb-885cd283ce24 admin admin] Validating token access rules against request {{(pid=80378) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:26:51.839605 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4eecfa2e-fd24-4826-8602-c395a2938e56 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '891005fa5ff74041b75747d16cc66798', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7047912c5acd4adeba5d87c3bf895e39', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80378) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:26:51.840233 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4eecfa2e-fd24-4826-8602-c395a2938e56 admin admin] REQUEST_METHOD: `POST` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:26:51.840518 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4eecfa2e-fd24-4826-8602-c395a2938e56 admin admin] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:26:51.840823 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4eecfa2e-fd24-4826-8602-c395a2938e56 admin admin] PATH_INFO: `/v3/users` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:26:51.841497 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4eecfa2e-fd24-4826-8602-c395a2938e56 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:26:51.843515 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4eecfa2e-fd24-4826-8602-c395a2938e56 admin admin] RBAC: Authorization granted {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:26:51.866824 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 1342/2698] 10.222.0.82 () {60 vars in 1144 bytes} [Tue Aug 2 10:26:51 2022] POST /identity/v3/users => generated 287 bytes in 163 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 02 10:26:51.871754 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6dea9c8-5390-4165-8954-62a20ba74df1 admin admin] Authenticating user token {{(pid=80377) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:26:51.873732 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.fernet_utils [None req-b6dea9c8-5390-4165-8954-62a20ba74df1 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80377) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 10:26:51.889855 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6dea9c8-5390-4165-8954-62a20ba74df1 admin admin] Validating token access rules against request {{(pid=80377) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:26:51.894832 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-142a32a0-792b-4c0d-aea3-e67f86d8556c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '891005fa5ff74041b75747d16cc66798', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7047912c5acd4adeba5d87c3bf895e39', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80377) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:26:51.895478 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-142a32a0-792b-4c0d-aea3-e67f86d8556c admin admin] REQUEST_METHOD: `GET` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:26:51.895770 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-142a32a0-792b-4c0d-aea3-e67f86d8556c admin admin] SCRIPT_NAME: `/identity` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:26:51.896048 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-142a32a0-792b-4c0d-aea3-e67f86d8556c admin admin] PATH_INFO: `/v3/roles` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:26:51.896769 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-142a32a0-792b-4c0d-aea3-e67f86d8556c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:26:51.898478 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-142a32a0-792b-4c0d-aea3-e67f86d8556c admin admin] RBAC: Authorization granted {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:26:51.914152 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: [pid: 80377|app: 0|req: 1357/2699] 10.222.0.82 () {58 vars in 1122 bytes} [Tue Aug 2 10:26:51 2022] GET /identity/v3/roles => generated 1420 bytes in 43 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 02 10:26:51.915131 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4eecfa2e-fd24-4826-8602-c395a2938e56 admin admin] Authenticating user token {{(pid=80378) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:26:51.925457 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4eecfa2e-fd24-4826-8602-c395a2938e56 admin admin] Validating token access rules against request {{(pid=80378) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:26:51.931148 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5521ac74-5401-483a-9c8a-aea61648b928 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '891005fa5ff74041b75747d16cc66798', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7047912c5acd4adeba5d87c3bf895e39', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80378) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:26:51.931817 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5521ac74-5401-483a-9c8a-aea61648b928 admin admin] REQUEST_METHOD: `GET` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:26:51.932101 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5521ac74-5401-483a-9c8a-aea61648b928 admin admin] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:26:51.932372 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5521ac74-5401-483a-9c8a-aea61648b928 admin admin] PATH_INFO: `/v3/roles` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:26:51.933082 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5521ac74-5401-483a-9c8a-aea61648b928 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:26:51.934765 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5521ac74-5401-483a-9c8a-aea61648b928 admin admin] RBAC: Authorization granted {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:26:51.947664 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 1343/2700] 10.222.0.82 () {58 vars in 1122 bytes} [Tue Aug 2 10:26:51 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 02 10:26:51.952179 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-142a32a0-792b-4c0d-aea3-e67f86d8556c admin admin] Authenticating user token {{(pid=80377) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:26:51.963057 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-142a32a0-792b-4c0d-aea3-e67f86d8556c admin admin] Validating token access rules against request {{(pid=80377) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:26:51.967862 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0cf34763-4845-4270-9605-a2d1e1d5d507 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '891005fa5ff74041b75747d16cc66798', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7047912c5acd4adeba5d87c3bf895e39', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80377) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:26:51.968553 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cf34763-4845-4270-9605-a2d1e1d5d507 admin admin] REQUEST_METHOD: `PUT` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:26:51.968838 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cf34763-4845-4270-9605-a2d1e1d5d507 admin admin] SCRIPT_NAME: `/identity` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:26:51.969115 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cf34763-4845-4270-9605-a2d1e1d5d507 admin admin] PATH_INFO: `/v3/projects/39d68c62b4d14344803f8cc90529dbcb/users/7cb48be16c684a288318ad01d8c3f383/roles/e1c04b19fc4543b8b54f58bbcffbc598` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:26:51.985746 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0cf34763-4845-4270-9605-a2d1e1d5d507 admin admin] RBAC: Authorizing `identity:create_grant(project_id=39d68c62b4d14344803f8cc90529dbcb, user_id=7cb48be16c684a288318ad01d8c3f383, role_id=e1c04b19fc4543b8b54f58bbcffbc598)` {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:26:51.988471 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0cf34763-4845-4270-9605-a2d1e1d5d507 admin admin] RBAC: Authorization granted {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:26:52.002931 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: [pid: 80377|app: 0|req: 1358/2701] 10.222.0.82 () {60 vars in 1483 bytes} [Tue Aug 2 10:26:51 2022] PUT /identity/v3/projects/39d68c62b4d14344803f8cc90529dbcb/users/7cb48be16c684a288318ad01d8c3f383/roles/e1c04b19fc4543b8b54f58bbcffbc598 => generated 0 bytes in 51 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 02 10:26:52.013983 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26b0c0fa-0c89-42da-8387-ad71a944980c None None] REQUEST_METHOD: `POST` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:26:52.014444 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26b0c0fa-0c89-42da-8387-ad71a944980c None None] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:26:52.014560 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26b0c0fa-0c89-42da-8387-ad71a944980c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:26:52.043556 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.auth.core [None req-26b0c0fa-0c89-42da-8387-ad71a944980c None None] MFA Rules not processed for user `7cb48be16c684a288318ad01d8c3f383`. Rule list: `[]` (Enabled: `True`). {{(pid=80378) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 02 10:26:52.044792 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.fernet_utils [None req-26b0c0fa-0c89-42da-8387-ad71a944980c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80378) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 10:26:52.107425 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 1344/2702] 10.222.0.82 () {58 vars in 958 bytes} [Tue Aug 2 10:26:52 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 02 10:26:52.118917 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0cf34763-4845-4270-9605-a2d1e1d5d507 admin admin] Authenticating user token {{(pid=80377) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:26:52.120635 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.fernet_utils [None req-0cf34763-4845-4270-9605-a2d1e1d5d507 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80377) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 10:26:52.176719 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0cf34763-4845-4270-9605-a2d1e1d5d507 admin admin] Validating token access rules against request {{(pid=80377) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:26:52.180348 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3b011bc-0f4b-40e6-8602-d411840420b2 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1023de281ae74e9d8a6d6c312c1c97a7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f1b38ebb804c41c69772c998aae72c41', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80377) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:26:52.181532 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3b011bc-0f4b-40e6-8602-d411840420b2 service cinder] REQUEST_METHOD: `GET` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:26:52.181892 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3b011bc-0f4b-40e6-8602-d411840420b2 service cinder] SCRIPT_NAME: `/identity` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:26:52.182170 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3b011bc-0f4b-40e6-8602-d411840420b2 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:26:52.190210 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3b011bc-0f4b-40e6-8602-d411840420b2 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:26:52.191624 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3b011bc-0f4b-40e6-8602-d411840420b2 service cinder] RBAC: Authorization granted {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:26:52.200464 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: [pid: 80377|app: 0|req: 1359/2703] 10.222.0.82 () {60 vars in 1365 bytes} [Tue Aug 2 10:26:52 2022] GET /identity/v3/auth/tokens => generated 3250 bytes in 83 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 02 10:27:02.104158 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG dbcounter [-] [80378] Writing DB stats keystone:SELECT=89,keystone:INSERT=3 {{(pid=80378) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 10:27:02.196556 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG dbcounter [-] [80377] Writing DB stats keystone:INSERT=3,keystone:DELETE=8,keystone:SELECT=73 {{(pid=80377) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 10:27:11.596043 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26b0c0fa-0c89-42da-8387-ad71a944980c None None] Authenticating user token {{(pid=80378) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:27:11.598163 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.fernet_utils [None req-26b0c0fa-0c89-42da-8387-ad71a944980c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80378) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 10:27:11.654440 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26b0c0fa-0c89-42da-8387-ad71a944980c None None] Validating token access rules against request {{(pid=80378) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:27:11.658078 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6252e074-41fc-4326-bc82-5f3cc8867092 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '33b87170f0634b67bce60750847b5cbe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f1b38ebb804c41c69772c998aae72c41', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80378) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:27:11.658693 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6252e074-41fc-4326-bc82-5f3cc8867092 service neutron] REQUEST_METHOD: `GET` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:27:11.658975 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6252e074-41fc-4326-bc82-5f3cc8867092 service neutron] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:27:11.659250 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6252e074-41fc-4326-bc82-5f3cc8867092 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:27:11.661056 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.fernet_utils [None req-6252e074-41fc-4326-bc82-5f3cc8867092 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80378) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 10:27:11.670352 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6252e074-41fc-4326-bc82-5f3cc8867092 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:27:11.671767 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6252e074-41fc-4326-bc82-5f3cc8867092 service neutron] RBAC: Authorization granted {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:27:11.680146 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 1345/2704] 10.222.0.82 () {60 vars in 1365 bytes} [Tue Aug 2 10:27:11 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 85 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 02 10:27:16.083211 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3b011bc-0f4b-40e6-8602-d411840420b2 service cinder] Authenticating user token {{(pid=80377) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:27:16.269370 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3b011bc-0f4b-40e6-8602-d411840420b2 service cinder] Validating token access rules against request {{(pid=80377) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:27:16.274470 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cead0938-3925-46da-a985-96fa85604b81 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '891005fa5ff74041b75747d16cc66798', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7047912c5acd4adeba5d87c3bf895e39', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80377) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:27:16.275099 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cead0938-3925-46da-a985-96fa85604b81 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:27:16.275375 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cead0938-3925-46da-a985-96fa85604b81 admin admin] SCRIPT_NAME: `/identity` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:27:16.275643 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cead0938-3925-46da-a985-96fa85604b81 admin admin] PATH_INFO: `/v3/users/7cb48be16c684a288318ad01d8c3f383` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:27:16.276774 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cead0938-3925-46da-a985-96fa85604b81 admin admin] RBAC: Authorizing `identity:delete_user(user_id=7cb48be16c684a288318ad01d8c3f383)` {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:27:16.278709 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cead0938-3925-46da-a985-96fa85604b81 admin admin] RBAC: Authorization granted {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:27:16.323379 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.notifications [None req-cead0938-3925-46da-a985-96fa85604b81 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '7cb48be16c684a288318ad01d8c3f383'} {{(pid=80377) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 10:27:16.331907 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.notifications [None req-cead0938-3925-46da-a985-96fa85604b81 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '7cb48be16c684a288318ad01d8c3f383'} {{(pid=80377) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 10:27:16.339386 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.notifications [None req-cead0938-3925-46da-a985-96fa85604b81 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '7cb48be16c684a288318ad01d8c3f383'} {{(pid=80377) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 10:27:16.340026 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.notifications [None req-cead0938-3925-46da-a985-96fa85604b81 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '7cb48be16c684a288318ad01d8c3f383'} {{(pid=80377) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 10:27:16.351855 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.notifications [None req-cead0938-3925-46da-a985-96fa85604b81 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '7cb48be16c684a288318ad01d8c3f383'} {{(pid=80377) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 10:27:16.357703 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: [pid: 80377|app: 0|req: 1360/2705] 10.222.0.82 () {58 vars in 1224 bytes} [Tue Aug 2 10:27:16 2022] DELETE /identity/v3/users/7cb48be16c684a288318ad01d8c3f383 => generated 0 bytes in 275 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 02 10:27:16.362335 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27f73b90-7507-45f3-8927-1acc17ddd3c7 None None] REQUEST_METHOD: `POST` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:27:16.362826 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27f73b90-7507-45f3-8927-1acc17ddd3c7 None None] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:27:16.362935 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27f73b90-7507-45f3-8927-1acc17ddd3c7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:27:16.381051 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.auth.core [None req-27f73b90-7507-45f3-8927-1acc17ddd3c7 None None] MFA Rules not processed for user `891005fa5ff74041b75747d16cc66798`. Rule list: `[]` (Enabled: `True`). {{(pid=80378) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 02 10:27:16.382289 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.fernet_utils [None req-27f73b90-7507-45f3-8927-1acc17ddd3c7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80378) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 10:27:16.440573 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 1346/2706] 10.222.0.82 () {58 vars in 958 bytes} [Tue Aug 2 10:27:16 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 79 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 02 10:27:16.451125 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cead0938-3925-46da-a985-96fa85604b81 admin admin] Authenticating user token {{(pid=80377) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:27:16.452586 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.fernet_utils [None req-cead0938-3925-46da-a985-96fa85604b81 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80377) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 10:27:16.507539 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cead0938-3925-46da-a985-96fa85604b81 admin admin] Validating token access rules against request {{(pid=80377) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:27:16.511710 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53ab2eca-0deb-43ba-a391-63e56784890c service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '33b87170f0634b67bce60750847b5cbe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f1b38ebb804c41c69772c998aae72c41', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80377) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:27:16.512325 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53ab2eca-0deb-43ba-a391-63e56784890c service neutron] REQUEST_METHOD: `GET` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:27:16.512610 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53ab2eca-0deb-43ba-a391-63e56784890c service neutron] SCRIPT_NAME: `/identity` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:27:16.512917 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53ab2eca-0deb-43ba-a391-63e56784890c service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:27:16.522796 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53ab2eca-0deb-43ba-a391-63e56784890c service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:27:16.524256 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53ab2eca-0deb-43ba-a391-63e56784890c service neutron] RBAC: Authorization granted {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:27:16.534824 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: [pid: 80377|app: 0|req: 1361/2707] 10.222.0.82 () {60 vars in 1365 bytes} [Tue Aug 2 10:27:16 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 84 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 02 10:27:16.799158 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27f73b90-7507-45f3-8927-1acc17ddd3c7 None None] Authenticating user token {{(pid=80378) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:27:16.800700 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.fernet_utils [None req-27f73b90-7507-45f3-8927-1acc17ddd3c7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80378) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 10:27:16.812730 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27f73b90-7507-45f3-8927-1acc17ddd3c7 None None] Validating token access rules against request {{(pid=80378) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:27:16.817858 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b7475b4-4812-4120-bca2-659875654b8d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '891005fa5ff74041b75747d16cc66798', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7047912c5acd4adeba5d87c3bf895e39', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80378) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:27:16.818507 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b7475b4-4812-4120-bca2-659875654b8d admin admin] REQUEST_METHOD: `DELETE` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:27:16.818793 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b7475b4-4812-4120-bca2-659875654b8d admin admin] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:27:16.819068 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b7475b4-4812-4120-bca2-659875654b8d admin admin] PATH_INFO: `/v3/projects/39d68c62b4d14344803f8cc90529dbcb` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:27:16.821234 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b7475b4-4812-4120-bca2-659875654b8d admin admin] RBAC: Authorizing `identity:delete_project(project_id=39d68c62b4d14344803f8cc90529dbcb)` {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:27:16.823164 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b7475b4-4812-4120-bca2-659875654b8d admin admin] RBAC: Authorization granted {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:27:16.875358 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.notifications [None req-7b7475b4-4812-4120-bca2-659875654b8d admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '39d68c62b4d14344803f8cc90529dbcb', 'request_id': 'req-7b7475b4-4812-4120-bca2-659875654b8d'} {{(pid=80378) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 10:27:16.881807 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.notifications [None req-7b7475b4-4812-4120-bca2-659875654b8d admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '39d68c62b4d14344803f8cc90529dbcb', 'request_id': 'req-7b7475b4-4812-4120-bca2-659875654b8d'} {{(pid=80378) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 10:27:16.887875 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.notifications [None req-7b7475b4-4812-4120-bca2-659875654b8d admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '39d68c62b4d14344803f8cc90529dbcb', 'request_id': 'req-7b7475b4-4812-4120-bca2-659875654b8d'} {{(pid=80378) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 10:27:16.900610 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.notifications [None req-7b7475b4-4812-4120-bca2-659875654b8d admin admin] The token cache is being invalidate because project 39d68c62b4d14344803f8cc90529dbcb was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80378) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 02 10:27:16.900897 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.notifications [None req-7b7475b4-4812-4120-bca2-659875654b8d admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80378) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 10:27:16.901515 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.notifications [None req-7b7475b4-4812-4120-bca2-659875654b8d admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80378) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 10:27:16.903056 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 1347/2708] 10.222.0.82 () {58 vars in 1233 bytes} [Tue Aug 2 10:27:16 2022] DELETE /identity/v3/projects/39d68c62b4d14344803f8cc90529dbcb => generated 0 bytes in 104 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 02 10:27:16.907936 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-099f6342-513f-4b7b-a54f-b3cc333a5847 None None] REQUEST_METHOD: `POST` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:27:16.908228 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-099f6342-513f-4b7b-a54f-b3cc333a5847 None None] SCRIPT_NAME: `/identity` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:27:16.908511 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-099f6342-513f-4b7b-a54f-b3cc333a5847 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:27:16.927643 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.auth.core [None req-099f6342-513f-4b7b-a54f-b3cc333a5847 None None] MFA Rules not processed for user `891005fa5ff74041b75747d16cc66798`. Rule list: `[]` (Enabled: `True`). {{(pid=80377) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 02 10:27:16.928839 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.fernet_utils [None req-099f6342-513f-4b7b-a54f-b3cc333a5847 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80377) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 10:27:16.945221 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG dbcounter [-] [80377] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80377) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 10:27:16.988800 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: [pid: 80377|app: 0|req: 1362/2709] 10.222.0.82 () {58 vars in 958 bytes} [Tue Aug 2 10:27:16 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 82 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 02 10:27:16.997542 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b952adc6-e23f-4ff5-8fa4-4ea8e821dfab None None] REQUEST_METHOD: `POST` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:27:16.997862 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b952adc6-e23f-4ff5-8fa4-4ea8e821dfab None None] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:27:16.998145 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b952adc6-e23f-4ff5-8fa4-4ea8e821dfab None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:27:17.017941 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.auth.core [None req-b952adc6-e23f-4ff5-8fa4-4ea8e821dfab None None] MFA Rules not processed for user `891005fa5ff74041b75747d16cc66798`. Rule list: `[]` (Enabled: `True`). {{(pid=80378) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 02 10:27:17.019152 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.fernet_utils [None req-b952adc6-e23f-4ff5-8fa4-4ea8e821dfab None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80378) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 10:27:17.027540 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 1348/2710] 10.222.0.82 () {58 vars in 958 bytes} [Tue Aug 2 10:27:16 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 02 10:27:17.032492 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-099f6342-513f-4b7b-a54f-b3cc333a5847 None None] Authenticating user token {{(pid=80377) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:27:17.041398 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-099f6342-513f-4b7b-a54f-b3cc333a5847 None None] Validating token access rules against request {{(pid=80377) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:27:17.046142 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c73a96f4-c76c-40e3-a660-f787cd4a5165 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '891005fa5ff74041b75747d16cc66798', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7047912c5acd4adeba5d87c3bf895e39', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80377) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:27:17.046696 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c73a96f4-c76c-40e3-a660-f787cd4a5165 admin admin] REQUEST_METHOD: `GET` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:27:17.046977 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c73a96f4-c76c-40e3-a660-f787cd4a5165 admin admin] SCRIPT_NAME: `/identity` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:27:17.047251 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c73a96f4-c76c-40e3-a660-f787cd4a5165 admin admin] PATH_INFO: `/v3/domains` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:27:17.047975 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c73a96f4-c76c-40e3-a660-f787cd4a5165 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80377) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 02 10:27:17.048286 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c73a96f4-c76c-40e3-a660-f787cd4a5165 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:27:17.049893 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c73a96f4-c76c-40e3-a660-f787cd4a5165 admin admin] RBAC: Authorization granted {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:27:17.061588 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: [pid: 80377|app: 0|req: 1363/2711] 10.222.0.82 () {58 vars in 1153 bytes} [Tue Aug 2 10:27:17 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 02 10:27:17.065807 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b952adc6-e23f-4ff5-8fa4-4ea8e821dfab None None] Authenticating user token {{(pid=80378) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:27:17.075648 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b952adc6-e23f-4ff5-8fa4-4ea8e821dfab None None] Validating token access rules against request {{(pid=80378) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:27:17.080207 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e314b9ab-3d3d-4b56-b846-25f5d3663369 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '891005fa5ff74041b75747d16cc66798', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7047912c5acd4adeba5d87c3bf895e39', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80378) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:27:17.080802 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e314b9ab-3d3d-4b56-b846-25f5d3663369 admin admin] REQUEST_METHOD: `POST` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:27:17.081085 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e314b9ab-3d3d-4b56-b846-25f5d3663369 admin admin] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:27:17.081357 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e314b9ab-3d3d-4b56-b846-25f5d3663369 admin admin] PATH_INFO: `/v3/projects` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:27:17.082065 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e314b9ab-3d3d-4b56-b846-25f5d3663369 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:27:17.083949 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e314b9ab-3d3d-4b56-b846-25f5d3663369 admin admin] RBAC: Authorization granted {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:27:17.104675 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG dbcounter [-] [80378] Writing DB stats keystone:SELECT=91,keystone:DELETE=7,keystone:INSERT=2 {{(pid=80378) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 10:27:17.112606 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 1349/2712] 10.222.0.82 () {60 vars in 1153 bytes} [Tue Aug 2 10:27:17 2022] POST /identity/v3/projects => generated 375 bytes in 47 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 02 10:27:17.116722 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c73a96f4-c76c-40e3-a660-f787cd4a5165 admin admin] Authenticating user token {{(pid=80377) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:27:17.126763 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c73a96f4-c76c-40e3-a660-f787cd4a5165 admin admin] Validating token access rules against request {{(pid=80377) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:27:17.182684 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42c5a4d0-7c21-41fc-b4df-c37048611b65 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '891005fa5ff74041b75747d16cc66798', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7047912c5acd4adeba5d87c3bf895e39', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80377) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:27:17.183275 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42c5a4d0-7c21-41fc-b4df-c37048611b65 admin admin] REQUEST_METHOD: `POST` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:27:17.183552 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42c5a4d0-7c21-41fc-b4df-c37048611b65 admin admin] SCRIPT_NAME: `/identity` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:27:17.183822 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42c5a4d0-7c21-41fc-b4df-c37048611b65 admin admin] PATH_INFO: `/v3/users` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:27:17.184472 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42c5a4d0-7c21-41fc-b4df-c37048611b65 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:27:17.186404 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42c5a4d0-7c21-41fc-b4df-c37048611b65 admin admin] RBAC: Authorization granted {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:27:17.208205 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: [pid: 80377|app: 0|req: 1364/2713] 10.222.0.82 () {60 vars in 1144 bytes} [Tue Aug 2 10:27:17 2022] POST /identity/v3/users => generated 288 bytes in 92 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 02 10:27:17.212346 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e314b9ab-3d3d-4b56-b846-25f5d3663369 admin admin] Authenticating user token {{(pid=80378) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:27:17.222561 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e314b9ab-3d3d-4b56-b846-25f5d3663369 admin admin] Validating token access rules against request {{(pid=80378) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:27:17.227145 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4dc3810e-a457-47ad-b3e8-b2a78aebccda admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '891005fa5ff74041b75747d16cc66798', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7047912c5acd4adeba5d87c3bf895e39', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80378) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:27:17.227750 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4dc3810e-a457-47ad-b3e8-b2a78aebccda admin admin] REQUEST_METHOD: `GET` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:27:17.228035 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4dc3810e-a457-47ad-b3e8-b2a78aebccda admin admin] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:27:17.228307 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4dc3810e-a457-47ad-b3e8-b2a78aebccda admin admin] PATH_INFO: `/v3/roles` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:27:17.228994 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4dc3810e-a457-47ad-b3e8-b2a78aebccda admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:27:17.230609 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4dc3810e-a457-47ad-b3e8-b2a78aebccda admin admin] RBAC: Authorization granted {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:27:17.240277 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 1350/2714] 10.222.0.82 () {58 vars in 1122 bytes} [Tue Aug 2 10:27:17 2022] GET /identity/v3/roles => generated 1420 bytes in 28 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 02 10:27:17.244591 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42c5a4d0-7c21-41fc-b4df-c37048611b65 admin admin] Authenticating user token {{(pid=80377) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:27:17.254382 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42c5a4d0-7c21-41fc-b4df-c37048611b65 admin admin] Validating token access rules against request {{(pid=80377) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:27:17.259500 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31e653b9-bf62-4026-a4de-1a57cf343bee admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '891005fa5ff74041b75747d16cc66798', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7047912c5acd4adeba5d87c3bf895e39', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80377) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:27:17.260084 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31e653b9-bf62-4026-a4de-1a57cf343bee admin admin] REQUEST_METHOD: `GET` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:27:17.260367 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31e653b9-bf62-4026-a4de-1a57cf343bee admin admin] SCRIPT_NAME: `/identity` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:27:17.260645 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31e653b9-bf62-4026-a4de-1a57cf343bee admin admin] PATH_INFO: `/v3/roles` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:27:17.261338 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31e653b9-bf62-4026-a4de-1a57cf343bee admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:27:17.262954 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31e653b9-bf62-4026-a4de-1a57cf343bee admin admin] RBAC: Authorization granted {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:27:17.272704 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: [pid: 80377|app: 0|req: 1365/2715] 10.222.0.82 () {58 vars in 1122 bytes} [Tue Aug 2 10:27:17 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 02 10:27:17.276637 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4dc3810e-a457-47ad-b3e8-b2a78aebccda admin admin] Authenticating user token {{(pid=80378) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:27:17.286289 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4dc3810e-a457-47ad-b3e8-b2a78aebccda admin admin] Validating token access rules against request {{(pid=80378) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:27:17.290781 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc0a3508-318e-459e-82e7-8804ba81e569 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '891005fa5ff74041b75747d16cc66798', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7047912c5acd4adeba5d87c3bf895e39', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80378) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:27:17.291425 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc0a3508-318e-459e-82e7-8804ba81e569 admin admin] REQUEST_METHOD: `PUT` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:27:17.291709 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc0a3508-318e-459e-82e7-8804ba81e569 admin admin] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:27:17.291983 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc0a3508-318e-459e-82e7-8804ba81e569 admin admin] PATH_INFO: `/v3/projects/dd53da20c3384939932d03445a207641/users/84316c610ee24715bedae2cc4448447e/roles/e1c04b19fc4543b8b54f58bbcffbc598` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:27:17.306746 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc0a3508-318e-459e-82e7-8804ba81e569 admin admin] RBAC: Authorizing `identity:create_grant(project_id=dd53da20c3384939932d03445a207641, user_id=84316c610ee24715bedae2cc4448447e, role_id=e1c04b19fc4543b8b54f58bbcffbc598)` {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:27:17.309291 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc0a3508-318e-459e-82e7-8804ba81e569 admin admin] RBAC: Authorization granted {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:27:17.322523 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 1351/2716] 10.222.0.82 () {60 vars in 1483 bytes} [Tue Aug 2 10:27:17 2022] PUT /identity/v3/projects/dd53da20c3384939932d03445a207641/users/84316c610ee24715bedae2cc4448447e/roles/e1c04b19fc4543b8b54f58bbcffbc598 => generated 0 bytes in 46 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 02 10:27:17.332860 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad3ed19f-0d46-4401-884e-78f5c44772de None None] REQUEST_METHOD: `POST` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:27:17.333143 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad3ed19f-0d46-4401-884e-78f5c44772de None None] SCRIPT_NAME: `/identity` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:27:17.333424 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad3ed19f-0d46-4401-884e-78f5c44772de None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:27:17.360643 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.auth.core [None req-ad3ed19f-0d46-4401-884e-78f5c44772de None None] MFA Rules not processed for user `84316c610ee24715bedae2cc4448447e`. Rule list: `[]` (Enabled: `True`). {{(pid=80377) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 02 10:27:17.361884 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.fernet_utils [None req-ad3ed19f-0d46-4401-884e-78f5c44772de None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80377) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 10:27:17.424507 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: [pid: 80377|app: 0|req: 1366/2717] 10.222.0.82 () {58 vars in 958 bytes} [Tue Aug 2 10:27:17 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 02 10:27:17.435804 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc0a3508-318e-459e-82e7-8804ba81e569 admin admin] Authenticating user token {{(pid=80378) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:27:17.437488 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.fernet_utils [None req-bc0a3508-318e-459e-82e7-8804ba81e569 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80378) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 10:27:17.492970 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc0a3508-318e-459e-82e7-8804ba81e569 admin admin] Validating token access rules against request {{(pid=80378) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:27:17.496565 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1848d20e-9f2e-44ff-8b23-479d29f62930 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1023de281ae74e9d8a6d6c312c1c97a7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f1b38ebb804c41c69772c998aae72c41', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80378) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:27:17.497175 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1848d20e-9f2e-44ff-8b23-479d29f62930 service cinder] REQUEST_METHOD: `GET` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:27:17.497457 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1848d20e-9f2e-44ff-8b23-479d29f62930 service cinder] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:27:17.497739 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1848d20e-9f2e-44ff-8b23-479d29f62930 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:27:17.505897 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1848d20e-9f2e-44ff-8b23-479d29f62930 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:27:17.507309 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1848d20e-9f2e-44ff-8b23-479d29f62930 service cinder] RBAC: Authorization granted {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:27:17.516760 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 1352/2718] 10.222.0.82 () {60 vars in 1365 bytes} [Tue Aug 2 10:27:17 2022] GET /identity/v3/auth/tokens => generated 3252 bytes in 81 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 02 10:27:20.518440 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad3ed19f-0d46-4401-884e-78f5c44772de None None] Authenticating user token {{(pid=80377) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:27:20.537242 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG dbcounter [-] [80377] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80377) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 10:27:20.583369 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad3ed19f-0d46-4401-884e-78f5c44772de None None] Validating token access rules against request {{(pid=80377) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:27:20.587843 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f16ebe0-f8e5-456d-899d-6032322794df admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '891005fa5ff74041b75747d16cc66798', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7047912c5acd4adeba5d87c3bf895e39', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80377) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:27:20.588753 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f16ebe0-f8e5-456d-899d-6032322794df admin admin] REQUEST_METHOD: `DELETE` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:27:20.589097 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f16ebe0-f8e5-456d-899d-6032322794df admin admin] SCRIPT_NAME: `/identity` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:27:20.589374 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f16ebe0-f8e5-456d-899d-6032322794df admin admin] PATH_INFO: `/v3/users/84316c610ee24715bedae2cc4448447e` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:27:20.590569 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f16ebe0-f8e5-456d-899d-6032322794df admin admin] RBAC: Authorizing `identity:delete_user(user_id=84316c610ee24715bedae2cc4448447e)` {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:27:20.592533 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f16ebe0-f8e5-456d-899d-6032322794df admin admin] RBAC: Authorization granted {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:27:20.637973 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.notifications [None req-2f16ebe0-f8e5-456d-899d-6032322794df admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '84316c610ee24715bedae2cc4448447e'} {{(pid=80377) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 10:27:20.645060 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.notifications [None req-2f16ebe0-f8e5-456d-899d-6032322794df admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '84316c610ee24715bedae2cc4448447e'} {{(pid=80377) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 10:27:20.651977 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.notifications [None req-2f16ebe0-f8e5-456d-899d-6032322794df admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '84316c610ee24715bedae2cc4448447e'} {{(pid=80377) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 10:27:20.652606 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.notifications [None req-2f16ebe0-f8e5-456d-899d-6032322794df admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '84316c610ee24715bedae2cc4448447e'} {{(pid=80377) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 10:27:20.664688 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.notifications [None req-2f16ebe0-f8e5-456d-899d-6032322794df admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '84316c610ee24715bedae2cc4448447e'} {{(pid=80377) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 10:27:20.670694 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: [pid: 80377|app: 0|req: 1367/2719] 10.222.0.82 () {58 vars in 1224 bytes} [Tue Aug 2 10:27:20 2022] DELETE /identity/v3/users/84316c610ee24715bedae2cc4448447e => generated 0 bytes in 153 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 02 10:27:20.675249 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbbbaf81-dda3-4b7f-8945-44e21ca77b8e None None] REQUEST_METHOD: `POST` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:27:20.675716 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbbbaf81-dda3-4b7f-8945-44e21ca77b8e None None] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:27:20.675834 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbbbaf81-dda3-4b7f-8945-44e21ca77b8e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:27:20.696602 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.auth.core [None req-fbbbaf81-dda3-4b7f-8945-44e21ca77b8e None None] MFA Rules not processed for user `891005fa5ff74041b75747d16cc66798`. Rule list: `[]` (Enabled: `True`). {{(pid=80378) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 02 10:27:20.697884 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.fernet_utils [None req-fbbbaf81-dda3-4b7f-8945-44e21ca77b8e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80378) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 10:27:20.756443 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 1353/2720] 10.222.0.82 () {58 vars in 958 bytes} [Tue Aug 2 10:27:20 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 82 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 02 10:27:20.766222 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f16ebe0-f8e5-456d-899d-6032322794df admin admin] Authenticating user token {{(pid=80377) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:27:20.767621 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.fernet_utils [None req-2f16ebe0-f8e5-456d-899d-6032322794df admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80377) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 10:27:20.824303 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f16ebe0-f8e5-456d-899d-6032322794df admin admin] Validating token access rules against request {{(pid=80377) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:27:20.827922 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d5580be-fbde-47be-8701-2f42fc6bff09 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '33b87170f0634b67bce60750847b5cbe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f1b38ebb804c41c69772c998aae72c41', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80377) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:27:20.828534 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d5580be-fbde-47be-8701-2f42fc6bff09 service neutron] REQUEST_METHOD: `GET` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:27:20.828827 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d5580be-fbde-47be-8701-2f42fc6bff09 service neutron] SCRIPT_NAME: `/identity` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:27:20.829085 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d5580be-fbde-47be-8701-2f42fc6bff09 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:27:20.837899 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d5580be-fbde-47be-8701-2f42fc6bff09 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:27:20.839312 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d5580be-fbde-47be-8701-2f42fc6bff09 service neutron] RBAC: Authorization granted {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:27:20.848527 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: [pid: 80377|app: 0|req: 1368/2721] 10.222.0.82 () {60 vars in 1365 bytes} [Tue Aug 2 10:27:20 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 83 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 02 10:27:21.118361 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbbbaf81-dda3-4b7f-8945-44e21ca77b8e None None] Authenticating user token {{(pid=80378) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:27:21.119824 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.fernet_utils [None req-fbbbaf81-dda3-4b7f-8945-44e21ca77b8e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80378) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 10:27:21.134733 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbbbaf81-dda3-4b7f-8945-44e21ca77b8e None None] Validating token access rules against request {{(pid=80378) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:27:21.139317 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-171aef15-0a9b-4046-8d3b-478ba7f8c37c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '891005fa5ff74041b75747d16cc66798', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7047912c5acd4adeba5d87c3bf895e39', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80378) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:27:21.139967 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-171aef15-0a9b-4046-8d3b-478ba7f8c37c admin admin] REQUEST_METHOD: `DELETE` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:27:21.140250 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-171aef15-0a9b-4046-8d3b-478ba7f8c37c admin admin] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:27:21.140527 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-171aef15-0a9b-4046-8d3b-478ba7f8c37c admin admin] PATH_INFO: `/v3/projects/dd53da20c3384939932d03445a207641` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:27:21.142217 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-171aef15-0a9b-4046-8d3b-478ba7f8c37c admin admin] RBAC: Authorizing `identity:delete_project(project_id=dd53da20c3384939932d03445a207641)` {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:27:21.144329 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-171aef15-0a9b-4046-8d3b-478ba7f8c37c admin admin] RBAC: Authorization granted {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:27:21.191695 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.notifications [None req-171aef15-0a9b-4046-8d3b-478ba7f8c37c admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'dd53da20c3384939932d03445a207641', 'request_id': 'req-171aef15-0a9b-4046-8d3b-478ba7f8c37c'} {{(pid=80378) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 10:27:21.198235 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.notifications [None req-171aef15-0a9b-4046-8d3b-478ba7f8c37c admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'dd53da20c3384939932d03445a207641', 'request_id': 'req-171aef15-0a9b-4046-8d3b-478ba7f8c37c'} {{(pid=80378) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 10:27:21.202645 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG dbcounter [-] [80378] Writing DB stats keystone:SELECT=93,keystone:INSERT=1,keystone:DELETE=6 {{(pid=80378) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 10:27:21.203838 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.notifications [None req-171aef15-0a9b-4046-8d3b-478ba7f8c37c admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'dd53da20c3384939932d03445a207641', 'request_id': 'req-171aef15-0a9b-4046-8d3b-478ba7f8c37c'} {{(pid=80378) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 10:27:21.216591 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.notifications [None req-171aef15-0a9b-4046-8d3b-478ba7f8c37c admin admin] The token cache is being invalidate because project dd53da20c3384939932d03445a207641 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80378) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 02 10:27:21.216874 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.notifications [None req-171aef15-0a9b-4046-8d3b-478ba7f8c37c admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80378) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 10:27:21.217506 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.notifications [None req-171aef15-0a9b-4046-8d3b-478ba7f8c37c admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80378) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 10:27:21.218991 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 1354/2722] 10.222.0.82 () {58 vars in 1233 bytes} [Tue Aug 2 10:27:21 2022] DELETE /identity/v3/projects/dd53da20c3384939932d03445a207641 => generated 0 bytes in 102 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 02 10:27:21.223950 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c530413a-53ba-4530-8861-bbc17dcbabc5 None None] REQUEST_METHOD: `POST` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:27:21.224234 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c530413a-53ba-4530-8861-bbc17dcbabc5 None None] SCRIPT_NAME: `/identity` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:27:21.224526 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c530413a-53ba-4530-8861-bbc17dcbabc5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:27:21.244083 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.auth.core [None req-c530413a-53ba-4530-8861-bbc17dcbabc5 None None] MFA Rules not processed for user `891005fa5ff74041b75747d16cc66798`. Rule list: `[]` (Enabled: `True`). {{(pid=80377) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 02 10:27:21.245297 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.fernet_utils [None req-c530413a-53ba-4530-8861-bbc17dcbabc5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80377) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 10:27:21.278074 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG dbcounter [-] [80377] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80377) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 10:27:21.304284 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: [pid: 80377|app: 0|req: 1369/2723] 10.222.0.82 () {58 vars in 958 bytes} [Tue Aug 2 10:27:21 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 81 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 02 10:27:21.312674 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc43e474-b202-4a2b-8424-671b91d3e22a None None] REQUEST_METHOD: `POST` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:27:21.313035 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc43e474-b202-4a2b-8424-671b91d3e22a None None] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:27:21.313318 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc43e474-b202-4a2b-8424-671b91d3e22a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:27:21.332891 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.auth.core [None req-bc43e474-b202-4a2b-8424-671b91d3e22a None None] MFA Rules not processed for user `891005fa5ff74041b75747d16cc66798`. Rule list: `[]` (Enabled: `True`). {{(pid=80378) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 02 10:27:21.334112 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.fernet_utils [None req-bc43e474-b202-4a2b-8424-671b91d3e22a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80378) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 10:27:21.341884 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 1355/2724] 10.222.0.82 () {58 vars in 958 bytes} [Tue Aug 2 10:27:21 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 02 10:27:21.346008 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c530413a-53ba-4530-8861-bbc17dcbabc5 None None] Authenticating user token {{(pid=80377) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:27:21.356266 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c530413a-53ba-4530-8861-bbc17dcbabc5 None None] Validating token access rules against request {{(pid=80377) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:27:21.360844 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78bcc362-8f2c-4d56-a163-e4fbb6077bbb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '891005fa5ff74041b75747d16cc66798', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7047912c5acd4adeba5d87c3bf895e39', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80377) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:27:21.361454 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78bcc362-8f2c-4d56-a163-e4fbb6077bbb admin admin] REQUEST_METHOD: `GET` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:27:21.361738 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78bcc362-8f2c-4d56-a163-e4fbb6077bbb admin admin] SCRIPT_NAME: `/identity` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:27:21.362052 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78bcc362-8f2c-4d56-a163-e4fbb6077bbb admin admin] PATH_INFO: `/v3/domains` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:27:21.362780 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78bcc362-8f2c-4d56-a163-e4fbb6077bbb admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80377) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 02 10:27:21.363085 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78bcc362-8f2c-4d56-a163-e4fbb6077bbb admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:27:21.364675 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78bcc362-8f2c-4d56-a163-e4fbb6077bbb admin admin] RBAC: Authorization granted {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:27:21.376786 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: [pid: 80377|app: 0|req: 1370/2725] 10.222.0.82 () {58 vars in 1153 bytes} [Tue Aug 2 10:27:21 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 02 10:27:21.381008 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc43e474-b202-4a2b-8424-671b91d3e22a None None] Authenticating user token {{(pid=80378) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:27:21.390423 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc43e474-b202-4a2b-8424-671b91d3e22a None None] Validating token access rules against request {{(pid=80378) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:27:21.395158 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35e98b1f-8b55-4024-b189-3228f8474b2c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '891005fa5ff74041b75747d16cc66798', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7047912c5acd4adeba5d87c3bf895e39', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80378) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:27:21.395799 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35e98b1f-8b55-4024-b189-3228f8474b2c admin admin] REQUEST_METHOD: `POST` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:27:21.396080 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35e98b1f-8b55-4024-b189-3228f8474b2c admin admin] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:27:21.396356 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35e98b1f-8b55-4024-b189-3228f8474b2c admin admin] PATH_INFO: `/v3/projects` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:27:21.397073 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35e98b1f-8b55-4024-b189-3228f8474b2c admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:27:21.399001 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35e98b1f-8b55-4024-b189-3228f8474b2c admin admin] RBAC: Authorization granted {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:27:21.426547 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 1356/2726] 10.222.0.82 () {60 vars in 1153 bytes} [Tue Aug 2 10:27:21 2022] POST /identity/v3/projects => generated 367 bytes in 46 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 02 10:27:21.431347 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78bcc362-8f2c-4d56-a163-e4fbb6077bbb admin admin] Authenticating user token {{(pid=80377) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:27:21.441039 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78bcc362-8f2c-4d56-a163-e4fbb6077bbb admin admin] Validating token access rules against request {{(pid=80377) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:27:21.494855 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14ebce25-dbb8-47b2-9d75-7f81ce01b5ef admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '891005fa5ff74041b75747d16cc66798', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7047912c5acd4adeba5d87c3bf895e39', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80377) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:27:21.495459 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14ebce25-dbb8-47b2-9d75-7f81ce01b5ef admin admin] REQUEST_METHOD: `POST` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:27:21.495736 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14ebce25-dbb8-47b2-9d75-7f81ce01b5ef admin admin] SCRIPT_NAME: `/identity` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:27:21.496006 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14ebce25-dbb8-47b2-9d75-7f81ce01b5ef admin admin] PATH_INFO: `/v3/users` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:27:21.496662 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14ebce25-dbb8-47b2-9d75-7f81ce01b5ef admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:27:21.498581 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14ebce25-dbb8-47b2-9d75-7f81ce01b5ef admin admin] RBAC: Authorization granted {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:27:21.519588 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: [pid: 80377|app: 0|req: 1371/2727] 10.222.0.82 () {60 vars in 1144 bytes} [Tue Aug 2 10:27:21 2022] POST /identity/v3/users => generated 284 bytes in 89 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 02 10:27:21.525272 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35e98b1f-8b55-4024-b189-3228f8474b2c admin admin] Authenticating user token {{(pid=80378) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:27:21.535774 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35e98b1f-8b55-4024-b189-3228f8474b2c admin admin] Validating token access rules against request {{(pid=80378) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:27:21.540918 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89159beb-a403-4767-8e3d-475f9dec902e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '891005fa5ff74041b75747d16cc66798', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7047912c5acd4adeba5d87c3bf895e39', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80378) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:27:21.541525 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89159beb-a403-4767-8e3d-475f9dec902e admin admin] REQUEST_METHOD: `GET` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:27:21.541835 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89159beb-a403-4767-8e3d-475f9dec902e admin admin] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:27:21.542117 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89159beb-a403-4767-8e3d-475f9dec902e admin admin] PATH_INFO: `/v3/roles` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:27:21.542814 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89159beb-a403-4767-8e3d-475f9dec902e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:27:21.544417 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89159beb-a403-4767-8e3d-475f9dec902e admin admin] RBAC: Authorization granted {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:27:21.554263 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 1357/2728] 10.222.0.82 () {58 vars in 1122 bytes} [Tue Aug 2 10:27:21 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 02 10:27:21.558593 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14ebce25-dbb8-47b2-9d75-7f81ce01b5ef admin admin] Authenticating user token {{(pid=80377) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:27:21.568590 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14ebce25-dbb8-47b2-9d75-7f81ce01b5ef admin admin] Validating token access rules against request {{(pid=80377) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:27:21.573172 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70147c66-dd0b-4ecf-86c3-c35abd1fc221 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '891005fa5ff74041b75747d16cc66798', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7047912c5acd4adeba5d87c3bf895e39', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80377) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:27:21.573777 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70147c66-dd0b-4ecf-86c3-c35abd1fc221 admin admin] REQUEST_METHOD: `GET` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:27:21.574099 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70147c66-dd0b-4ecf-86c3-c35abd1fc221 admin admin] SCRIPT_NAME: `/identity` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:27:21.574377 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70147c66-dd0b-4ecf-86c3-c35abd1fc221 admin admin] PATH_INFO: `/v3/roles` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:27:21.575078 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70147c66-dd0b-4ecf-86c3-c35abd1fc221 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:27:21.576658 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70147c66-dd0b-4ecf-86c3-c35abd1fc221 admin admin] RBAC: Authorization granted {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:27:21.587208 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: [pid: 80377|app: 0|req: 1372/2729] 10.222.0.82 () {58 vars in 1122 bytes} [Tue Aug 2 10:27:21 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 02 10:27:21.591269 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89159beb-a403-4767-8e3d-475f9dec902e admin admin] Authenticating user token {{(pid=80378) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:27:21.600909 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89159beb-a403-4767-8e3d-475f9dec902e admin admin] Validating token access rules against request {{(pid=80378) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:27:21.605949 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42d88214-a57b-46e2-bfb8-2c5d973c2ecb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '891005fa5ff74041b75747d16cc66798', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7047912c5acd4adeba5d87c3bf895e39', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80378) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:27:21.606594 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42d88214-a57b-46e2-bfb8-2c5d973c2ecb admin admin] REQUEST_METHOD: `PUT` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:27:21.606873 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42d88214-a57b-46e2-bfb8-2c5d973c2ecb admin admin] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:27:21.607145 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42d88214-a57b-46e2-bfb8-2c5d973c2ecb admin admin] PATH_INFO: `/v3/projects/5fe353add8184117a2d0e53516a40d86/users/f94b50222fcd4ef2af23dbb86e8ed95c/roles/e1c04b19fc4543b8b54f58bbcffbc598` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:27:21.621969 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42d88214-a57b-46e2-bfb8-2c5d973c2ecb admin admin] RBAC: Authorizing `identity:create_grant(project_id=5fe353add8184117a2d0e53516a40d86, user_id=f94b50222fcd4ef2af23dbb86e8ed95c, role_id=e1c04b19fc4543b8b54f58bbcffbc598)` {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:27:21.624614 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42d88214-a57b-46e2-bfb8-2c5d973c2ecb admin admin] RBAC: Authorization granted {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:27:21.637027 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 1358/2730] 10.222.0.82 () {60 vars in 1483 bytes} [Tue Aug 2 10:27:21 2022] PUT /identity/v3/projects/5fe353add8184117a2d0e53516a40d86/users/f94b50222fcd4ef2af23dbb86e8ed95c/roles/e1c04b19fc4543b8b54f58bbcffbc598 => generated 0 bytes in 46 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 02 10:27:21.646796 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cebabef-bc33-4b36-bdd5-e9c25ec3274a None None] REQUEST_METHOD: `POST` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:27:21.647087 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cebabef-bc33-4b36-bdd5-e9c25ec3274a None None] SCRIPT_NAME: `/identity` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:27:21.647367 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cebabef-bc33-4b36-bdd5-e9c25ec3274a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:27:21.674777 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.auth.core [None req-8cebabef-bc33-4b36-bdd5-e9c25ec3274a None None] MFA Rules not processed for user `f94b50222fcd4ef2af23dbb86e8ed95c`. Rule list: `[]` (Enabled: `True`). {{(pid=80377) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 02 10:27:21.675986 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.fernet_utils [None req-8cebabef-bc33-4b36-bdd5-e9c25ec3274a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80377) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 10:27:21.739380 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: [pid: 80377|app: 0|req: 1373/2731] 10.222.0.82 () {58 vars in 958 bytes} [Tue Aug 2 10:27:21 2022] POST /identity/v3/auth/tokens => generated 3244 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 02 10:27:21.750620 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42d88214-a57b-46e2-bfb8-2c5d973c2ecb admin admin] Authenticating user token {{(pid=80378) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:27:21.752387 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.fernet_utils [None req-42d88214-a57b-46e2-bfb8-2c5d973c2ecb admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80378) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 10:27:21.808046 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42d88214-a57b-46e2-bfb8-2c5d973c2ecb admin admin] Validating token access rules against request {{(pid=80378) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:27:21.811641 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be4ac91a-f648-4f19-93ca-1101a75377b0 service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa964af53a2b4fb9ace6afb544d54d40', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f1b38ebb804c41c69772c998aae72c41', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80378) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:27:21.812257 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be4ac91a-f648-4f19-93ca-1101a75377b0 service glance] REQUEST_METHOD: `GET` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:27:21.812541 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be4ac91a-f648-4f19-93ca-1101a75377b0 service glance] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:27:21.812817 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be4ac91a-f648-4f19-93ca-1101a75377b0 service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:27:21.821509 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be4ac91a-f648-4f19-93ca-1101a75377b0 service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:27:21.823023 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be4ac91a-f648-4f19-93ca-1101a75377b0 service glance] RBAC: Authorization granted {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:27:21.831854 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 1359/2732] 10.222.0.82 () {60 vars in 1365 bytes} [Tue Aug 2 10:27:21 2022] GET /identity/v3/auth/tokens => generated 3244 bytes in 82 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 02 10:27:31.736577 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG dbcounter [-] [80377] Writing DB stats keystone:SELECT=81,keystone:INSERT=3 {{(pid=80377) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 10:27:31.827828 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG dbcounter [-] [80378] Writing DB stats keystone:SELECT=56,keystone:INSERT=3,keystone:DELETE=1 {{(pid=80378) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 10:28:06.858289 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8cebabef-bc33-4b36-bdd5-e9c25ec3274a None None] Authenticating user token {{(pid=80377) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:28:06.923315 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8cebabef-bc33-4b36-bdd5-e9c25ec3274a None None] Validating token access rules against request {{(pid=80377) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:28:06.927892 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3900ef87-b621-40ab-bab1-f4e0d2d073c4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '891005fa5ff74041b75747d16cc66798', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7047912c5acd4adeba5d87c3bf895e39', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80377) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:28:06.928672 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3900ef87-b621-40ab-bab1-f4e0d2d073c4 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:28:06.928959 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3900ef87-b621-40ab-bab1-f4e0d2d073c4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:28:06.929235 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3900ef87-b621-40ab-bab1-f4e0d2d073c4 admin admin] PATH_INFO: `/v3/users/f94b50222fcd4ef2af23dbb86e8ed95c` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:28:06.930427 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3900ef87-b621-40ab-bab1-f4e0d2d073c4 admin admin] RBAC: Authorizing `identity:delete_user(user_id=f94b50222fcd4ef2af23dbb86e8ed95c)` {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:28:06.932385 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3900ef87-b621-40ab-bab1-f4e0d2d073c4 admin admin] RBAC: Authorization granted {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:28:06.981265 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.notifications [None req-3900ef87-b621-40ab-bab1-f4e0d2d073c4 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'f94b50222fcd4ef2af23dbb86e8ed95c'} {{(pid=80377) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 10:28:06.988947 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.notifications [None req-3900ef87-b621-40ab-bab1-f4e0d2d073c4 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'f94b50222fcd4ef2af23dbb86e8ed95c'} {{(pid=80377) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 10:28:06.996374 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.notifications [None req-3900ef87-b621-40ab-bab1-f4e0d2d073c4 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'f94b50222fcd4ef2af23dbb86e8ed95c'} {{(pid=80377) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 10:28:06.997003 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.notifications [None req-3900ef87-b621-40ab-bab1-f4e0d2d073c4 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'f94b50222fcd4ef2af23dbb86e8ed95c'} {{(pid=80377) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 10:28:07.008571 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.notifications [None req-3900ef87-b621-40ab-bab1-f4e0d2d073c4 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'f94b50222fcd4ef2af23dbb86e8ed95c'} {{(pid=80377) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 10:28:07.014485 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: [pid: 80377|app: 0|req: 1374/2733] 10.222.0.82 () {58 vars in 1224 bytes} [Tue Aug 2 10:28:06 2022] DELETE /identity/v3/users/f94b50222fcd4ef2af23dbb86e8ed95c => generated 0 bytes in 157 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 02 10:28:07.018942 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1f12e11-be2d-4b12-9ec1-8692627274fd None None] REQUEST_METHOD: `POST` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:28:07.019465 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1f12e11-be2d-4b12-9ec1-8692627274fd None None] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:28:07.019610 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1f12e11-be2d-4b12-9ec1-8692627274fd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:28:07.038402 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.auth.core [None req-d1f12e11-be2d-4b12-9ec1-8692627274fd None None] MFA Rules not processed for user `891005fa5ff74041b75747d16cc66798`. Rule list: `[]` (Enabled: `True`). {{(pid=80378) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 02 10:28:07.039600 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.fernet_utils [None req-d1f12e11-be2d-4b12-9ec1-8692627274fd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80378) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 10:28:07.106414 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 1360/2734] 10.222.0.82 () {58 vars in 958 bytes} [Tue Aug 2 10:28:07 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 02 10:28:07.116040 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3900ef87-b621-40ab-bab1-f4e0d2d073c4 admin admin] Authenticating user token {{(pid=80377) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:28:07.117504 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.fernet_utils [None req-3900ef87-b621-40ab-bab1-f4e0d2d073c4 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80377) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 10:28:07.173430 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3900ef87-b621-40ab-bab1-f4e0d2d073c4 admin admin] Validating token access rules against request {{(pid=80377) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:28:07.178216 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d38d8776-c958-42a6-9ce2-7b1dc5465b4a service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '33b87170f0634b67bce60750847b5cbe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f1b38ebb804c41c69772c998aae72c41', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80377) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:28:07.178870 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d38d8776-c958-42a6-9ce2-7b1dc5465b4a service neutron] REQUEST_METHOD: `GET` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:28:07.179154 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d38d8776-c958-42a6-9ce2-7b1dc5465b4a service neutron] SCRIPT_NAME: `/identity` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:28:07.179430 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d38d8776-c958-42a6-9ce2-7b1dc5465b4a service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:28:07.189446 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d38d8776-c958-42a6-9ce2-7b1dc5465b4a service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:28:07.191337 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d38d8776-c958-42a6-9ce2-7b1dc5465b4a service neutron] RBAC: Authorization granted {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:28:07.200895 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: [pid: 80377|app: 0|req: 1375/2735] 10.222.0.82 () {60 vars in 1365 bytes} [Tue Aug 2 10:28:07 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 85 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 02 10:28:07.481322 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1f12e11-be2d-4b12-9ec1-8692627274fd None None] Authenticating user token {{(pid=80378) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:28:07.483037 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.fernet_utils [None req-d1f12e11-be2d-4b12-9ec1-8692627274fd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80378) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 10:28:07.495756 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1f12e11-be2d-4b12-9ec1-8692627274fd None None] Validating token access rules against request {{(pid=80378) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:28:07.502778 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ffd0b31-5981-40de-8cf3-9dd738f2e186 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '891005fa5ff74041b75747d16cc66798', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7047912c5acd4adeba5d87c3bf895e39', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80378) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:28:07.503442 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ffd0b31-5981-40de-8cf3-9dd738f2e186 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:28:07.503864 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ffd0b31-5981-40de-8cf3-9dd738f2e186 admin admin] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:28:07.504146 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ffd0b31-5981-40de-8cf3-9dd738f2e186 admin admin] PATH_INFO: `/v3/projects/5fe353add8184117a2d0e53516a40d86` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:28:07.505354 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ffd0b31-5981-40de-8cf3-9dd738f2e186 admin admin] RBAC: Authorizing `identity:delete_project(project_id=5fe353add8184117a2d0e53516a40d86)` {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:28:07.507324 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ffd0b31-5981-40de-8cf3-9dd738f2e186 admin admin] RBAC: Authorization granted {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:28:07.564235 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.notifications [None req-9ffd0b31-5981-40de-8cf3-9dd738f2e186 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '5fe353add8184117a2d0e53516a40d86', 'request_id': 'req-9ffd0b31-5981-40de-8cf3-9dd738f2e186'} {{(pid=80378) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 10:28:07.571584 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.notifications [None req-9ffd0b31-5981-40de-8cf3-9dd738f2e186 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '5fe353add8184117a2d0e53516a40d86', 'request_id': 'req-9ffd0b31-5981-40de-8cf3-9dd738f2e186'} {{(pid=80378) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 10:28:07.578265 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.notifications [None req-9ffd0b31-5981-40de-8cf3-9dd738f2e186 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '5fe353add8184117a2d0e53516a40d86', 'request_id': 'req-9ffd0b31-5981-40de-8cf3-9dd738f2e186'} {{(pid=80378) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 10:28:07.591411 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.notifications [None req-9ffd0b31-5981-40de-8cf3-9dd738f2e186 admin admin] The token cache is being invalidate because project 5fe353add8184117a2d0e53516a40d86 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80378) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 02 10:28:07.591708 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.notifications [None req-9ffd0b31-5981-40de-8cf3-9dd738f2e186 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80378) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 10:28:07.592374 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.notifications [None req-9ffd0b31-5981-40de-8cf3-9dd738f2e186 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80378) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 10:28:07.594062 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 1361/2736] 10.222.0.82 () {58 vars in 1233 bytes} [Tue Aug 2 10:28:07 2022] DELETE /identity/v3/projects/5fe353add8184117a2d0e53516a40d86 => generated 0 bytes in 114 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 02 10:28:07.599229 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38ae2d0e-fa8c-48ce-8d99-f1eb23c58dee None None] REQUEST_METHOD: `POST` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:28:07.599516 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38ae2d0e-fa8c-48ce-8d99-f1eb23c58dee None None] SCRIPT_NAME: `/identity` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:28:07.599796 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38ae2d0e-fa8c-48ce-8d99-f1eb23c58dee None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:28:07.619654 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.auth.core [None req-38ae2d0e-fa8c-48ce-8d99-f1eb23c58dee None None] MFA Rules not processed for user `891005fa5ff74041b75747d16cc66798`. Rule list: `[]` (Enabled: `True`). {{(pid=80377) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 02 10:28:07.620818 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.fernet_utils [None req-38ae2d0e-fa8c-48ce-8d99-f1eb23c58dee None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80377) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 10:28:07.638632 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG dbcounter [-] [80377] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80377) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 10:28:07.681582 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: [pid: 80377|app: 0|req: 1376/2737] 10.222.0.82 () {58 vars in 958 bytes} [Tue Aug 2 10:28:07 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 02 10:28:07.690866 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db893337-d6f1-4b62-9291-9ff29f70a3da None None] REQUEST_METHOD: `POST` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:28:07.691157 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db893337-d6f1-4b62-9291-9ff29f70a3da None None] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:28:07.691432 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db893337-d6f1-4b62-9291-9ff29f70a3da None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:28:07.711168 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.auth.core [None req-db893337-d6f1-4b62-9291-9ff29f70a3da None None] MFA Rules not processed for user `891005fa5ff74041b75747d16cc66798`. Rule list: `[]` (Enabled: `True`). {{(pid=80378) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 02 10:28:07.712359 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.fernet_utils [None req-db893337-d6f1-4b62-9291-9ff29f70a3da None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80378) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 10:28:07.720944 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 1362/2738] 10.222.0.82 () {58 vars in 958 bytes} [Tue Aug 2 10:28:07 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 02 10:28:07.725148 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38ae2d0e-fa8c-48ce-8d99-f1eb23c58dee None None] Authenticating user token {{(pid=80377) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:28:07.734830 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38ae2d0e-fa8c-48ce-8d99-f1eb23c58dee None None] Validating token access rules against request {{(pid=80377) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:28:07.739320 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af5941b7-f473-4747-bacc-2ff874e88795 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '891005fa5ff74041b75747d16cc66798', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7047912c5acd4adeba5d87c3bf895e39', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80377) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:28:07.739941 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af5941b7-f473-4747-bacc-2ff874e88795 admin admin] REQUEST_METHOD: `GET` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:28:07.740217 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af5941b7-f473-4747-bacc-2ff874e88795 admin admin] SCRIPT_NAME: `/identity` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:28:07.740489 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af5941b7-f473-4747-bacc-2ff874e88795 admin admin] PATH_INFO: `/v3/domains` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:28:07.741182 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af5941b7-f473-4747-bacc-2ff874e88795 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80377) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 02 10:28:07.741484 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af5941b7-f473-4747-bacc-2ff874e88795 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:28:07.743108 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af5941b7-f473-4747-bacc-2ff874e88795 admin admin] RBAC: Authorization granted {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:28:07.755288 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: [pid: 80377|app: 0|req: 1377/2739] 10.222.0.82 () {58 vars in 1153 bytes} [Tue Aug 2 10:28:07 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 02 10:28:07.759715 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db893337-d6f1-4b62-9291-9ff29f70a3da None None] Authenticating user token {{(pid=80378) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:28:07.769715 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db893337-d6f1-4b62-9291-9ff29f70a3da None None] Validating token access rules against request {{(pid=80378) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:28:07.775118 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e235e64-06d9-4e48-b3d1-083fc85d960c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '891005fa5ff74041b75747d16cc66798', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7047912c5acd4adeba5d87c3bf895e39', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80378) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:28:07.775805 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e235e64-06d9-4e48-b3d1-083fc85d960c admin admin] REQUEST_METHOD: `POST` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:28:07.776187 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e235e64-06d9-4e48-b3d1-083fc85d960c admin admin] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:28:07.776528 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e235e64-06d9-4e48-b3d1-083fc85d960c admin admin] PATH_INFO: `/v3/projects` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:28:07.777908 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e235e64-06d9-4e48-b3d1-083fc85d960c admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:28:07.779797 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e235e64-06d9-4e48-b3d1-083fc85d960c admin admin] RBAC: Authorization granted {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:28:07.807211 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 1363/2740] 10.222.0.82 () {60 vars in 1153 bytes} [Tue Aug 2 10:28:07 2022] POST /identity/v3/projects => generated 375 bytes in 48 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 02 10:28:07.811351 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af5941b7-f473-4747-bacc-2ff874e88795 admin admin] Authenticating user token {{(pid=80377) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:28:07.822152 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af5941b7-f473-4747-bacc-2ff874e88795 admin admin] Validating token access rules against request {{(pid=80377) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:28:07.885397 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-512a4268-190c-4a92-80a1-9e6c237b21bb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '891005fa5ff74041b75747d16cc66798', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7047912c5acd4adeba5d87c3bf895e39', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80377) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:28:07.886586 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-512a4268-190c-4a92-80a1-9e6c237b21bb admin admin] REQUEST_METHOD: `POST` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:28:07.886881 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-512a4268-190c-4a92-80a1-9e6c237b21bb admin admin] SCRIPT_NAME: `/identity` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:28:07.887158 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-512a4268-190c-4a92-80a1-9e6c237b21bb admin admin] PATH_INFO: `/v3/users` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:28:07.887898 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-512a4268-190c-4a92-80a1-9e6c237b21bb admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:28:07.889882 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-512a4268-190c-4a92-80a1-9e6c237b21bb admin admin] RBAC: Authorization granted {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:28:07.919387 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: [pid: 80377|app: 0|req: 1378/2741] 10.222.0.82 () {60 vars in 1144 bytes} [Tue Aug 2 10:28:07 2022] POST /identity/v3/users => generated 288 bytes in 108 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 02 10:28:07.924215 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e235e64-06d9-4e48-b3d1-083fc85d960c admin admin] Authenticating user token {{(pid=80378) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:28:07.936857 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e235e64-06d9-4e48-b3d1-083fc85d960c admin admin] Validating token access rules against request {{(pid=80378) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:28:07.942123 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fa59d6a-e253-44ef-9099-8ebef3b482a2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '891005fa5ff74041b75747d16cc66798', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7047912c5acd4adeba5d87c3bf895e39', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80378) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:28:07.942834 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fa59d6a-e253-44ef-9099-8ebef3b482a2 admin admin] REQUEST_METHOD: `GET` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:28:07.943185 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fa59d6a-e253-44ef-9099-8ebef3b482a2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:28:07.943529 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fa59d6a-e253-44ef-9099-8ebef3b482a2 admin admin] PATH_INFO: `/v3/roles` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:28:07.944403 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6fa59d6a-e253-44ef-9099-8ebef3b482a2 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:28:07.946108 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6fa59d6a-e253-44ef-9099-8ebef3b482a2 admin admin] RBAC: Authorization granted {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:28:07.957263 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 1364/2742] 10.222.0.82 () {58 vars in 1122 bytes} [Tue Aug 2 10:28:07 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 02 10:28:07.961280 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-512a4268-190c-4a92-80a1-9e6c237b21bb admin admin] Authenticating user token {{(pid=80377) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:28:07.973272 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-512a4268-190c-4a92-80a1-9e6c237b21bb admin admin] Validating token access rules against request {{(pid=80377) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:28:07.978327 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a91ddd1f-0845-4710-949f-0918d9a8ed93 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '891005fa5ff74041b75747d16cc66798', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7047912c5acd4adeba5d87c3bf895e39', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80377) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:28:07.978941 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a91ddd1f-0845-4710-949f-0918d9a8ed93 admin admin] REQUEST_METHOD: `GET` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:28:07.979226 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a91ddd1f-0845-4710-949f-0918d9a8ed93 admin admin] SCRIPT_NAME: `/identity` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:28:07.979498 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a91ddd1f-0845-4710-949f-0918d9a8ed93 admin admin] PATH_INFO: `/v3/roles` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:28:07.980233 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a91ddd1f-0845-4710-949f-0918d9a8ed93 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:28:07.981875 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a91ddd1f-0845-4710-949f-0918d9a8ed93 admin admin] RBAC: Authorization granted {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:28:07.993405 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: [pid: 80377|app: 0|req: 1379/2743] 10.222.0.82 () {58 vars in 1122 bytes} [Tue Aug 2 10:28:07 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 02 10:28:07.997722 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fa59d6a-e253-44ef-9099-8ebef3b482a2 admin admin] Authenticating user token {{(pid=80378) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:28:08.007765 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fa59d6a-e253-44ef-9099-8ebef3b482a2 admin admin] Validating token access rules against request {{(pid=80378) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:28:08.012942 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f9b297f-be68-46bb-9165-4e5e709f9711 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '891005fa5ff74041b75747d16cc66798', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7047912c5acd4adeba5d87c3bf895e39', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80378) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:28:08.013619 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f9b297f-be68-46bb-9165-4e5e709f9711 admin admin] REQUEST_METHOD: `PUT` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:28:08.013943 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f9b297f-be68-46bb-9165-4e5e709f9711 admin admin] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:28:08.014219 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f9b297f-be68-46bb-9165-4e5e709f9711 admin admin] PATH_INFO: `/v3/projects/5dd64ce69554438583ff94520225323f/users/6823d533e71b42be9f4ab8106106ca11/roles/e1c04b19fc4543b8b54f58bbcffbc598` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:28:08.030985 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f9b297f-be68-46bb-9165-4e5e709f9711 admin admin] RBAC: Authorizing `identity:create_grant(project_id=5dd64ce69554438583ff94520225323f, user_id=6823d533e71b42be9f4ab8106106ca11, role_id=e1c04b19fc4543b8b54f58bbcffbc598)` {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:28:08.034999 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f9b297f-be68-46bb-9165-4e5e709f9711 admin admin] RBAC: Authorization granted {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:28:08.049132 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 1365/2744] 10.222.0.82 () {60 vars in 1483 bytes} [Tue Aug 2 10:28:07 2022] PUT /identity/v3/projects/5dd64ce69554438583ff94520225323f/users/6823d533e71b42be9f4ab8106106ca11/roles/e1c04b19fc4543b8b54f58bbcffbc598 => generated 0 bytes in 52 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 02 10:28:08.118584 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e03dad7-d597-41e6-83a9-57a999ac116f None None] REQUEST_METHOD: `POST` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:28:08.118868 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e03dad7-d597-41e6-83a9-57a999ac116f None None] SCRIPT_NAME: `/identity` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:28:08.119141 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e03dad7-d597-41e6-83a9-57a999ac116f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:28:08.148800 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.auth.core [None req-2e03dad7-d597-41e6-83a9-57a999ac116f None None] MFA Rules not processed for user `6823d533e71b42be9f4ab8106106ca11`. Rule list: `[]` (Enabled: `True`). {{(pid=80377) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 02 10:28:08.149964 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.fernet_utils [None req-2e03dad7-d597-41e6-83a9-57a999ac116f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80377) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 10:28:08.220294 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: [pid: 80377|app: 0|req: 1380/2745] 10.222.0.82 () {58 vars in 958 bytes} [Tue Aug 2 10:28:08 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 02 10:28:08.230680 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f9b297f-be68-46bb-9165-4e5e709f9711 admin admin] Authenticating user token {{(pid=80378) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:28:08.232410 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.fernet_utils [None req-2f9b297f-be68-46bb-9165-4e5e709f9711 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80378) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 10:28:08.256410 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG dbcounter [-] [80378] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=80378) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 10:28:08.294759 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f9b297f-be68-46bb-9165-4e5e709f9711 admin admin] Validating token access rules against request {{(pid=80378) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:28:08.298527 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21d97c26-edd9-4563-8c69-dbe3110b9ba4 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1023de281ae74e9d8a6d6c312c1c97a7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f1b38ebb804c41c69772c998aae72c41', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80378) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:28:08.299148 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21d97c26-edd9-4563-8c69-dbe3110b9ba4 service cinder] REQUEST_METHOD: `GET` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:28:08.299432 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21d97c26-edd9-4563-8c69-dbe3110b9ba4 service cinder] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:28:08.299706 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21d97c26-edd9-4563-8c69-dbe3110b9ba4 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:28:08.308562 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21d97c26-edd9-4563-8c69-dbe3110b9ba4 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:28:08.310098 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21d97c26-edd9-4563-8c69-dbe3110b9ba4 service cinder] RBAC: Authorization granted {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:28:08.319138 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 1366/2746] 10.222.0.82 () {60 vars in 1365 bytes} [Tue Aug 2 10:28:08 2022] GET /identity/v3/auth/tokens => generated 3252 bytes in 89 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 02 10:28:08.383689 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e03dad7-d597-41e6-83a9-57a999ac116f None None] Authenticating user token {{(pid=80377) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:28:08.385234 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.fernet_utils [None req-2e03dad7-d597-41e6-83a9-57a999ac116f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80377) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 10:28:08.415547 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG dbcounter [-] [80377] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80377) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 10:28:08.509751 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e03dad7-d597-41e6-83a9-57a999ac116f None None] Validating token access rules against request {{(pid=80377) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:28:08.526627 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9595448-65cd-44b4-bff0-846a29bd84e6 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88af518196674fe593ad88d30111fb7d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f1b38ebb804c41c69772c998aae72c41', 'project_domain_id': 'default', 'roles': ['admin', 'ResellerAdmin', 'reader', 'service', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80377) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:28:08.527944 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9595448-65cd-44b4-bff0-846a29bd84e6 service nova] REQUEST_METHOD: `GET` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:28:08.528230 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9595448-65cd-44b4-bff0-846a29bd84e6 service nova] SCRIPT_NAME: `/identity` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:28:08.528500 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9595448-65cd-44b4-bff0-846a29bd84e6 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:28:08.530374 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.fernet_utils [None req-e9595448-65cd-44b4-bff0-846a29bd84e6 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80377) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 10:28:08.544264 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9595448-65cd-44b4-bff0-846a29bd84e6 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:28:08.546184 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9595448-65cd-44b4-bff0-846a29bd84e6 service nova] RBAC: Authorization granted {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:28:08.563732 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: [pid: 80377|app: 0|req: 1381/2747] 10.222.0.82 () {60 vars in 1365 bytes} [Tue Aug 2 10:28:08 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 181 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 02 10:28:18.300276 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21d97c26-edd9-4563-8c69-dbe3110b9ba4 service cinder] Authenticating user token {{(pid=80378) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:28:18.369552 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21d97c26-edd9-4563-8c69-dbe3110b9ba4 service cinder] Validating token access rules against request {{(pid=80378) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:28:18.374270 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af0f8d9c-9eb7-460d-8696-d6612f4045c8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '891005fa5ff74041b75747d16cc66798', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7047912c5acd4adeba5d87c3bf895e39', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80378) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:28:18.374908 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af0f8d9c-9eb7-460d-8696-d6612f4045c8 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:28:18.375183 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af0f8d9c-9eb7-460d-8696-d6612f4045c8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:28:18.375449 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af0f8d9c-9eb7-460d-8696-d6612f4045c8 admin admin] PATH_INFO: `/v3/users/6823d533e71b42be9f4ab8106106ca11` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:28:18.376644 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af0f8d9c-9eb7-460d-8696-d6612f4045c8 admin admin] RBAC: Authorizing `identity:delete_user(user_id=6823d533e71b42be9f4ab8106106ca11)` {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:28:18.378584 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af0f8d9c-9eb7-460d-8696-d6612f4045c8 admin admin] RBAC: Authorization granted {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:28:18.433311 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.notifications [None req-af0f8d9c-9eb7-460d-8696-d6612f4045c8 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '6823d533e71b42be9f4ab8106106ca11'} {{(pid=80378) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 10:28:18.441564 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.notifications [None req-af0f8d9c-9eb7-460d-8696-d6612f4045c8 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '6823d533e71b42be9f4ab8106106ca11'} {{(pid=80378) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 10:28:18.450344 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.notifications [None req-af0f8d9c-9eb7-460d-8696-d6612f4045c8 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '6823d533e71b42be9f4ab8106106ca11'} {{(pid=80378) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 10:28:18.451214 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.notifications [None req-af0f8d9c-9eb7-460d-8696-d6612f4045c8 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '6823d533e71b42be9f4ab8106106ca11'} {{(pid=80378) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 10:28:18.463266 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.notifications [None req-af0f8d9c-9eb7-460d-8696-d6612f4045c8 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '6823d533e71b42be9f4ab8106106ca11'} {{(pid=80378) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 10:28:18.476056 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 1367/2748] 10.222.0.82 () {58 vars in 1224 bytes} [Tue Aug 2 10:28:18 2022] DELETE /identity/v3/users/6823d533e71b42be9f4ab8106106ca11 => generated 0 bytes in 176 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 02 10:28:18.480651 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5a32348-8cf2-483a-87c6-eaebdd19b802 None None] REQUEST_METHOD: `POST` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:28:18.481158 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5a32348-8cf2-483a-87c6-eaebdd19b802 None None] SCRIPT_NAME: `/identity` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:28:18.481272 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5a32348-8cf2-483a-87c6-eaebdd19b802 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:28:18.500365 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.auth.core [None req-d5a32348-8cf2-483a-87c6-eaebdd19b802 None None] MFA Rules not processed for user `891005fa5ff74041b75747d16cc66798`. Rule list: `[]` (Enabled: `True`). {{(pid=80377) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 02 10:28:18.501628 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.fernet_utils [None req-d5a32348-8cf2-483a-87c6-eaebdd19b802 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80377) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 10:28:18.563229 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: [pid: 80377|app: 0|req: 1382/2749] 10.222.0.82 () {58 vars in 958 bytes} [Tue Aug 2 10:28:18 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 83 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 02 10:28:18.572786 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af0f8d9c-9eb7-460d-8696-d6612f4045c8 admin admin] Authenticating user token {{(pid=80378) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:28:18.574868 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.fernet_utils [None req-af0f8d9c-9eb7-460d-8696-d6612f4045c8 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80378) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 10:28:18.622864 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG dbcounter [-] [80378] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80378) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 10:28:18.637293 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af0f8d9c-9eb7-460d-8696-d6612f4045c8 admin admin] Validating token access rules against request {{(pid=80378) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:28:18.641180 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f481b5f3-119c-4e0a-99f4-d2789971fa97 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '33b87170f0634b67bce60750847b5cbe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f1b38ebb804c41c69772c998aae72c41', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80378) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:28:18.641833 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f481b5f3-119c-4e0a-99f4-d2789971fa97 service neutron] REQUEST_METHOD: `GET` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:28:18.642116 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f481b5f3-119c-4e0a-99f4-d2789971fa97 service neutron] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:28:18.642393 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f481b5f3-119c-4e0a-99f4-d2789971fa97 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:28:18.653251 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f481b5f3-119c-4e0a-99f4-d2789971fa97 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:28:18.654820 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f481b5f3-119c-4e0a-99f4-d2789971fa97 service neutron] RBAC: Authorization granted {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:28:18.666358 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 1368/2750] 10.222.0.82 () {60 vars in 1365 bytes} [Tue Aug 2 10:28:18 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 94 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 02 10:28:18.939831 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5a32348-8cf2-483a-87c6-eaebdd19b802 None None] Authenticating user token {{(pid=80377) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:28:18.941536 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.fernet_utils [None req-d5a32348-8cf2-483a-87c6-eaebdd19b802 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80377) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 10:28:18.962838 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5a32348-8cf2-483a-87c6-eaebdd19b802 None None] Validating token access rules against request {{(pid=80377) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:28:18.976695 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b62b7b9a-9f12-4250-917e-8e35fff52bb5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '891005fa5ff74041b75747d16cc66798', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7047912c5acd4adeba5d87c3bf895e39', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80377) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:28:18.977438 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b62b7b9a-9f12-4250-917e-8e35fff52bb5 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:28:18.977734 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b62b7b9a-9f12-4250-917e-8e35fff52bb5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:28:18.978045 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b62b7b9a-9f12-4250-917e-8e35fff52bb5 admin admin] PATH_INFO: `/v3/projects/5dd64ce69554438583ff94520225323f` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:28:18.979355 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b62b7b9a-9f12-4250-917e-8e35fff52bb5 admin admin] RBAC: Authorizing `identity:delete_project(project_id=5dd64ce69554438583ff94520225323f)` {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:28:18.981279 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b62b7b9a-9f12-4250-917e-8e35fff52bb5 admin admin] RBAC: Authorization granted {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:28:19.032732 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.notifications [None req-b62b7b9a-9f12-4250-917e-8e35fff52bb5 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '5dd64ce69554438583ff94520225323f', 'request_id': 'req-b62b7b9a-9f12-4250-917e-8e35fff52bb5'} {{(pid=80377) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 10:28:19.038147 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.notifications [None req-b62b7b9a-9f12-4250-917e-8e35fff52bb5 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '5dd64ce69554438583ff94520225323f', 'request_id': 'req-b62b7b9a-9f12-4250-917e-8e35fff52bb5'} {{(pid=80377) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 10:28:19.044307 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.notifications [None req-b62b7b9a-9f12-4250-917e-8e35fff52bb5 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '5dd64ce69554438583ff94520225323f', 'request_id': 'req-b62b7b9a-9f12-4250-917e-8e35fff52bb5'} {{(pid=80377) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 10:28:19.056905 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.notifications [None req-b62b7b9a-9f12-4250-917e-8e35fff52bb5 admin admin] The token cache is being invalidate because project 5dd64ce69554438583ff94520225323f was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80377) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 02 10:28:19.057200 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.notifications [None req-b62b7b9a-9f12-4250-917e-8e35fff52bb5 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80377) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 10:28:19.057901 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.notifications [None req-b62b7b9a-9f12-4250-917e-8e35fff52bb5 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80377) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 10:28:19.059419 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: [pid: 80377|app: 0|req: 1383/2751] 10.222.0.82 () {58 vars in 1233 bytes} [Tue Aug 2 10:28:18 2022] DELETE /identity/v3/projects/5dd64ce69554438583ff94520225323f => generated 0 bytes in 120 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 02 10:28:19.064420 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a88d524-4040-4241-8de0-42d0e4c50a88 None None] REQUEST_METHOD: `POST` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:28:19.064823 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a88d524-4040-4241-8de0-42d0e4c50a88 None None] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:28:19.065157 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a88d524-4040-4241-8de0-42d0e4c50a88 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:28:19.088233 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.auth.core [None req-7a88d524-4040-4241-8de0-42d0e4c50a88 None None] MFA Rules not processed for user `891005fa5ff74041b75747d16cc66798`. Rule list: `[]` (Enabled: `True`). {{(pid=80378) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 02 10:28:19.090028 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.fernet_utils [None req-7a88d524-4040-4241-8de0-42d0e4c50a88 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80378) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 10:28:19.154285 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 1369/2752] 10.222.0.82 () {58 vars in 958 bytes} [Tue Aug 2 10:28:19 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 91 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 02 10:28:19.162504 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-849537d7-7104-4773-8665-bc96e969f89a None None] REQUEST_METHOD: `POST` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:28:19.162792 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-849537d7-7104-4773-8665-bc96e969f89a None None] SCRIPT_NAME: `/identity` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:28:19.163074 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-849537d7-7104-4773-8665-bc96e969f89a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:28:19.183104 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.auth.core [None req-849537d7-7104-4773-8665-bc96e969f89a None None] MFA Rules not processed for user `891005fa5ff74041b75747d16cc66798`. Rule list: `[]` (Enabled: `True`). {{(pid=80377) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 02 10:28:19.184502 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.fernet_utils [None req-849537d7-7104-4773-8665-bc96e969f89a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80377) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 10:28:19.197310 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: [pid: 80377|app: 0|req: 1384/2753] 10.222.0.82 () {58 vars in 958 bytes} [Tue Aug 2 10:28:19 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 36 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 02 10:28:19.202258 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a88d524-4040-4241-8de0-42d0e4c50a88 None None] Authenticating user token {{(pid=80378) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:28:19.215257 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a88d524-4040-4241-8de0-42d0e4c50a88 None None] Validating token access rules against request {{(pid=80378) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:28:19.224472 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c16ad609-7aeb-43b0-b029-144cb521bbab admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '891005fa5ff74041b75747d16cc66798', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7047912c5acd4adeba5d87c3bf895e39', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80378) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:28:19.224472 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c16ad609-7aeb-43b0-b029-144cb521bbab admin admin] REQUEST_METHOD: `GET` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:28:19.224472 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c16ad609-7aeb-43b0-b029-144cb521bbab admin admin] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:28:19.224472 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c16ad609-7aeb-43b0-b029-144cb521bbab admin admin] PATH_INFO: `/v3/domains` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:28:19.224820 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c16ad609-7aeb-43b0-b029-144cb521bbab admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80378) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 02 10:28:19.224820 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c16ad609-7aeb-43b0-b029-144cb521bbab admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:28:19.224820 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c16ad609-7aeb-43b0-b029-144cb521bbab admin admin] RBAC: Authorization granted {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:28:19.229620 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f24cb1d-9b07-4d16-9a23-f852367a0bef None None] REQUEST_METHOD: `POST` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:28:19.229939 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f24cb1d-9b07-4d16-9a23-f852367a0bef None None] SCRIPT_NAME: `/identity` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:28:19.230219 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f24cb1d-9b07-4d16-9a23-f852367a0bef None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:28:19.236568 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 1370/2754] 10.222.0.82 () {58 vars in 1153 bytes} [Tue Aug 2 10:28:19 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 02 10:28:19.240784 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c16ad609-7aeb-43b0-b029-144cb521bbab admin admin] Authenticating user token {{(pid=80378) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:28:19.249058 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.auth.core [None req-5f24cb1d-9b07-4d16-9a23-f852367a0bef None None] MFA Rules not processed for user `8774306d95c0476b80cc8e53b62e8a4b`. Rule list: `[]` (Enabled: `True`). {{(pid=80377) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 02 10:28:19.250308 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.fernet_utils [None req-5f24cb1d-9b07-4d16-9a23-f852367a0bef None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80377) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 10:28:19.250666 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c16ad609-7aeb-43b0-b029-144cb521bbab admin admin] Validating token access rules against request {{(pid=80378) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:28:19.255505 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ba393d9-d168-4839-9ba6-b8b6e311275b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '891005fa5ff74041b75747d16cc66798', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7047912c5acd4adeba5d87c3bf895e39', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80378) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:28:19.256115 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ba393d9-d168-4839-9ba6-b8b6e311275b admin admin] REQUEST_METHOD: `POST` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:28:19.256401 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ba393d9-d168-4839-9ba6-b8b6e311275b admin admin] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:28:19.256677 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ba393d9-d168-4839-9ba6-b8b6e311275b admin admin] PATH_INFO: `/v3/projects` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:28:19.257336 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ba393d9-d168-4839-9ba6-b8b6e311275b admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:28:19.259251 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ba393d9-d168-4839-9ba6-b8b6e311275b admin admin] RBAC: Authorization granted {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:28:19.265034 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG dbcounter [-] [80377] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80377) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 10:28:19.290268 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 1371/2755] 10.222.0.82 () {60 vars in 1153 bytes} [Tue Aug 2 10:28:19 2022] POST /identity/v3/projects => generated 381 bytes in 50 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 02 10:28:19.294299 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ba393d9-d168-4839-9ba6-b8b6e311275b admin admin] Authenticating user token {{(pid=80378) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:28:19.318745 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: [pid: 80377|app: 0|req: 1385/2756] 10.222.0.82 () {58 vars in 958 bytes} [Tue Aug 2 10:28:19 2022] POST /identity/v3/auth/tokens => generated 3262 bytes in 90 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 02 10:28:19.359339 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ba393d9-d168-4839-9ba6-b8b6e311275b admin admin] Validating token access rules against request {{(pid=80378) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:28:19.364217 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e630727-68db-4cae-8534-ec4da7aedb60 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '891005fa5ff74041b75747d16cc66798', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7047912c5acd4adeba5d87c3bf895e39', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80378) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:28:19.364839 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e630727-68db-4cae-8534-ec4da7aedb60 admin admin] REQUEST_METHOD: `POST` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:28:19.365124 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e630727-68db-4cae-8534-ec4da7aedb60 admin admin] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:28:19.365400 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e630727-68db-4cae-8534-ec4da7aedb60 admin admin] PATH_INFO: `/v3/users` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:28:19.366099 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e630727-68db-4cae-8534-ec4da7aedb60 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:28:19.368091 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e630727-68db-4cae-8534-ec4da7aedb60 admin admin] RBAC: Authorization granted {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:28:19.391239 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 1372/2757] 10.222.0.82 () {60 vars in 1144 bytes} [Tue Aug 2 10:28:19 2022] POST /identity/v3/users => generated 291 bytes in 97 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 02 10:28:19.395342 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f24cb1d-9b07-4d16-9a23-f852367a0bef None None] Authenticating user token {{(pid=80377) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:28:19.406297 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f24cb1d-9b07-4d16-9a23-f852367a0bef None None] Validating token access rules against request {{(pid=80377) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:28:19.411016 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1e680a7-43ac-4812-86ea-fa147bd72c76 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '891005fa5ff74041b75747d16cc66798', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7047912c5acd4adeba5d87c3bf895e39', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80377) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:28:19.411727 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1e680a7-43ac-4812-86ea-fa147bd72c76 admin admin] REQUEST_METHOD: `GET` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:28:19.412672 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1e680a7-43ac-4812-86ea-fa147bd72c76 admin admin] SCRIPT_NAME: `/identity` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:28:19.413021 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1e680a7-43ac-4812-86ea-fa147bd72c76 admin admin] PATH_INFO: `/v3/roles` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:28:19.413755 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1e680a7-43ac-4812-86ea-fa147bd72c76 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:28:19.415383 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1e680a7-43ac-4812-86ea-fa147bd72c76 admin admin] RBAC: Authorization granted {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:28:19.426099 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: [pid: 80377|app: 0|req: 1386/2758] 10.222.0.82 () {58 vars in 1122 bytes} [Tue Aug 2 10:28:19 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 02 10:28:19.430091 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e630727-68db-4cae-8534-ec4da7aedb60 admin admin] Authenticating user token {{(pid=80378) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:28:19.440747 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e630727-68db-4cae-8534-ec4da7aedb60 admin admin] Validating token access rules against request {{(pid=80378) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:28:19.445585 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a90f5e9-ade5-4405-a0e5-d25c5a85a980 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '891005fa5ff74041b75747d16cc66798', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7047912c5acd4adeba5d87c3bf895e39', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80378) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:28:19.446239 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a90f5e9-ade5-4405-a0e5-d25c5a85a980 admin admin] REQUEST_METHOD: `GET` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:28:19.446518 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a90f5e9-ade5-4405-a0e5-d25c5a85a980 admin admin] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:28:19.446790 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a90f5e9-ade5-4405-a0e5-d25c5a85a980 admin admin] PATH_INFO: `/v3/roles` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:28:19.447500 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a90f5e9-ade5-4405-a0e5-d25c5a85a980 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:28:19.449764 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a90f5e9-ade5-4405-a0e5-d25c5a85a980 admin admin] RBAC: Authorization granted {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:28:19.460064 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 1373/2759] 10.222.0.82 () {58 vars in 1122 bytes} [Tue Aug 2 10:28:19 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 02 10:28:19.464265 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1e680a7-43ac-4812-86ea-fa147bd72c76 admin admin] Authenticating user token {{(pid=80377) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:28:19.474415 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1e680a7-43ac-4812-86ea-fa147bd72c76 admin admin] Validating token access rules against request {{(pid=80377) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:28:19.479043 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec69d14d-ee06-47d9-93f1-c2c2a23ef224 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '891005fa5ff74041b75747d16cc66798', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7047912c5acd4adeba5d87c3bf895e39', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80377) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:28:19.479711 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec69d14d-ee06-47d9-93f1-c2c2a23ef224 admin admin] REQUEST_METHOD: `PUT` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:28:19.479998 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec69d14d-ee06-47d9-93f1-c2c2a23ef224 admin admin] SCRIPT_NAME: `/identity` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:28:19.480276 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec69d14d-ee06-47d9-93f1-c2c2a23ef224 admin admin] PATH_INFO: `/v3/projects/f99abdc12f9d404498711c2eb6c3363c/users/28db30abecd2452e89be5b3b89cc39e7/roles/e1c04b19fc4543b8b54f58bbcffbc598` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:28:19.496123 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec69d14d-ee06-47d9-93f1-c2c2a23ef224 admin admin] RBAC: Authorizing `identity:create_grant(project_id=f99abdc12f9d404498711c2eb6c3363c, user_id=28db30abecd2452e89be5b3b89cc39e7, role_id=e1c04b19fc4543b8b54f58bbcffbc598)` {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:28:19.499106 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec69d14d-ee06-47d9-93f1-c2c2a23ef224 admin admin] RBAC: Authorization granted {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:28:19.513400 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: [pid: 80377|app: 0|req: 1387/2760] 10.222.0.82 () {60 vars in 1483 bytes} [Tue Aug 2 10:28:19 2022] PUT /identity/v3/projects/f99abdc12f9d404498711c2eb6c3363c/users/28db30abecd2452e89be5b3b89cc39e7/roles/e1c04b19fc4543b8b54f58bbcffbc598 => generated 0 bytes in 49 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 02 10:28:19.520150 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3caeaf4-1d95-44c0-8378-802bf3cc3f61 None None] REQUEST_METHOD: `POST` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:28:19.520343 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3caeaf4-1d95-44c0-8378-802bf3cc3f61 None None] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:28:19.520525 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3caeaf4-1d95-44c0-8378-802bf3cc3f61 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:28:19.539964 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.auth.core [None req-f3caeaf4-1d95-44c0-8378-802bf3cc3f61 None None] MFA Rules not processed for user `891005fa5ff74041b75747d16cc66798`. Rule list: `[]` (Enabled: `True`). {{(pid=80378) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 02 10:28:19.541242 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.fernet_utils [None req-f3caeaf4-1d95-44c0-8378-802bf3cc3f61 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80378) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 10:28:19.566855 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG dbcounter [-] [80378] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=80378) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 10:28:19.603975 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 1374/2761] 10.222.0.82 () {58 vars in 958 bytes} [Tue Aug 2 10:28:19 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 02 10:28:19.613690 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec69d14d-ee06-47d9-93f1-c2c2a23ef224 admin admin] Authenticating user token {{(pid=80377) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:28:19.615483 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.fernet_utils [None req-ec69d14d-ee06-47d9-93f1-c2c2a23ef224 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80377) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 10:28:19.647064 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3caeaf4-1d95-44c0-8378-802bf3cc3f61 None None] Authenticating user token {{(pid=80378) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:28:19.648481 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.fernet_utils [None req-f3caeaf4-1d95-44c0-8378-802bf3cc3f61 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80378) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 10:28:19.674776 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec69d14d-ee06-47d9-93f1-c2c2a23ef224 admin admin] Validating token access rules against request {{(pid=80377) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:28:19.678539 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03fa8633-1e70-4812-9560-d0d3ba2e60f1 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '33b87170f0634b67bce60750847b5cbe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f1b38ebb804c41c69772c998aae72c41', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80377) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:28:19.679273 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03fa8633-1e70-4812-9560-d0d3ba2e60f1 service neutron] REQUEST_METHOD: `GET` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:28:19.679616 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03fa8633-1e70-4812-9560-d0d3ba2e60f1 service neutron] SCRIPT_NAME: `/identity` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:28:19.679952 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03fa8633-1e70-4812-9560-d0d3ba2e60f1 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:28:19.690620 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03fa8633-1e70-4812-9560-d0d3ba2e60f1 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:28:19.692045 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03fa8633-1e70-4812-9560-d0d3ba2e60f1 service neutron] RBAC: Authorization granted {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:28:19.701857 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: [pid: 80377|app: 0|req: 1388/2762] 10.222.0.82 () {60 vars in 1365 bytes} [Tue Aug 2 10:28:19 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 89 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 02 10:28:19.706638 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3caeaf4-1d95-44c0-8378-802bf3cc3f61 None None] Validating token access rules against request {{(pid=80378) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:28:19.710413 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7586010-4e58-41c0-9815-1a2112babe9d service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '33b87170f0634b67bce60750847b5cbe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f1b38ebb804c41c69772c998aae72c41', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80378) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:28:19.711036 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7586010-4e58-41c0-9815-1a2112babe9d service neutron] REQUEST_METHOD: `GET` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:28:19.711349 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7586010-4e58-41c0-9815-1a2112babe9d service neutron] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:28:19.711637 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7586010-4e58-41c0-9815-1a2112babe9d service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:28:19.713459 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.fernet_utils [None req-d7586010-4e58-41c0-9815-1a2112babe9d service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80378) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 10:28:19.723363 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7586010-4e58-41c0-9815-1a2112babe9d service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:28:19.724848 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7586010-4e58-41c0-9815-1a2112babe9d service neutron] RBAC: Authorization granted {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:28:19.735079 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 1375/2763] 10.222.0.82 () {60 vars in 1365 bytes} [Tue Aug 2 10:28:19 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 89 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 02 10:28:20.103916 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03fa8633-1e70-4812-9560-d0d3ba2e60f1 service neutron] Authenticating user token {{(pid=80377) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:28:20.105546 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.fernet_utils [None req-03fa8633-1e70-4812-9560-d0d3ba2e60f1 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80377) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 10:28:20.184624 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03fa8633-1e70-4812-9560-d0d3ba2e60f1 service neutron] Validating token access rules against request {{(pid=80377) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:28:20.189650 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8d110be-8192-4df1-96a5-4d8ddd4f661c service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ae6e910e50d74f29bfb89bf18d95a908', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f1b38ebb804c41c69772c998aae72c41', 'project_domain_id': 'default', 'roles': ['admin', 'service', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80377) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:28:20.190568 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8d110be-8192-4df1-96a5-4d8ddd4f661c service placement] REQUEST_METHOD: `GET` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:28:20.190846 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8d110be-8192-4df1-96a5-4d8ddd4f661c service placement] SCRIPT_NAME: `/identity` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:28:20.191114 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8d110be-8192-4df1-96a5-4d8ddd4f661c service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:28:20.192853 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.fernet_utils [None req-e8d110be-8192-4df1-96a5-4d8ddd4f661c service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80377) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 10:28:20.205589 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8d110be-8192-4df1-96a5-4d8ddd4f661c service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:28:20.207278 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8d110be-8192-4df1-96a5-4d8ddd4f661c service placement] RBAC: Authorization granted {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:28:20.218119 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: [pid: 80377|app: 0|req: 1389/2764] 10.222.0.82 () {60 vars in 1365 bytes} [Tue Aug 2 10:28:20 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 115 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 02 10:28:20.406882 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31432aad-9c11-42b8-b4ab-9c19e7865f1b None None] REQUEST_METHOD: `POST` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:28:20.408101 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31432aad-9c11-42b8-b4ab-9c19e7865f1b None None] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:28:20.408671 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31432aad-9c11-42b8-b4ab-9c19e7865f1b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:28:20.446690 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.auth.core [None req-31432aad-9c11-42b8-b4ab-9c19e7865f1b None None] MFA Rules not processed for user `28db30abecd2452e89be5b3b89cc39e7`. Rule list: `[]` (Enabled: `True`). {{(pid=80378) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 02 10:28:20.448460 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.fernet_utils [None req-31432aad-9c11-42b8-b4ab-9c19e7865f1b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80378) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 10:28:20.537022 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 1376/2765] 10.222.0.82 () {58 vars in 958 bytes} [Tue Aug 2 10:28:20 2022] POST /identity/v3/auth/tokens => generated 3258 bytes in 128 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 02 10:28:20.556364 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8d110be-8192-4df1-96a5-4d8ddd4f661c service placement] Authenticating user token {{(pid=80377) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:28:20.558591 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.fernet_utils [None req-e8d110be-8192-4df1-96a5-4d8ddd4f661c service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80377) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 10:28:20.590204 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG dbcounter [-] [80377] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80377) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 10:28:20.627062 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8d110be-8192-4df1-96a5-4d8ddd4f661c service placement] Validating token access rules against request {{(pid=80377) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:28:20.632735 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3688d214-38bd-4d36-8e1d-f55f391faaa6 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1023de281ae74e9d8a6d6c312c1c97a7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f1b38ebb804c41c69772c998aae72c41', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80377) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:28:20.633412 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3688d214-38bd-4d36-8e1d-f55f391faaa6 service cinder] REQUEST_METHOD: `GET` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:28:20.633659 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3688d214-38bd-4d36-8e1d-f55f391faaa6 service cinder] SCRIPT_NAME: `/identity` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:28:20.634746 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3688d214-38bd-4d36-8e1d-f55f391faaa6 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:28:20.644283 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3688d214-38bd-4d36-8e1d-f55f391faaa6 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:28:20.645827 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3688d214-38bd-4d36-8e1d-f55f391faaa6 service cinder] RBAC: Authorization granted {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:28:20.656711 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: [pid: 80377|app: 0|req: 1390/2766] 10.222.0.82 () {60 vars in 1365 bytes} [Tue Aug 2 10:28:20 2022] GET /identity/v3/auth/tokens => generated 3258 bytes in 103 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 02 10:28:22.041909 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae6f580d-f984-4998-810d-dc59b4f3b09f None None] REQUEST_METHOD: `POST` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:28:22.042474 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae6f580d-f984-4998-810d-dc59b4f3b09f None None] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:28:22.042474 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae6f580d-f984-4998-810d-dc59b4f3b09f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:28:22.062006 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.auth.core [None req-ae6f580d-f984-4998-810d-dc59b4f3b09f None None] MFA Rules not processed for user `28db30abecd2452e89be5b3b89cc39e7`. Rule list: `[]` (Enabled: `True`). {{(pid=80378) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 02 10:28:22.063233 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.fernet_utils [None req-ae6f580d-f984-4998-810d-dc59b4f3b09f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80378) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 10:28:22.070345 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 1377/2767] 10.222.0.82 () {58 vars in 958 bytes} [Tue Aug 2 10:28:22 2022] POST /identity/v3/auth/tokens => generated 3258 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 02 10:28:22.452522 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3688d214-38bd-4d36-8e1d-f55f391faaa6 service cinder] Authenticating user token {{(pid=80377) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:28:22.462117 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3688d214-38bd-4d36-8e1d-f55f391faaa6 service cinder] Validating token access rules against request {{(pid=80377) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:28:22.466349 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3bf4776a-3799-4ec1-b715-83cfd53dc66c service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '33b87170f0634b67bce60750847b5cbe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f1b38ebb804c41c69772c998aae72c41', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80377) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:28:22.466962 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bf4776a-3799-4ec1-b715-83cfd53dc66c service neutron] REQUEST_METHOD: `GET` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:28:22.467242 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bf4776a-3799-4ec1-b715-83cfd53dc66c service neutron] SCRIPT_NAME: `/identity` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:28:22.467512 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bf4776a-3799-4ec1-b715-83cfd53dc66c service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:28:22.470488 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.fernet_utils [None req-3bf4776a-3799-4ec1-b715-83cfd53dc66c service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80377) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 10:28:22.481798 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3bf4776a-3799-4ec1-b715-83cfd53dc66c service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:28:22.483784 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3bf4776a-3799-4ec1-b715-83cfd53dc66c service neutron] RBAC: Authorization granted {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:28:22.499593 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: [pid: 80377|app: 0|req: 1391/2768] 10.222.0.82 () {60 vars in 1365 bytes} [Tue Aug 2 10:28:22 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 48 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 02 10:28:28.729730 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae6f580d-f984-4998-810d-dc59b4f3b09f None None] Authenticating user token {{(pid=80378) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:28:28.731760 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.fernet_utils [None req-ae6f580d-f984-4998-810d-dc59b4f3b09f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80378) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 10:28:28.794492 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG dbcounter [-] [80378] Writing DB stats keystone:SELECT=100 {{(pid=80378) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 10:28:28.818095 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae6f580d-f984-4998-810d-dc59b4f3b09f None None] Validating token access rules against request {{(pid=80378) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:28:28.823558 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-315db38d-3731-4345-94c0-19e9f5aad842 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88af518196674fe593ad88d30111fb7d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f1b38ebb804c41c69772c998aae72c41', 'project_domain_id': 'default', 'roles': ['admin', 'ResellerAdmin', 'reader', 'service', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80378) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:28:28.824181 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-315db38d-3731-4345-94c0-19e9f5aad842 service nova] REQUEST_METHOD: `GET` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:28:28.824465 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-315db38d-3731-4345-94c0-19e9f5aad842 service nova] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:28:28.824742 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-315db38d-3731-4345-94c0-19e9f5aad842 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:28:28.826519 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.fernet_utils [None req-315db38d-3731-4345-94c0-19e9f5aad842 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80378) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 10:28:28.840477 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-315db38d-3731-4345-94c0-19e9f5aad842 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:28:28.842200 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-315db38d-3731-4345-94c0-19e9f5aad842 service nova] RBAC: Authorization granted {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:28:28.857577 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 1378/2769] 10.222.0.82 () {60 vars in 1365 bytes} [Tue Aug 2 10:28:28 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 130 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 02 10:28:32.493891 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG dbcounter [-] [80377] Writing DB stats keystone:SELECT=22 {{(pid=80377) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 10:28:38.857691 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG dbcounter [-] [80378] Writing DB stats keystone:SELECT=9 {{(pid=80378) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 10:29:01.048649 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3bf4776a-3799-4ec1-b715-83cfd53dc66c service neutron] Authenticating user token {{(pid=80377) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:29:01.057741 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3bf4776a-3799-4ec1-b715-83cfd53dc66c service neutron] Validating token access rules against request {{(pid=80377) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:29:01.061535 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d5b9ca9-fc32-4bf7-85c8-4e4cef516271 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1023de281ae74e9d8a6d6c312c1c97a7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f1b38ebb804c41c69772c998aae72c41', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80377) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:29:01.062223 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d5b9ca9-fc32-4bf7-85c8-4e4cef516271 service cinder] REQUEST_METHOD: `GET` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:29:01.062509 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d5b9ca9-fc32-4bf7-85c8-4e4cef516271 service cinder] SCRIPT_NAME: `/identity` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:29:01.062782 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d5b9ca9-fc32-4bf7-85c8-4e4cef516271 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:29:01.065517 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.fernet_utils [None req-2d5b9ca9-fc32-4bf7-85c8-4e4cef516271 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80377) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 10:29:01.126784 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d5b9ca9-fc32-4bf7-85c8-4e4cef516271 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:29:01.128147 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d5b9ca9-fc32-4bf7-85c8-4e4cef516271 service cinder] RBAC: Authorization granted {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:29:01.136950 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: [pid: 80377|app: 0|req: 1392/2770] 10.222.0.82 () {60 vars in 1365 bytes} [Tue Aug 2 10:29:01 2022] GET /identity/v3/auth/tokens => generated 3252 bytes in 90 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 02 10:29:11.132936 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG dbcounter [-] [80377] Writing DB stats keystone:SELECT=28 {{(pid=80377) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 10:29:13.239200 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-daab8393-9c86-4d18-9542-d50665f0dcb1 None None] REQUEST_METHOD: `POST` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:29:13.239747 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-daab8393-9c86-4d18-9542-d50665f0dcb1 None None] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:29:13.239895 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-daab8393-9c86-4d18-9542-d50665f0dcb1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:29:13.258721 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.auth.core [None req-daab8393-9c86-4d18-9542-d50665f0dcb1 None None] MFA Rules not processed for user `e08bc25f122445f592652671790cc658`. Rule list: `[]` (Enabled: `True`). {{(pid=80378) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 02 10:29:13.259914 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.fernet_utils [None req-daab8393-9c86-4d18-9542-d50665f0dcb1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80378) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 10:29:13.267519 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 1379/2771] 10.222.0.82 () {58 vars in 958 bytes} [Tue Aug 2 10:29:13 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 02 10:29:23.253466 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG dbcounter [-] [80378] Writing DB stats keystone:SELECT=3 {{(pid=80378) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 10:29:40.021151 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d5b9ca9-fc32-4bf7-85c8-4e4cef516271 service cinder] Authenticating user token {{(pid=80377) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:29:40.030605 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d5b9ca9-fc32-4bf7-85c8-4e4cef516271 service cinder] Validating token access rules against request {{(pid=80377) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:29:40.034125 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e9795ba-4db0-4613-b134-7ee29a855887 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '33b87170f0634b67bce60750847b5cbe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f1b38ebb804c41c69772c998aae72c41', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80377) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:29:40.034738 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e9795ba-4db0-4613-b134-7ee29a855887 service neutron] REQUEST_METHOD: `GET` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:29:40.035020 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e9795ba-4db0-4613-b134-7ee29a855887 service neutron] SCRIPT_NAME: `/identity` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:29:40.035291 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e9795ba-4db0-4613-b134-7ee29a855887 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:29:40.037059 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.fernet_utils [None req-4e9795ba-4db0-4613-b134-7ee29a855887 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80377) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 10:29:40.104003 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e9795ba-4db0-4613-b134-7ee29a855887 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:29:40.105646 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e9795ba-4db0-4613-b134-7ee29a855887 service neutron] RBAC: Authorization granted {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:29:40.116482 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: [pid: 80377|app: 0|req: 1393/2772] 10.222.0.82 () {60 vars in 1365 bytes} [Tue Aug 2 10:29:40 2022] GET /identity/v3/auth/tokens => generated 3321 bytes in 96 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 02 10:29:50.112062 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG dbcounter [-] [80377] Writing DB stats keystone:SELECT=30 {{(pid=80377) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 10:29:53.502071 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acb4cac6-7561-4a01-8e59-7a393d2f157c None None] REQUEST_METHOD: `POST` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:29:53.502573 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acb4cac6-7561-4a01-8e59-7a393d2f157c None None] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:29:53.502680 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acb4cac6-7561-4a01-8e59-7a393d2f157c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:29:53.521023 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.auth.core [None req-acb4cac6-7561-4a01-8e59-7a393d2f157c None None] MFA Rules not processed for user `28db30abecd2452e89be5b3b89cc39e7`. Rule list: `[]` (Enabled: `True`). {{(pid=80378) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 02 10:29:53.522242 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.fernet_utils [None req-acb4cac6-7561-4a01-8e59-7a393d2f157c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80378) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 10:29:53.529554 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 1380/2773] 10.222.0.82 () {58 vars in 958 bytes} [Tue Aug 2 10:29:53 2022] POST /identity/v3/auth/tokens => generated 3258 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 02 10:29:54.515286 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e9795ba-4db0-4613-b134-7ee29a855887 service neutron] Authenticating user token {{(pid=80377) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:29:54.516748 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.fernet_utils [None req-4e9795ba-4db0-4613-b134-7ee29a855887 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80377) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 10:29:54.530742 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e9795ba-4db0-4613-b134-7ee29a855887 service neutron] Validating token access rules against request {{(pid=80377) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:29:54.536120 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82160fa7-4fc5-4e20-b93e-64843dcb3e3a service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ae6e910e50d74f29bfb89bf18d95a908', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f1b38ebb804c41c69772c998aae72c41', 'project_domain_id': 'default', 'roles': ['admin', 'service', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80377) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:29:54.536727 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82160fa7-4fc5-4e20-b93e-64843dcb3e3a service placement] REQUEST_METHOD: `GET` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:29:54.537040 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82160fa7-4fc5-4e20-b93e-64843dcb3e3a service placement] SCRIPT_NAME: `/identity` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:29:54.537315 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82160fa7-4fc5-4e20-b93e-64843dcb3e3a service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:29:54.538987 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.fernet_utils [None req-82160fa7-4fc5-4e20-b93e-64843dcb3e3a service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80377) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 10:29:54.550509 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82160fa7-4fc5-4e20-b93e-64843dcb3e3a service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:29:54.552067 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82160fa7-4fc5-4e20-b93e-64843dcb3e3a service placement] RBAC: Authorization granted {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:29:54.561944 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: [pid: 80377|app: 0|req: 1394/2774] 10.222.0.82 () {60 vars in 1365 bytes} [Tue Aug 2 10:29:54 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 47 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 02 10:30:03.515902 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG dbcounter [-] [80378] Writing DB stats keystone:SELECT=3 {{(pid=80378) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 10:30:04.557644 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG dbcounter [-] [80377] Writing DB stats keystone:SELECT=6 {{(pid=80377) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 10:31:01.684584 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-acb4cac6-7561-4a01-8e59-7a393d2f157c None None] Authenticating user token {{(pid=80378) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:31:01.696525 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-acb4cac6-7561-4a01-8e59-7a393d2f157c None None] Validating token access rules against request {{(pid=80378) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:31:01.701987 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6bc225df-df9e-4fc4-b295-39aa5bc8842a service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ae6e910e50d74f29bfb89bf18d95a908', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f1b38ebb804c41c69772c998aae72c41', 'project_domain_id': 'default', 'roles': ['admin', 'service', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80378) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:31:01.702719 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bc225df-df9e-4fc4-b295-39aa5bc8842a service placement] REQUEST_METHOD: `GET` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:31:01.703006 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bc225df-df9e-4fc4-b295-39aa5bc8842a service placement] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:31:01.703277 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bc225df-df9e-4fc4-b295-39aa5bc8842a service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:31:01.705193 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.fernet_utils [None req-6bc225df-df9e-4fc4-b295-39aa5bc8842a service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80378) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 10:31:01.717613 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6bc225df-df9e-4fc4-b295-39aa5bc8842a service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:31:01.719221 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6bc225df-df9e-4fc4-b295-39aa5bc8842a service placement] RBAC: Authorization granted {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:31:01.730749 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 1381/2775] 10.222.0.82 () {60 vars in 1365 bytes} [Tue Aug 2 10:31:01 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 46 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 02 10:31:01.737668 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82160fa7-4fc5-4e20-b93e-64843dcb3e3a service placement] Authenticating user token {{(pid=80377) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:31:01.749920 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82160fa7-4fc5-4e20-b93e-64843dcb3e3a service placement] Validating token access rules against request {{(pid=80377) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:31:01.761213 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b408940-dc9a-4c54-9c97-abd244996354 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '891005fa5ff74041b75747d16cc66798', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7047912c5acd4adeba5d87c3bf895e39', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80377) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:31:01.761985 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b408940-dc9a-4c54-9c97-abd244996354 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:31:01.762275 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b408940-dc9a-4c54-9c97-abd244996354 admin admin] SCRIPT_NAME: `/identity` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:31:01.762523 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b408940-dc9a-4c54-9c97-abd244996354 admin admin] PATH_INFO: `/v3/users/28db30abecd2452e89be5b3b89cc39e7` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:31:01.765354 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b408940-dc9a-4c54-9c97-abd244996354 admin admin] RBAC: Authorizing `identity:delete_user(user_id=28db30abecd2452e89be5b3b89cc39e7)` {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:31:01.767294 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b408940-dc9a-4c54-9c97-abd244996354 admin admin] RBAC: Authorization granted {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:31:01.812490 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.notifications [None req-8b408940-dc9a-4c54-9c97-abd244996354 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '28db30abecd2452e89be5b3b89cc39e7'} {{(pid=80377) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 10:31:01.820325 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.notifications [None req-8b408940-dc9a-4c54-9c97-abd244996354 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '28db30abecd2452e89be5b3b89cc39e7'} {{(pid=80377) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 10:31:01.827499 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.notifications [None req-8b408940-dc9a-4c54-9c97-abd244996354 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '28db30abecd2452e89be5b3b89cc39e7'} {{(pid=80377) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 10:31:01.828128 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.notifications [None req-8b408940-dc9a-4c54-9c97-abd244996354 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '28db30abecd2452e89be5b3b89cc39e7'} {{(pid=80377) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 10:31:01.839468 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.notifications [None req-8b408940-dc9a-4c54-9c97-abd244996354 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '28db30abecd2452e89be5b3b89cc39e7'} {{(pid=80377) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 10:31:01.845129 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: [pid: 80377|app: 0|req: 1395/2776] 10.222.0.82 () {58 vars in 1224 bytes} [Tue Aug 2 10:31:01 2022] DELETE /identity/v3/users/28db30abecd2452e89be5b3b89cc39e7 => generated 0 bytes in 109 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 02 10:31:02.019391 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6bc225df-df9e-4fc4-b295-39aa5bc8842a service placement] Authenticating user token {{(pid=80378) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:31:02.031590 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6bc225df-df9e-4fc4-b295-39aa5bc8842a service placement] Validating token access rules against request {{(pid=80378) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:31:02.089084 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ccc732f5-cf23-455c-aae1-ee64acc64d20 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '891005fa5ff74041b75747d16cc66798', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7047912c5acd4adeba5d87c3bf895e39', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80378) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:31:02.089805 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccc732f5-cf23-455c-aae1-ee64acc64d20 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:31:02.090092 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccc732f5-cf23-455c-aae1-ee64acc64d20 admin admin] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:31:02.090367 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccc732f5-cf23-455c-aae1-ee64acc64d20 admin admin] PATH_INFO: `/v3/projects/f99abdc12f9d404498711c2eb6c3363c` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:31:02.091536 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ccc732f5-cf23-455c-aae1-ee64acc64d20 admin admin] RBAC: Authorizing `identity:delete_project(project_id=f99abdc12f9d404498711c2eb6c3363c)` {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:31:02.093421 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ccc732f5-cf23-455c-aae1-ee64acc64d20 admin admin] RBAC: Authorization granted {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:31:02.138293 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.notifications [None req-ccc732f5-cf23-455c-aae1-ee64acc64d20 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'f99abdc12f9d404498711c2eb6c3363c', 'request_id': 'req-ccc732f5-cf23-455c-aae1-ee64acc64d20'} {{(pid=80378) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 10:31:02.144310 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.notifications [None req-ccc732f5-cf23-455c-aae1-ee64acc64d20 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'f99abdc12f9d404498711c2eb6c3363c', 'request_id': 'req-ccc732f5-cf23-455c-aae1-ee64acc64d20'} {{(pid=80378) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 10:31:02.149667 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.notifications [None req-ccc732f5-cf23-455c-aae1-ee64acc64d20 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'f99abdc12f9d404498711c2eb6c3363c', 'request_id': 'req-ccc732f5-cf23-455c-aae1-ee64acc64d20'} {{(pid=80378) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 10:31:02.162167 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.notifications [None req-ccc732f5-cf23-455c-aae1-ee64acc64d20 admin admin] The token cache is being invalidate because project f99abdc12f9d404498711c2eb6c3363c was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80378) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 02 10:31:02.162453 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.notifications [None req-ccc732f5-cf23-455c-aae1-ee64acc64d20 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80378) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 10:31:02.163076 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.notifications [None req-ccc732f5-cf23-455c-aae1-ee64acc64d20 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80378) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 10:31:02.164716 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 1382/2777] 10.222.0.82 () {58 vars in 1233 bytes} [Tue Aug 2 10:31:02 2022] DELETE /identity/v3/projects/f99abdc12f9d404498711c2eb6c3363c => generated 0 bytes in 146 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 02 10:31:02.389929 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b408940-dc9a-4c54-9c97-abd244996354 admin admin] Authenticating user token {{(pid=80377) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:31:02.391625 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.fernet_utils [None req-8b408940-dc9a-4c54-9c97-abd244996354 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80377) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 10:31:02.467717 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b408940-dc9a-4c54-9c97-abd244996354 admin admin] Validating token access rules against request {{(pid=80377) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:31:02.530446 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-655524cd-c8f7-4399-b436-1e840e99d627 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88af518196674fe593ad88d30111fb7d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f1b38ebb804c41c69772c998aae72c41', 'project_domain_id': 'default', 'roles': ['admin', 'ResellerAdmin', 'reader', 'service', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80377) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:31:02.531056 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-655524cd-c8f7-4399-b436-1e840e99d627 service nova] REQUEST_METHOD: `GET` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:31:02.531334 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-655524cd-c8f7-4399-b436-1e840e99d627 service nova] SCRIPT_NAME: `/identity` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:31:02.531606 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-655524cd-c8f7-4399-b436-1e840e99d627 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:31:02.533422 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.fernet_utils [None req-655524cd-c8f7-4399-b436-1e840e99d627 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80377) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 10:31:02.542162 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG dbcounter [-] [80377] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80377) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 10:31:02.595192 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-655524cd-c8f7-4399-b436-1e840e99d627 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:31:02.596778 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-655524cd-c8f7-4399-b436-1e840e99d627 service nova] RBAC: Authorization granted {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:31:02.606232 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: [pid: 80377|app: 0|req: 1396/2778] 10.222.0.82 () {60 vars in 1365 bytes} [Tue Aug 2 10:31:02 2022] GET /identity/v3/auth/tokens => generated 3262 bytes in 217 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 02 10:31:04.024346 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ccc732f5-cf23-455c-aae1-ee64acc64d20 admin admin] Authenticating user token {{(pid=80378) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:31:04.025742 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.fernet_utils [None req-ccc732f5-cf23-455c-aae1-ee64acc64d20 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80378) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 10:31:04.081750 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ccc732f5-cf23-455c-aae1-ee64acc64d20 admin admin] Validating token access rules against request {{(pid=80378) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:31:04.085926 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a87de0c2-7b51-481b-8412-800e0dc0996b service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1023de281ae74e9d8a6d6c312c1c97a7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f1b38ebb804c41c69772c998aae72c41', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80378) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:31:04.086541 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a87de0c2-7b51-481b-8412-800e0dc0996b service cinder] REQUEST_METHOD: `GET` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:31:04.086825 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a87de0c2-7b51-481b-8412-800e0dc0996b service cinder] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:31:04.087103 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a87de0c2-7b51-481b-8412-800e0dc0996b service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:31:04.088823 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.fernet_utils [None req-a87de0c2-7b51-481b-8412-800e0dc0996b service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80378) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 10:31:04.102451 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a87de0c2-7b51-481b-8412-800e0dc0996b service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:31:04.103809 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a87de0c2-7b51-481b-8412-800e0dc0996b service cinder] RBAC: Authorization granted {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:31:04.114615 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 1383/2779] 10.222.0.82 () {60 vars in 1365 bytes} [Tue Aug 2 10:31:04 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 91 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 02 10:31:11.900542 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-655524cd-c8f7-4399-b436-1e840e99d627 service nova] Authenticating user token {{(pid=80377) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:31:11.912496 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-655524cd-c8f7-4399-b436-1e840e99d627 service nova] Validating token access rules against request {{(pid=80377) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:31:11.916011 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8706e6b-db4c-4aa1-af4d-37351b69dda8 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1023de281ae74e9d8a6d6c312c1c97a7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f1b38ebb804c41c69772c998aae72c41', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80377) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:31:11.916625 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8706e6b-db4c-4aa1-af4d-37351b69dda8 service cinder] REQUEST_METHOD: `GET` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:31:11.916904 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8706e6b-db4c-4aa1-af4d-37351b69dda8 service cinder] SCRIPT_NAME: `/identity` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:31:11.917178 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8706e6b-db4c-4aa1-af4d-37351b69dda8 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:31:11.918959 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.fernet_utils [None req-f8706e6b-db4c-4aa1-af4d-37351b69dda8 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80377) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 10:31:11.932886 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8706e6b-db4c-4aa1-af4d-37351b69dda8 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:31:11.934353 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8706e6b-db4c-4aa1-af4d-37351b69dda8 service cinder] RBAC: Authorization granted {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:31:11.945367 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: [pid: 80377|app: 0|req: 1397/2780] 10.222.0.82 () {60 vars in 1365 bytes} [Tue Aug 2 10:31:11 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 45 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 02 10:31:14.109713 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG dbcounter [-] [80378] Writing DB stats keystone:SELECT=83,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80378) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 10:31:21.132963 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32205d47-0d2b-4ed9-8e14-9d8c17338de2 None None] REQUEST_METHOD: `POST` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:31:21.133609 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32205d47-0d2b-4ed9-8e14-9d8c17338de2 None None] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:31:21.133609 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32205d47-0d2b-4ed9-8e14-9d8c17338de2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:31:21.152511 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.auth.core [None req-32205d47-0d2b-4ed9-8e14-9d8c17338de2 None None] MFA Rules not processed for user `8774306d95c0476b80cc8e53b62e8a4b`. Rule list: `[]` (Enabled: `True`). {{(pid=80378) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 02 10:31:21.153698 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.fernet_utils [None req-32205d47-0d2b-4ed9-8e14-9d8c17338de2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80378) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 10:31:21.160447 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 1384/2781] 10.222.0.82 () {58 vars in 958 bytes} [Tue Aug 2 10:31:21 2022] POST /identity/v3/auth/tokens => generated 3262 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 02 10:31:21.940592 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG dbcounter [-] [80377] Writing DB stats keystone:SELECT=30 {{(pid=80377) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 10:31:31.147945 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG dbcounter [-] [80378] Writing DB stats keystone:SELECT=3 {{(pid=80378) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 10:32:12.402574 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8706e6b-db4c-4aa1-af4d-37351b69dda8 service cinder] Authenticating user token {{(pid=80377) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:32:12.404005 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.fernet_utils [None req-f8706e6b-db4c-4aa1-af4d-37351b69dda8 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80377) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 10:32:12.460693 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8706e6b-db4c-4aa1-af4d-37351b69dda8 service cinder] Validating token access rules against request {{(pid=80377) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:32:12.464232 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63761327-8a64-43ec-bf24-06ddf7390cab service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '33b87170f0634b67bce60750847b5cbe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f1b38ebb804c41c69772c998aae72c41', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80377) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:32:12.464846 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63761327-8a64-43ec-bf24-06ddf7390cab service neutron] REQUEST_METHOD: `GET` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:32:12.465180 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63761327-8a64-43ec-bf24-06ddf7390cab service neutron] SCRIPT_NAME: `/identity` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:32:12.465435 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63761327-8a64-43ec-bf24-06ddf7390cab service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:32:12.467211 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.fernet_utils [None req-63761327-8a64-43ec-bf24-06ddf7390cab service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80377) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 10:32:12.532350 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63761327-8a64-43ec-bf24-06ddf7390cab service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:32:12.533823 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63761327-8a64-43ec-bf24-06ddf7390cab service neutron] RBAC: Authorization granted {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:32:12.543538 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: [pid: 80377|app: 0|req: 1398/2782] 10.222.0.82 () {60 vars in 1365 bytes} [Tue Aug 2 10:32:12 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 142 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 02 10:32:15.834137 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32205d47-0d2b-4ed9-8e14-9d8c17338de2 None None] Authenticating user token {{(pid=80378) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:32:15.835694 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.fernet_utils [None req-32205d47-0d2b-4ed9-8e14-9d8c17338de2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80378) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 10:32:15.953751 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32205d47-0d2b-4ed9-8e14-9d8c17338de2 None None] Validating token access rules against request {{(pid=80378) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:32:15.958903 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d2b481c-3da6-43b3-8344-1b2a4fe65266 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ae6e910e50d74f29bfb89bf18d95a908', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f1b38ebb804c41c69772c998aae72c41', 'project_domain_id': 'default', 'roles': ['admin', 'service', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80378) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:32:15.959590 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d2b481c-3da6-43b3-8344-1b2a4fe65266 service placement] REQUEST_METHOD: `GET` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:32:15.959937 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d2b481c-3da6-43b3-8344-1b2a4fe65266 service placement] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:32:15.960276 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d2b481c-3da6-43b3-8344-1b2a4fe65266 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:32:15.962184 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.fernet_utils [None req-4d2b481c-3da6-43b3-8344-1b2a4fe65266 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80378) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 10:32:15.975935 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d2b481c-3da6-43b3-8344-1b2a4fe65266 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:32:15.977617 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d2b481c-3da6-43b3-8344-1b2a4fe65266 service placement] RBAC: Authorization granted {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:32:15.988228 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 1385/2783] 10.222.0.82 () {60 vars in 1365 bytes} [Tue Aug 2 10:32:15 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 154 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 02 10:32:18.786937 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63761327-8a64-43ec-bf24-06ddf7390cab service neutron] Authenticating user token {{(pid=80377) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:32:18.788444 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.fernet_utils [None req-63761327-8a64-43ec-bf24-06ddf7390cab service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80377) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 10:32:18.805979 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63761327-8a64-43ec-bf24-06ddf7390cab service neutron] Validating token access rules against request {{(pid=80377) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:32:18.812178 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b93bafcc-27cf-4d66-b8d3-6ec7cfdc900f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '891005fa5ff74041b75747d16cc66798', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7047912c5acd4adeba5d87c3bf895e39', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80377) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:32:18.813274 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b93bafcc-27cf-4d66-b8d3-6ec7cfdc900f admin admin] REQUEST_METHOD: `DELETE` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:32:18.813580 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b93bafcc-27cf-4d66-b8d3-6ec7cfdc900f admin admin] SCRIPT_NAME: `/identity` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:32:18.813893 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b93bafcc-27cf-4d66-b8d3-6ec7cfdc900f admin admin] PATH_INFO: `/v3/users/e08bc25f122445f592652671790cc658` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:32:18.815528 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b93bafcc-27cf-4d66-b8d3-6ec7cfdc900f admin admin] RBAC: Authorizing `identity:delete_user(user_id=e08bc25f122445f592652671790cc658)` {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:32:18.817485 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b93bafcc-27cf-4d66-b8d3-6ec7cfdc900f admin admin] RBAC: Authorization granted {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:32:18.881930 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.notifications [None req-b93bafcc-27cf-4d66-b8d3-6ec7cfdc900f admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'e08bc25f122445f592652671790cc658'} {{(pid=80377) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 10:32:18.889347 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.notifications [None req-b93bafcc-27cf-4d66-b8d3-6ec7cfdc900f admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'e08bc25f122445f592652671790cc658'} {{(pid=80377) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 10:32:18.896398 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.notifications [None req-b93bafcc-27cf-4d66-b8d3-6ec7cfdc900f admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'e08bc25f122445f592652671790cc658'} {{(pid=80377) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 10:32:18.897096 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.notifications [None req-b93bafcc-27cf-4d66-b8d3-6ec7cfdc900f admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'e08bc25f122445f592652671790cc658'} {{(pid=80377) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 10:32:18.908741 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.notifications [None req-b93bafcc-27cf-4d66-b8d3-6ec7cfdc900f admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'e08bc25f122445f592652671790cc658'} {{(pid=80377) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 10:32:18.914580 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: [pid: 80377|app: 0|req: 1399/2784] 10.222.0.82 () {58 vars in 1224 bytes} [Tue Aug 2 10:32:18 2022] DELETE /identity/v3/users/e08bc25f122445f592652671790cc658 => generated 0 bytes in 128 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 02 10:32:19.092407 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d2b481c-3da6-43b3-8344-1b2a4fe65266 service placement] Authenticating user token {{(pid=80378) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:32:19.106093 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d2b481c-3da6-43b3-8344-1b2a4fe65266 service placement] Validating token access rules against request {{(pid=80378) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:32:19.163304 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-878e31ec-0db0-4a17-a9bc-06f17cb0c46a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '891005fa5ff74041b75747d16cc66798', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7047912c5acd4adeba5d87c3bf895e39', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80378) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:32:19.163947 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-878e31ec-0db0-4a17-a9bc-06f17cb0c46a admin admin] REQUEST_METHOD: `DELETE` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:32:19.164231 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-878e31ec-0db0-4a17-a9bc-06f17cb0c46a admin admin] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:32:19.164514 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-878e31ec-0db0-4a17-a9bc-06f17cb0c46a admin admin] PATH_INFO: `/v3/projects/06660bbc73344c53b5a0337808514df7` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:32:19.165662 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-878e31ec-0db0-4a17-a9bc-06f17cb0c46a admin admin] RBAC: Authorizing `identity:delete_project(project_id=06660bbc73344c53b5a0337808514df7)` {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:32:19.168212 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-878e31ec-0db0-4a17-a9bc-06f17cb0c46a admin admin] RBAC: Authorization granted {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:32:19.212175 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.notifications [None req-878e31ec-0db0-4a17-a9bc-06f17cb0c46a admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '06660bbc73344c53b5a0337808514df7', 'request_id': 'req-878e31ec-0db0-4a17-a9bc-06f17cb0c46a'} {{(pid=80378) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 10:32:19.218607 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.notifications [None req-878e31ec-0db0-4a17-a9bc-06f17cb0c46a admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '06660bbc73344c53b5a0337808514df7', 'request_id': 'req-878e31ec-0db0-4a17-a9bc-06f17cb0c46a'} {{(pid=80378) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 10:32:19.224045 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.notifications [None req-878e31ec-0db0-4a17-a9bc-06f17cb0c46a admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '06660bbc73344c53b5a0337808514df7', 'request_id': 'req-878e31ec-0db0-4a17-a9bc-06f17cb0c46a'} {{(pid=80378) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 10:32:19.236463 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.notifications [None req-878e31ec-0db0-4a17-a9bc-06f17cb0c46a admin admin] The token cache is being invalidate because project 06660bbc73344c53b5a0337808514df7 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80378) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 02 10:32:19.236754 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.notifications [None req-878e31ec-0db0-4a17-a9bc-06f17cb0c46a admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80378) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 10:32:19.237377 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.notifications [None req-878e31ec-0db0-4a17-a9bc-06f17cb0c46a admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80378) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 10:32:19.238926 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 1386/2785] 10.222.0.82 () {58 vars in 1233 bytes} [Tue Aug 2 10:32:19 2022] DELETE /identity/v3/projects/06660bbc73344c53b5a0337808514df7 => generated 0 bytes in 147 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 02 10:32:19.242975 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b93bafcc-27cf-4d66-b8d3-6ec7cfdc900f admin admin] Authenticating user token {{(pid=80377) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:32:19.244367 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.fernet_utils [None req-b93bafcc-27cf-4d66-b8d3-6ec7cfdc900f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80377) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 10:32:19.257496 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b93bafcc-27cf-4d66-b8d3-6ec7cfdc900f admin admin] Validating token access rules against request {{(pid=80377) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:32:19.275547 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG dbcounter [-] [80377] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80377) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 10:32:19.316947 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30216dcf-b097-4e75-acbc-36d58a7e14fd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '891005fa5ff74041b75747d16cc66798', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7047912c5acd4adeba5d87c3bf895e39', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80377) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:32:19.317513 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30216dcf-b097-4e75-acbc-36d58a7e14fd admin admin] REQUEST_METHOD: `DELETE` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:32:19.317816 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30216dcf-b097-4e75-acbc-36d58a7e14fd admin admin] SCRIPT_NAME: `/identity` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:32:19.318144 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30216dcf-b097-4e75-acbc-36d58a7e14fd admin admin] PATH_INFO: `/v3/users/1b65b5367f034856b93e957ce0897170` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:32:19.319302 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30216dcf-b097-4e75-acbc-36d58a7e14fd admin admin] RBAC: Authorizing `identity:delete_user(user_id=1b65b5367f034856b93e957ce0897170)` {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:32:19.321201 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30216dcf-b097-4e75-acbc-36d58a7e14fd admin admin] RBAC: Authorization granted {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:32:19.367483 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.notifications [None req-30216dcf-b097-4e75-acbc-36d58a7e14fd admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '1b65b5367f034856b93e957ce0897170'} {{(pid=80377) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 10:32:19.374806 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.notifications [None req-30216dcf-b097-4e75-acbc-36d58a7e14fd admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '1b65b5367f034856b93e957ce0897170'} {{(pid=80377) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 10:32:19.381543 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.notifications [None req-30216dcf-b097-4e75-acbc-36d58a7e14fd admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '1b65b5367f034856b93e957ce0897170'} {{(pid=80377) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 10:32:19.382200 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.notifications [None req-30216dcf-b097-4e75-acbc-36d58a7e14fd admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '1b65b5367f034856b93e957ce0897170'} {{(pid=80377) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 10:32:19.393742 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.notifications [None req-30216dcf-b097-4e75-acbc-36d58a7e14fd admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '1b65b5367f034856b93e957ce0897170'} {{(pid=80377) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 10:32:19.399646 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: [pid: 80377|app: 0|req: 1400/2786] 10.222.0.82 () {58 vars in 1224 bytes} [Tue Aug 2 10:32:19 2022] DELETE /identity/v3/users/1b65b5367f034856b93e957ce0897170 => generated 0 bytes in 157 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 02 10:32:19.569038 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-878e31ec-0db0-4a17-a9bc-06f17cb0c46a admin admin] Authenticating user token {{(pid=80378) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:32:19.570492 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.fernet_utils [None req-878e31ec-0db0-4a17-a9bc-06f17cb0c46a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80378) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 10:32:19.584471 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-878e31ec-0db0-4a17-a9bc-06f17cb0c46a admin admin] Validating token access rules against request {{(pid=80378) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:32:19.604333 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG dbcounter [-] [80378] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80378) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 10:32:19.655864 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a9479e9-77ce-4f74-bacb-dcd116375649 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '891005fa5ff74041b75747d16cc66798', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7047912c5acd4adeba5d87c3bf895e39', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80378) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:32:19.656509 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a9479e9-77ce-4f74-bacb-dcd116375649 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:32:19.656791 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a9479e9-77ce-4f74-bacb-dcd116375649 admin admin] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:32:19.657071 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a9479e9-77ce-4f74-bacb-dcd116375649 admin admin] PATH_INFO: `/v3/projects/d79d77dd089c4b3895fa177c86127bd7` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:32:19.658266 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a9479e9-77ce-4f74-bacb-dcd116375649 admin admin] RBAC: Authorizing `identity:delete_project(project_id=d79d77dd089c4b3895fa177c86127bd7)` {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:32:19.660175 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a9479e9-77ce-4f74-bacb-dcd116375649 admin admin] RBAC: Authorization granted {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:32:19.706944 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.notifications [None req-0a9479e9-77ce-4f74-bacb-dcd116375649 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'd79d77dd089c4b3895fa177c86127bd7', 'request_id': 'req-0a9479e9-77ce-4f74-bacb-dcd116375649'} {{(pid=80378) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 10:32:19.713060 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.notifications [None req-0a9479e9-77ce-4f74-bacb-dcd116375649 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'd79d77dd089c4b3895fa177c86127bd7', 'request_id': 'req-0a9479e9-77ce-4f74-bacb-dcd116375649'} {{(pid=80378) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 10:32:19.719312 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.notifications [None req-0a9479e9-77ce-4f74-bacb-dcd116375649 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'd79d77dd089c4b3895fa177c86127bd7', 'request_id': 'req-0a9479e9-77ce-4f74-bacb-dcd116375649'} {{(pid=80378) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 10:32:19.731105 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.notifications [None req-0a9479e9-77ce-4f74-bacb-dcd116375649 admin admin] The token cache is being invalidate because project d79d77dd089c4b3895fa177c86127bd7 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80378) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 02 10:32:19.731398 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.notifications [None req-0a9479e9-77ce-4f74-bacb-dcd116375649 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80378) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 10:32:19.732046 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.notifications [None req-0a9479e9-77ce-4f74-bacb-dcd116375649 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80378) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 10:32:19.733902 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 1387/2787] 10.222.0.82 () {58 vars in 1233 bytes} [Tue Aug 2 10:32:19 2022] DELETE /identity/v3/projects/d79d77dd089c4b3895fa177c86127bd7 => generated 0 bytes in 165 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 02 10:32:29.396157 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG dbcounter [-] [80377] Writing DB stats keystone:SELECT=43,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80377) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 10:32:29.727919 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG dbcounter [-] [80378] Writing DB stats keystone:SELECT=42,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80378) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 10:33:08.682817 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30216dcf-b097-4e75-acbc-36d58a7e14fd admin admin] Authenticating user token {{(pid=80377) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:33:08.684313 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.fernet_utils [None req-30216dcf-b097-4e75-acbc-36d58a7e14fd admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80377) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 10:33:08.742685 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30216dcf-b097-4e75-acbc-36d58a7e14fd admin admin] Validating token access rules against request {{(pid=80377) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:33:08.788612 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-038ed982-6ccb-473c-a349-c91d9bd65d30 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '33b87170f0634b67bce60750847b5cbe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f1b38ebb804c41c69772c998aae72c41', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80377) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:33:08.789230 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-038ed982-6ccb-473c-a349-c91d9bd65d30 service neutron] REQUEST_METHOD: `GET` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:33:08.789511 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-038ed982-6ccb-473c-a349-c91d9bd65d30 service neutron] SCRIPT_NAME: `/identity` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:33:08.789804 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-038ed982-6ccb-473c-a349-c91d9bd65d30 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:33:08.791552 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.fernet_utils [None req-038ed982-6ccb-473c-a349-c91d9bd65d30 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80377) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 10:33:08.853516 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-038ed982-6ccb-473c-a349-c91d9bd65d30 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:33:08.855026 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-038ed982-6ccb-473c-a349-c91d9bd65d30 service neutron] RBAC: Authorization granted {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:33:08.865415 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: [pid: 80377|app: 0|req: 1401/2788] 10.222.0.82 () {60 vars in 1365 bytes} [Tue Aug 2 10:33:08 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 183 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 02 10:33:11.947103 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a9479e9-77ce-4f74-bacb-dcd116375649 admin admin] Authenticating user token {{(pid=80378) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:33:11.948592 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.fernet_utils [None req-0a9479e9-77ce-4f74-bacb-dcd116375649 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80378) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 10:33:11.962563 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a9479e9-77ce-4f74-bacb-dcd116375649 admin admin] Validating token access rules against request {{(pid=80378) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:33:11.967337 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1771bbb-3495-4d2e-891b-9616c2ad04e3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '891005fa5ff74041b75747d16cc66798', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7047912c5acd4adeba5d87c3bf895e39', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80378) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:33:11.967984 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1771bbb-3495-4d2e-891b-9616c2ad04e3 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:33:11.968266 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1771bbb-3495-4d2e-891b-9616c2ad04e3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:33:11.968613 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1771bbb-3495-4d2e-891b-9616c2ad04e3 admin admin] PATH_INFO: `/v3/users/8774306d95c0476b80cc8e53b62e8a4b` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:33:11.969812 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1771bbb-3495-4d2e-891b-9616c2ad04e3 admin admin] RBAC: Authorizing `identity:delete_user(user_id=8774306d95c0476b80cc8e53b62e8a4b)` {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:33:11.971752 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1771bbb-3495-4d2e-891b-9616c2ad04e3 admin admin] RBAC: Authorization granted {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:33:12.020420 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.notifications [None req-b1771bbb-3495-4d2e-891b-9616c2ad04e3 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '8774306d95c0476b80cc8e53b62e8a4b'} {{(pid=80378) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 10:33:12.027815 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.notifications [None req-b1771bbb-3495-4d2e-891b-9616c2ad04e3 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '8774306d95c0476b80cc8e53b62e8a4b'} {{(pid=80378) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 10:33:12.035011 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.notifications [None req-b1771bbb-3495-4d2e-891b-9616c2ad04e3 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '8774306d95c0476b80cc8e53b62e8a4b'} {{(pid=80378) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 10:33:12.035618 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.notifications [None req-b1771bbb-3495-4d2e-891b-9616c2ad04e3 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '8774306d95c0476b80cc8e53b62e8a4b'} {{(pid=80378) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 10:33:12.047631 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.notifications [None req-b1771bbb-3495-4d2e-891b-9616c2ad04e3 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '8774306d95c0476b80cc8e53b62e8a4b'} {{(pid=80378) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 10:33:12.053352 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 1388/2789] 10.222.0.82 () {58 vars in 1224 bytes} [Tue Aug 2 10:33:11 2022] DELETE /identity/v3/users/8774306d95c0476b80cc8e53b62e8a4b => generated 0 bytes in 107 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 02 10:33:12.223913 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-038ed982-6ccb-473c-a349-c91d9bd65d30 service neutron] Authenticating user token {{(pid=80377) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:33:12.240192 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-038ed982-6ccb-473c-a349-c91d9bd65d30 service neutron] Validating token access rules against request {{(pid=80377) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:33:12.297574 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ad87a35-a706-42be-aae5-2837da8a8644 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '891005fa5ff74041b75747d16cc66798', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7047912c5acd4adeba5d87c3bf895e39', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80377) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:33:12.298215 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ad87a35-a706-42be-aae5-2837da8a8644 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:33:12.298540 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ad87a35-a706-42be-aae5-2837da8a8644 admin admin] SCRIPT_NAME: `/identity` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:33:12.299120 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ad87a35-a706-42be-aae5-2837da8a8644 admin admin] PATH_INFO: `/v3/projects/9389c81a2710466ea8be45d2fe2342ef` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:33:12.300370 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ad87a35-a706-42be-aae5-2837da8a8644 admin admin] RBAC: Authorizing `identity:delete_project(project_id=9389c81a2710466ea8be45d2fe2342ef)` {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:33:12.302310 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ad87a35-a706-42be-aae5-2837da8a8644 admin admin] RBAC: Authorization granted {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:33:12.310144 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG dbcounter [-] [80377] Writing DB stats keystone:SELECT=100 {{(pid=80377) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 10:33:12.348865 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.notifications [None req-3ad87a35-a706-42be-aae5-2837da8a8644 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '9389c81a2710466ea8be45d2fe2342ef', 'request_id': 'req-3ad87a35-a706-42be-aae5-2837da8a8644'} {{(pid=80377) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 10:33:12.354438 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.notifications [None req-3ad87a35-a706-42be-aae5-2837da8a8644 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '9389c81a2710466ea8be45d2fe2342ef', 'request_id': 'req-3ad87a35-a706-42be-aae5-2837da8a8644'} {{(pid=80377) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 10:33:12.360357 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.notifications [None req-3ad87a35-a706-42be-aae5-2837da8a8644 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '9389c81a2710466ea8be45d2fe2342ef', 'request_id': 'req-3ad87a35-a706-42be-aae5-2837da8a8644'} {{(pid=80377) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 10:33:12.372353 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.notifications [None req-3ad87a35-a706-42be-aae5-2837da8a8644 admin admin] The token cache is being invalidate because project 9389c81a2710466ea8be45d2fe2342ef was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80377) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 02 10:33:12.372647 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.notifications [None req-3ad87a35-a706-42be-aae5-2837da8a8644 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80377) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 10:33:12.373862 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.notifications [None req-3ad87a35-a706-42be-aae5-2837da8a8644 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80377) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 10:33:12.375311 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: [pid: 80377|app: 0|req: 1402/2790] 10.222.0.82 () {58 vars in 1233 bytes} [Tue Aug 2 10:33:12 2022] DELETE /identity/v3/projects/9389c81a2710466ea8be45d2fe2342ef => generated 0 bytes in 152 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 02 10:33:12.380257 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63f6302b-3431-4b28-9cdc-8de6aa7be0d1 None None] REQUEST_METHOD: `POST` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:33:12.380548 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63f6302b-3431-4b28-9cdc-8de6aa7be0d1 None None] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:33:12.380841 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63f6302b-3431-4b28-9cdc-8de6aa7be0d1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:33:12.400809 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.auth.core [None req-63f6302b-3431-4b28-9cdc-8de6aa7be0d1 None None] MFA Rules not processed for user `891005fa5ff74041b75747d16cc66798`. Rule list: `[]` (Enabled: `True`). {{(pid=80378) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 02 10:33:12.402034 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.fernet_utils [None req-63f6302b-3431-4b28-9cdc-8de6aa7be0d1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80378) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 10:33:12.463796 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 1389/2791] 10.222.0.82 () {58 vars in 958 bytes} [Tue Aug 2 10:33:12 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 02 10:33:12.472646 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-981c792f-f0ec-4ebf-bf9a-9ad8c63ffb90 None None] REQUEST_METHOD: `POST` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:33:12.472932 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-981c792f-f0ec-4ebf-bf9a-9ad8c63ffb90 None None] SCRIPT_NAME: `/identity` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:33:12.473211 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-981c792f-f0ec-4ebf-bf9a-9ad8c63ffb90 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:33:12.492342 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.auth.core [None req-981c792f-f0ec-4ebf-bf9a-9ad8c63ffb90 None None] MFA Rules not processed for user `891005fa5ff74041b75747d16cc66798`. Rule list: `[]` (Enabled: `True`). {{(pid=80377) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 02 10:33:12.493532 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.fernet_utils [None req-981c792f-f0ec-4ebf-bf9a-9ad8c63ffb90 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80377) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 10:33:12.501969 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: [pid: 80377|app: 0|req: 1403/2792] 10.222.0.82 () {58 vars in 958 bytes} [Tue Aug 2 10:33:12 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 02 10:33:12.506138 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63f6302b-3431-4b28-9cdc-8de6aa7be0d1 None None] Authenticating user token {{(pid=80378) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:33:12.516367 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63f6302b-3431-4b28-9cdc-8de6aa7be0d1 None None] Validating token access rules against request {{(pid=80378) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:33:12.522298 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36f39458-98ba-4cd4-ae49-97bd1da80c19 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '891005fa5ff74041b75747d16cc66798', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7047912c5acd4adeba5d87c3bf895e39', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80378) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:33:12.523332 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36f39458-98ba-4cd4-ae49-97bd1da80c19 admin admin] REQUEST_METHOD: `GET` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:33:12.523839 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36f39458-98ba-4cd4-ae49-97bd1da80c19 admin admin] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:33:12.524340 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36f39458-98ba-4cd4-ae49-97bd1da80c19 admin admin] PATH_INFO: `/v3/domains` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:33:12.525625 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36f39458-98ba-4cd4-ae49-97bd1da80c19 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80378) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 02 10:33:12.526233 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36f39458-98ba-4cd4-ae49-97bd1da80c19 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:33:12.529248 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36f39458-98ba-4cd4-ae49-97bd1da80c19 admin admin] RBAC: Authorization granted {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:33:12.546327 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 1390/2793] 10.222.0.82 () {58 vars in 1153 bytes} [Tue Aug 2 10:33:12 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 41 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 02 10:33:12.550737 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-981c792f-f0ec-4ebf-bf9a-9ad8c63ffb90 None None] Authenticating user token {{(pid=80377) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:33:12.559636 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-981c792f-f0ec-4ebf-bf9a-9ad8c63ffb90 None None] Validating token access rules against request {{(pid=80377) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:33:12.564740 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-534670b6-5cfc-4736-8a4c-156deef20c83 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '891005fa5ff74041b75747d16cc66798', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7047912c5acd4adeba5d87c3bf895e39', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80377) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:33:12.565587 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-534670b6-5cfc-4736-8a4c-156deef20c83 admin admin] REQUEST_METHOD: `POST` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:33:12.565907 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-534670b6-5cfc-4736-8a4c-156deef20c83 admin admin] SCRIPT_NAME: `/identity` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:33:12.566180 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-534670b6-5cfc-4736-8a4c-156deef20c83 admin admin] PATH_INFO: `/v3/projects` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:33:12.566957 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-534670b6-5cfc-4736-8a4c-156deef20c83 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:33:12.568831 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-534670b6-5cfc-4736-8a4c-156deef20c83 admin admin] RBAC: Authorization granted {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:33:12.595667 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: [pid: 80377|app: 0|req: 1404/2794] 10.222.0.82 () {60 vars in 1153 bytes} [Tue Aug 2 10:33:12 2022] POST /identity/v3/projects => generated 373 bytes in 45 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 02 10:33:12.599776 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36f39458-98ba-4cd4-ae49-97bd1da80c19 admin admin] Authenticating user token {{(pid=80378) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:33:12.609566 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36f39458-98ba-4cd4-ae49-97bd1da80c19 admin admin] Validating token access rules against request {{(pid=80378) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:33:12.665461 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75785f94-ed53-418c-81e2-117c30ac13d5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '891005fa5ff74041b75747d16cc66798', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7047912c5acd4adeba5d87c3bf895e39', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80378) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:33:12.666108 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75785f94-ed53-418c-81e2-117c30ac13d5 admin admin] REQUEST_METHOD: `POST` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:33:12.666389 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75785f94-ed53-418c-81e2-117c30ac13d5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:33:12.666661 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75785f94-ed53-418c-81e2-117c30ac13d5 admin admin] PATH_INFO: `/v3/users` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:33:12.667315 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75785f94-ed53-418c-81e2-117c30ac13d5 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:33:12.669217 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75785f94-ed53-418c-81e2-117c30ac13d5 admin admin] RBAC: Authorization granted {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:33:12.677447 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG dbcounter [-] [80378] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80378) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 10:33:12.692865 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 1391/2795] 10.222.0.82 () {60 vars in 1144 bytes} [Tue Aug 2 10:33:12 2022] POST /identity/v3/users => generated 287 bytes in 94 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 02 10:33:12.696989 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-534670b6-5cfc-4736-8a4c-156deef20c83 admin admin] Authenticating user token {{(pid=80377) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:33:12.707358 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-534670b6-5cfc-4736-8a4c-156deef20c83 admin admin] Validating token access rules against request {{(pid=80377) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:33:12.712661 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2e93343-c18e-4659-bb6b-88d4d8bf2024 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '891005fa5ff74041b75747d16cc66798', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7047912c5acd4adeba5d87c3bf895e39', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80377) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:33:12.713272 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2e93343-c18e-4659-bb6b-88d4d8bf2024 admin admin] REQUEST_METHOD: `GET` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:33:12.713557 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2e93343-c18e-4659-bb6b-88d4d8bf2024 admin admin] SCRIPT_NAME: `/identity` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:33:12.713870 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2e93343-c18e-4659-bb6b-88d4d8bf2024 admin admin] PATH_INFO: `/v3/roles` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:33:12.714569 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2e93343-c18e-4659-bb6b-88d4d8bf2024 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:33:12.716164 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2e93343-c18e-4659-bb6b-88d4d8bf2024 admin admin] RBAC: Authorization granted {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:33:12.726603 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: [pid: 80377|app: 0|req: 1405/2796] 10.222.0.82 () {58 vars in 1122 bytes} [Tue Aug 2 10:33:12 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 02 10:33:12.734883 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75785f94-ed53-418c-81e2-117c30ac13d5 admin admin] Authenticating user token {{(pid=80378) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:33:12.745487 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75785f94-ed53-418c-81e2-117c30ac13d5 admin admin] Validating token access rules against request {{(pid=80378) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:33:12.750757 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be9bb372-d03e-424a-8aa1-356b6ba6cf06 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '891005fa5ff74041b75747d16cc66798', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7047912c5acd4adeba5d87c3bf895e39', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80378) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:33:12.751369 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be9bb372-d03e-424a-8aa1-356b6ba6cf06 admin admin] REQUEST_METHOD: `GET` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:33:12.751648 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be9bb372-d03e-424a-8aa1-356b6ba6cf06 admin admin] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:33:12.751953 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be9bb372-d03e-424a-8aa1-356b6ba6cf06 admin admin] PATH_INFO: `/v3/roles` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:33:12.752660 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be9bb372-d03e-424a-8aa1-356b6ba6cf06 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:33:12.754279 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be9bb372-d03e-424a-8aa1-356b6ba6cf06 admin admin] RBAC: Authorization granted {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:33:12.764939 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 1392/2797] 10.222.0.82 () {58 vars in 1122 bytes} [Tue Aug 2 10:33:12 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 02 10:33:12.769822 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2e93343-c18e-4659-bb6b-88d4d8bf2024 admin admin] Authenticating user token {{(pid=80377) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:33:12.780188 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2e93343-c18e-4659-bb6b-88d4d8bf2024 admin admin] Validating token access rules against request {{(pid=80377) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:33:12.786683 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-169f7ade-8173-469d-95b6-50baed5c4b9e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '891005fa5ff74041b75747d16cc66798', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7047912c5acd4adeba5d87c3bf895e39', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80377) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:33:12.787878 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-169f7ade-8173-469d-95b6-50baed5c4b9e admin admin] REQUEST_METHOD: `PUT` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:33:12.788399 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-169f7ade-8173-469d-95b6-50baed5c4b9e admin admin] SCRIPT_NAME: `/identity` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:33:12.788911 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-169f7ade-8173-469d-95b6-50baed5c4b9e admin admin] PATH_INFO: `/v3/projects/9f1160d2f3e14f3c945867fcabcc3a12/users/ad9d9f2dcb3a47809e07efb4f70df558/roles/e1c04b19fc4543b8b54f58bbcffbc598` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:33:12.810207 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-169f7ade-8173-469d-95b6-50baed5c4b9e admin admin] RBAC: Authorizing `identity:create_grant(project_id=9f1160d2f3e14f3c945867fcabcc3a12, user_id=ad9d9f2dcb3a47809e07efb4f70df558, role_id=e1c04b19fc4543b8b54f58bbcffbc598)` {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:33:12.813401 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-169f7ade-8173-469d-95b6-50baed5c4b9e admin admin] RBAC: Authorization granted {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:33:12.828190 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: [pid: 80377|app: 0|req: 1406/2798] 10.222.0.82 () {60 vars in 1483 bytes} [Tue Aug 2 10:33:12 2022] PUT /identity/v3/projects/9f1160d2f3e14f3c945867fcabcc3a12/users/ad9d9f2dcb3a47809e07efb4f70df558/roles/e1c04b19fc4543b8b54f58bbcffbc598 => generated 0 bytes in 60 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 02 10:33:12.838366 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9d3e39c-66c4-4bdd-a7aa-970184a9e69c None None] REQUEST_METHOD: `POST` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:33:12.839282 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9d3e39c-66c4-4bdd-a7aa-970184a9e69c None None] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:33:12.839561 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9d3e39c-66c4-4bdd-a7aa-970184a9e69c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:33:12.867616 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.auth.core [None req-a9d3e39c-66c4-4bdd-a7aa-970184a9e69c None None] MFA Rules not processed for user `ad9d9f2dcb3a47809e07efb4f70df558`. Rule list: `[]` (Enabled: `True`). {{(pid=80378) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 02 10:33:12.868957 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.fernet_utils [None req-a9d3e39c-66c4-4bdd-a7aa-970184a9e69c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80378) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 10:33:12.938485 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 1393/2799] 10.222.0.82 () {58 vars in 958 bytes} [Tue Aug 2 10:33:12 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 02 10:33:12.943201 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-169f7ade-8173-469d-95b6-50baed5c4b9e admin admin] Authenticating user token {{(pid=80377) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:33:13.006892 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-169f7ade-8173-469d-95b6-50baed5c4b9e admin admin] Validating token access rules against request {{(pid=80377) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:33:13.011543 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a94f4afe-12ad-4c1b-bfbb-13f44ec879f8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '891005fa5ff74041b75747d16cc66798', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7047912c5acd4adeba5d87c3bf895e39', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80377) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:33:13.012164 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a94f4afe-12ad-4c1b-bfbb-13f44ec879f8 admin admin] REQUEST_METHOD: `POST` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:33:13.014558 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a94f4afe-12ad-4c1b-bfbb-13f44ec879f8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:33:13.014863 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a94f4afe-12ad-4c1b-bfbb-13f44ec879f8 admin admin] PATH_INFO: `/v3/projects` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:33:13.015557 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a94f4afe-12ad-4c1b-bfbb-13f44ec879f8 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:33:13.018783 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a94f4afe-12ad-4c1b-bfbb-13f44ec879f8 admin admin] RBAC: Authorization granted {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:33:13.049301 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: [pid: 80377|app: 0|req: 1407/2800] 10.222.0.82 () {60 vars in 1153 bytes} [Tue Aug 2 10:33:12 2022] POST /identity/v3/projects => generated 373 bytes in 107 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 02 10:33:13.053469 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9d3e39c-66c4-4bdd-a7aa-970184a9e69c None None] Authenticating user token {{(pid=80378) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:33:13.064607 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9d3e39c-66c4-4bdd-a7aa-970184a9e69c None None] Validating token access rules against request {{(pid=80378) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:33:13.130836 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95a50c32-1d2f-4902-8f67-5fed26825e66 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '891005fa5ff74041b75747d16cc66798', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7047912c5acd4adeba5d87c3bf895e39', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80378) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:33:13.131462 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95a50c32-1d2f-4902-8f67-5fed26825e66 admin admin] REQUEST_METHOD: `POST` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:33:13.131748 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95a50c32-1d2f-4902-8f67-5fed26825e66 admin admin] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:33:13.132023 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95a50c32-1d2f-4902-8f67-5fed26825e66 admin admin] PATH_INFO: `/v3/users` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:33:13.132821 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95a50c32-1d2f-4902-8f67-5fed26825e66 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:33:13.134811 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95a50c32-1d2f-4902-8f67-5fed26825e66 admin admin] RBAC: Authorization granted {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:33:13.159870 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 1394/2801] 10.222.0.82 () {60 vars in 1144 bytes} [Tue Aug 2 10:33:13 2022] POST /identity/v3/users => generated 293 bytes in 107 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 02 10:33:13.163726 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a94f4afe-12ad-4c1b-bfbb-13f44ec879f8 admin admin] Authenticating user token {{(pid=80377) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:33:13.174583 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a94f4afe-12ad-4c1b-bfbb-13f44ec879f8 admin admin] Validating token access rules against request {{(pid=80377) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:33:13.179748 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86bc7ebe-297e-4d74-a6ce-78615dddbeb3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '891005fa5ff74041b75747d16cc66798', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7047912c5acd4adeba5d87c3bf895e39', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80377) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:33:13.180364 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86bc7ebe-297e-4d74-a6ce-78615dddbeb3 admin admin] REQUEST_METHOD: `GET` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:33:13.180659 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86bc7ebe-297e-4d74-a6ce-78615dddbeb3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:33:13.180939 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86bc7ebe-297e-4d74-a6ce-78615dddbeb3 admin admin] PATH_INFO: `/v3/roles` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:33:13.181661 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86bc7ebe-297e-4d74-a6ce-78615dddbeb3 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:33:13.183302 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86bc7ebe-297e-4d74-a6ce-78615dddbeb3 admin admin] RBAC: Authorization granted {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:33:13.194752 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: [pid: 80377|app: 0|req: 1408/2802] 10.222.0.82 () {58 vars in 1122 bytes} [Tue Aug 2 10:33:13 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 02 10:33:13.199435 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95a50c32-1d2f-4902-8f67-5fed26825e66 admin admin] Authenticating user token {{(pid=80378) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:33:13.210518 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95a50c32-1d2f-4902-8f67-5fed26825e66 admin admin] Validating token access rules against request {{(pid=80378) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:33:13.215238 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b868d14e-4b21-45c6-b710-9ef92a7ea798 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '891005fa5ff74041b75747d16cc66798', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7047912c5acd4adeba5d87c3bf895e39', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80378) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:33:13.215930 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b868d14e-4b21-45c6-b710-9ef92a7ea798 admin admin] REQUEST_METHOD: `PUT` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:33:13.216215 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b868d14e-4b21-45c6-b710-9ef92a7ea798 admin admin] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:33:13.216998 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b868d14e-4b21-45c6-b710-9ef92a7ea798 admin admin] PATH_INFO: `/v3/projects/d072cc2b72dc4b4995178109f3b584a7/users/7b60bdda0d76427b84ece3d8dcb9ec5c/roles/237966442f05411191105bf336bc14f3` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:33:13.233036 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b868d14e-4b21-45c6-b710-9ef92a7ea798 admin admin] RBAC: Authorizing `identity:create_grant(project_id=d072cc2b72dc4b4995178109f3b584a7, user_id=7b60bdda0d76427b84ece3d8dcb9ec5c, role_id=237966442f05411191105bf336bc14f3)` {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:33:13.235753 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b868d14e-4b21-45c6-b710-9ef92a7ea798 admin admin] RBAC: Authorization granted {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:33:13.248383 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 1395/2803] 10.222.0.82 () {60 vars in 1483 bytes} [Tue Aug 2 10:33:13 2022] PUT /identity/v3/projects/d072cc2b72dc4b4995178109f3b584a7/users/7b60bdda0d76427b84ece3d8dcb9ec5c/roles/237966442f05411191105bf336bc14f3 => generated 0 bytes in 49 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 02 10:33:13.252934 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86bc7ebe-297e-4d74-a6ce-78615dddbeb3 admin admin] Authenticating user token {{(pid=80377) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:33:13.263033 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86bc7ebe-297e-4d74-a6ce-78615dddbeb3 admin admin] Validating token access rules against request {{(pid=80377) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:33:13.266825 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG dbcounter [-] [80377] Writing DB stats keystone:SELECT=89,keystone:DELETE=7,keystone:INSERT=4 {{(pid=80377) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 10:33:13.326978 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb43dace-2ead-4b09-b567-ec49486484f5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '891005fa5ff74041b75747d16cc66798', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7047912c5acd4adeba5d87c3bf895e39', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80377) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:33:13.327803 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb43dace-2ead-4b09-b567-ec49486484f5 admin admin] REQUEST_METHOD: `GET` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:33:13.328310 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb43dace-2ead-4b09-b567-ec49486484f5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:33:13.328716 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb43dace-2ead-4b09-b567-ec49486484f5 admin admin] PATH_INFO: `/v3/roles` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:33:13.329421 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb43dace-2ead-4b09-b567-ec49486484f5 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:33:13.331049 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb43dace-2ead-4b09-b567-ec49486484f5 admin admin] RBAC: Authorization granted {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:33:13.341204 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: [pid: 80377|app: 0|req: 1409/2804] 10.222.0.82 () {58 vars in 1122 bytes} [Tue Aug 2 10:33:13 2022] GET /identity/v3/roles => generated 1420 bytes in 89 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 02 10:33:13.345427 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b868d14e-4b21-45c6-b710-9ef92a7ea798 admin admin] Authenticating user token {{(pid=80378) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:33:13.358347 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b868d14e-4b21-45c6-b710-9ef92a7ea798 admin admin] Validating token access rules against request {{(pid=80378) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:33:13.363336 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-790b906d-ecb3-4fe6-906e-69aa20aa0ac8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '891005fa5ff74041b75747d16cc66798', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7047912c5acd4adeba5d87c3bf895e39', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80378) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:33:13.364013 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-790b906d-ecb3-4fe6-906e-69aa20aa0ac8 admin admin] REQUEST_METHOD: `PUT` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:33:13.364300 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-790b906d-ecb3-4fe6-906e-69aa20aa0ac8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:33:13.364578 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-790b906d-ecb3-4fe6-906e-69aa20aa0ac8 admin admin] PATH_INFO: `/v3/projects/d072cc2b72dc4b4995178109f3b584a7/users/7b60bdda0d76427b84ece3d8dcb9ec5c/roles/237966442f05411191105bf336bc14f3` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:33:13.366599 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-790b906d-ecb3-4fe6-906e-69aa20aa0ac8 admin admin] RBAC: Authorizing `identity:create_grant(project_id=d072cc2b72dc4b4995178109f3b584a7, user_id=7b60bdda0d76427b84ece3d8dcb9ec5c, role_id=237966442f05411191105bf336bc14f3)` {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:33:13.369237 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-790b906d-ecb3-4fe6-906e-69aa20aa0ac8 admin admin] RBAC: Authorization granted {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:33:13.382592 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 1396/2805] 10.222.0.82 () {60 vars in 1483 bytes} [Tue Aug 2 10:33:13 2022] PUT /identity/v3/projects/d072cc2b72dc4b4995178109f3b584a7/users/7b60bdda0d76427b84ece3d8dcb9ec5c/roles/237966442f05411191105bf336bc14f3 => generated 0 bytes in 38 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 02 10:33:13.393589 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-946d2fb9-91f8-4ee1-983d-2c6fb59955e7 None None] REQUEST_METHOD: `POST` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:33:13.393917 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-946d2fb9-91f8-4ee1-983d-2c6fb59955e7 None None] SCRIPT_NAME: `/identity` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:33:13.394288 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-946d2fb9-91f8-4ee1-983d-2c6fb59955e7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:33:13.425528 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.auth.core [None req-946d2fb9-91f8-4ee1-983d-2c6fb59955e7 None None] MFA Rules not processed for user `7b60bdda0d76427b84ece3d8dcb9ec5c`. Rule list: `[]` (Enabled: `True`). {{(pid=80377) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 02 10:33:13.426789 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.fernet_utils [None req-946d2fb9-91f8-4ee1-983d-2c6fb59955e7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80377) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 10:33:13.497330 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: [pid: 80377|app: 0|req: 1410/2806] 10.222.0.82 () {58 vars in 958 bytes} [Tue Aug 2 10:33:13 2022] POST /identity/v3/auth/tokens => generated 3317 bytes in 105 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 02 10:33:13.509328 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-790b906d-ecb3-4fe6-906e-69aa20aa0ac8 admin admin] Authenticating user token {{(pid=80378) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:33:13.511085 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.fernet_utils [None req-790b906d-ecb3-4fe6-906e-69aa20aa0ac8 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80378) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 10:33:13.534248 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG dbcounter [-] [80378] Writing DB stats keystone:INSERT=8,keystone:SELECT=92 {{(pid=80378) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 10:33:13.573302 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-790b906d-ecb3-4fe6-906e-69aa20aa0ac8 admin admin] Validating token access rules against request {{(pid=80378) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:33:13.577707 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-637e8c0c-dc6f-4344-9b0c-a7a99f468121 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1023de281ae74e9d8a6d6c312c1c97a7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f1b38ebb804c41c69772c998aae72c41', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80378) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:33:13.578291 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-637e8c0c-dc6f-4344-9b0c-a7a99f468121 service cinder] REQUEST_METHOD: `GET` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:33:13.578574 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-637e8c0c-dc6f-4344-9b0c-a7a99f468121 service cinder] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:33:13.578847 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-637e8c0c-dc6f-4344-9b0c-a7a99f468121 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:33:13.588272 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-637e8c0c-dc6f-4344-9b0c-a7a99f468121 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:33:13.590625 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-637e8c0c-dc6f-4344-9b0c-a7a99f468121 service cinder] RBAC: Authorization granted {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:33:13.600922 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 1397/2807] 10.222.0.82 () {60 vars in 1365 bytes} [Tue Aug 2 10:33:13 2022] GET /identity/v3/auth/tokens => generated 3317 bytes in 92 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 02 10:33:13.685432 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-946d2fb9-91f8-4ee1-983d-2c6fb59955e7 None None] Authenticating user token {{(pid=80377) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:33:13.695466 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-946d2fb9-91f8-4ee1-983d-2c6fb59955e7 None None] Validating token access rules against request {{(pid=80377) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:33:13.699112 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-267345be-eb04-4c8f-b56d-4c186ef4ab33 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1023de281ae74e9d8a6d6c312c1c97a7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f1b38ebb804c41c69772c998aae72c41', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80377) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:33:13.699740 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-267345be-eb04-4c8f-b56d-4c186ef4ab33 service cinder] REQUEST_METHOD: `GET` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:33:13.700053 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-267345be-eb04-4c8f-b56d-4c186ef4ab33 service cinder] SCRIPT_NAME: `/identity` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:33:13.700335 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-267345be-eb04-4c8f-b56d-4c186ef4ab33 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:33:13.756109 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-267345be-eb04-4c8f-b56d-4c186ef4ab33 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:33:13.757531 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-267345be-eb04-4c8f-b56d-4c186ef4ab33 service cinder] RBAC: Authorization granted {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:33:13.766738 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: [pid: 80377|app: 0|req: 1411/2808] 10.222.0.82 () {60 vars in 1365 bytes} [Tue Aug 2 10:33:13 2022] GET /identity/v3/auth/tokens => generated 3250 bytes in 82 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 02 10:33:23.596609 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG dbcounter [-] [80378] Writing DB stats keystone:SELECT=17 {{(pid=80378) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 10:33:23.762256 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG dbcounter [-] [80377] Writing DB stats keystone:SELECT=88 {{(pid=80377) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 10:33:57.151723 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-637e8c0c-dc6f-4344-9b0c-a7a99f468121 service cinder] Authenticating user token {{(pid=80378) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:33:57.212848 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-637e8c0c-dc6f-4344-9b0c-a7a99f468121 service cinder] Validating token access rules against request {{(pid=80378) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:33:57.217386 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92cc8cb1-ffc9-4adb-b9ca-a1c4242c4c06 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '891005fa5ff74041b75747d16cc66798', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7047912c5acd4adeba5d87c3bf895e39', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80378) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:33:57.218044 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92cc8cb1-ffc9-4adb-b9ca-a1c4242c4c06 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:33:57.218329 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92cc8cb1-ffc9-4adb-b9ca-a1c4242c4c06 admin admin] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:33:57.218603 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92cc8cb1-ffc9-4adb-b9ca-a1c4242c4c06 admin admin] PATH_INFO: `/v3/users/ad9d9f2dcb3a47809e07efb4f70df558` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:33:57.219756 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92cc8cb1-ffc9-4adb-b9ca-a1c4242c4c06 admin admin] RBAC: Authorizing `identity:delete_user(user_id=ad9d9f2dcb3a47809e07efb4f70df558)` {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:33:57.221661 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92cc8cb1-ffc9-4adb-b9ca-a1c4242c4c06 admin admin] RBAC: Authorization granted {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:33:57.266727 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.notifications [None req-92cc8cb1-ffc9-4adb-b9ca-a1c4242c4c06 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'ad9d9f2dcb3a47809e07efb4f70df558'} {{(pid=80378) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 10:33:57.273801 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.notifications [None req-92cc8cb1-ffc9-4adb-b9ca-a1c4242c4c06 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'ad9d9f2dcb3a47809e07efb4f70df558'} {{(pid=80378) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 10:33:57.280592 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.notifications [None req-92cc8cb1-ffc9-4adb-b9ca-a1c4242c4c06 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'ad9d9f2dcb3a47809e07efb4f70df558'} {{(pid=80378) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 10:33:57.281222 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.notifications [None req-92cc8cb1-ffc9-4adb-b9ca-a1c4242c4c06 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'ad9d9f2dcb3a47809e07efb4f70df558'} {{(pid=80378) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 10:33:57.293485 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.notifications [None req-92cc8cb1-ffc9-4adb-b9ca-a1c4242c4c06 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'ad9d9f2dcb3a47809e07efb4f70df558'} {{(pid=80378) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 10:33:57.299241 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 1398/2809] 10.222.0.82 () {58 vars in 1224 bytes} [Tue Aug 2 10:33:57 2022] DELETE /identity/v3/users/ad9d9f2dcb3a47809e07efb4f70df558 => generated 0 bytes in 148 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 02 10:33:57.304312 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3614eeac-a0f8-4342-a0cc-4deb40e255b9 None None] REQUEST_METHOD: `POST` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:33:57.304813 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3614eeac-a0f8-4342-a0cc-4deb40e255b9 None None] SCRIPT_NAME: `/identity` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:33:57.304925 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3614eeac-a0f8-4342-a0cc-4deb40e255b9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:33:57.323661 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.auth.core [None req-3614eeac-a0f8-4342-a0cc-4deb40e255b9 None None] MFA Rules not processed for user `891005fa5ff74041b75747d16cc66798`. Rule list: `[]` (Enabled: `True`). {{(pid=80377) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 02 10:33:57.324854 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.fernet_utils [None req-3614eeac-a0f8-4342-a0cc-4deb40e255b9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80377) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 10:33:57.381735 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: [pid: 80377|app: 0|req: 1412/2810] 10.222.0.82 () {58 vars in 958 bytes} [Tue Aug 2 10:33:57 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 78 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 02 10:33:57.391428 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92cc8cb1-ffc9-4adb-b9ca-a1c4242c4c06 admin admin] Authenticating user token {{(pid=80378) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:33:57.392913 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.fernet_utils [None req-92cc8cb1-ffc9-4adb-b9ca-a1c4242c4c06 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80378) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 10:33:57.449559 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92cc8cb1-ffc9-4adb-b9ca-a1c4242c4c06 admin admin] Validating token access rules against request {{(pid=80378) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:33:57.454216 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50709fe5-bf4e-4aac-8d82-31e72f9a5bd8 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '33b87170f0634b67bce60750847b5cbe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f1b38ebb804c41c69772c998aae72c41', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80378) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:33:57.454871 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50709fe5-bf4e-4aac-8d82-31e72f9a5bd8 service neutron] REQUEST_METHOD: `GET` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:33:57.455154 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50709fe5-bf4e-4aac-8d82-31e72f9a5bd8 service neutron] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:33:57.455430 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50709fe5-bf4e-4aac-8d82-31e72f9a5bd8 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:33:57.464495 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50709fe5-bf4e-4aac-8d82-31e72f9a5bd8 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:33:57.465957 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50709fe5-bf4e-4aac-8d82-31e72f9a5bd8 service neutron] RBAC: Authorization granted {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:33:57.475856 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 1399/2811] 10.222.0.82 () {60 vars in 1365 bytes} [Tue Aug 2 10:33:57 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 85 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 02 10:33:57.745625 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3614eeac-a0f8-4342-a0cc-4deb40e255b9 None None] Authenticating user token {{(pid=80377) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:33:57.747505 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.fernet_utils [None req-3614eeac-a0f8-4342-a0cc-4deb40e255b9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80377) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 10:33:57.759694 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3614eeac-a0f8-4342-a0cc-4deb40e255b9 None None] Validating token access rules against request {{(pid=80377) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:33:57.764409 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b7d9b17-f0ba-450d-b494-e1425d148684 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '891005fa5ff74041b75747d16cc66798', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7047912c5acd4adeba5d87c3bf895e39', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80377) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:33:57.765048 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b7d9b17-f0ba-450d-b494-e1425d148684 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:33:57.765331 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b7d9b17-f0ba-450d-b494-e1425d148684 admin admin] SCRIPT_NAME: `/identity` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:33:57.765603 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b7d9b17-f0ba-450d-b494-e1425d148684 admin admin] PATH_INFO: `/v3/projects/9f1160d2f3e14f3c945867fcabcc3a12` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:33:57.766829 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b7d9b17-f0ba-450d-b494-e1425d148684 admin admin] RBAC: Authorizing `identity:delete_project(project_id=9f1160d2f3e14f3c945867fcabcc3a12)` {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:33:57.769438 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b7d9b17-f0ba-450d-b494-e1425d148684 admin admin] RBAC: Authorization granted {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:33:57.816208 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.notifications [None req-1b7d9b17-f0ba-450d-b494-e1425d148684 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '9f1160d2f3e14f3c945867fcabcc3a12', 'request_id': 'req-1b7d9b17-f0ba-450d-b494-e1425d148684'} {{(pid=80377) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 10:33:57.821841 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.notifications [None req-1b7d9b17-f0ba-450d-b494-e1425d148684 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '9f1160d2f3e14f3c945867fcabcc3a12', 'request_id': 'req-1b7d9b17-f0ba-450d-b494-e1425d148684'} {{(pid=80377) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 10:33:57.827878 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.notifications [None req-1b7d9b17-f0ba-450d-b494-e1425d148684 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '9f1160d2f3e14f3c945867fcabcc3a12', 'request_id': 'req-1b7d9b17-f0ba-450d-b494-e1425d148684'} {{(pid=80377) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 10:33:57.840445 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.notifications [None req-1b7d9b17-f0ba-450d-b494-e1425d148684 admin admin] The token cache is being invalidate because project 9f1160d2f3e14f3c945867fcabcc3a12 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80377) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 02 10:33:57.840735 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.notifications [None req-1b7d9b17-f0ba-450d-b494-e1425d148684 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80377) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 10:33:57.841359 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.notifications [None req-1b7d9b17-f0ba-450d-b494-e1425d148684 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80377) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 10:33:57.842886 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: [pid: 80377|app: 0|req: 1413/2812] 10.222.0.82 () {58 vars in 1233 bytes} [Tue Aug 2 10:33:57 2022] DELETE /identity/v3/projects/9f1160d2f3e14f3c945867fcabcc3a12 => generated 0 bytes in 98 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 02 10:33:57.847983 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50709fe5-bf4e-4aac-8d82-31e72f9a5bd8 service neutron] Authenticating user token {{(pid=80378) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:33:57.857497 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50709fe5-bf4e-4aac-8d82-31e72f9a5bd8 service neutron] Validating token access rules against request {{(pid=80378) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:33:57.875087 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG dbcounter [-] [80378] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80378) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 10:33:57.914421 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1959616-8caf-4114-bf0d-eeace547609d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '891005fa5ff74041b75747d16cc66798', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7047912c5acd4adeba5d87c3bf895e39', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80378) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:33:57.915054 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1959616-8caf-4114-bf0d-eeace547609d admin admin] REQUEST_METHOD: `DELETE` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:33:57.915333 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1959616-8caf-4114-bf0d-eeace547609d admin admin] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:33:57.915606 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1959616-8caf-4114-bf0d-eeace547609d admin admin] PATH_INFO: `/v3/users/7b60bdda0d76427b84ece3d8dcb9ec5c` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:33:57.916745 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1959616-8caf-4114-bf0d-eeace547609d admin admin] RBAC: Authorizing `identity:delete_user(user_id=7b60bdda0d76427b84ece3d8dcb9ec5c)` {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:33:57.918691 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1959616-8caf-4114-bf0d-eeace547609d admin admin] RBAC: Authorization granted {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:33:57.965330 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.notifications [None req-c1959616-8caf-4114-bf0d-eeace547609d admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '7b60bdda0d76427b84ece3d8dcb9ec5c'} {{(pid=80378) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 10:33:57.972681 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.notifications [None req-c1959616-8caf-4114-bf0d-eeace547609d admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '7b60bdda0d76427b84ece3d8dcb9ec5c'} {{(pid=80378) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 10:33:57.979485 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.notifications [None req-c1959616-8caf-4114-bf0d-eeace547609d admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '7b60bdda0d76427b84ece3d8dcb9ec5c'} {{(pid=80378) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 10:33:57.980119 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.notifications [None req-c1959616-8caf-4114-bf0d-eeace547609d admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '7b60bdda0d76427b84ece3d8dcb9ec5c'} {{(pid=80378) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 10:33:57.991633 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.notifications [None req-c1959616-8caf-4114-bf0d-eeace547609d admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '7b60bdda0d76427b84ece3d8dcb9ec5c'} {{(pid=80378) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 10:33:57.997609 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 1400/2813] 10.222.0.82 () {58 vars in 1224 bytes} [Tue Aug 2 10:33:57 2022] DELETE /identity/v3/users/7b60bdda0d76427b84ece3d8dcb9ec5c => generated 0 bytes in 151 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 02 10:33:58.261292 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b7d9b17-f0ba-450d-b494-e1425d148684 admin admin] Authenticating user token {{(pid=80377) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:33:58.262755 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.fernet_utils [None req-1b7d9b17-f0ba-450d-b494-e1425d148684 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80377) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 10:33:58.276389 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b7d9b17-f0ba-450d-b494-e1425d148684 admin admin] Validating token access rules against request {{(pid=80377) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:33:58.332121 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4ba9d13-37df-48f1-8150-2bed346a3310 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '891005fa5ff74041b75747d16cc66798', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7047912c5acd4adeba5d87c3bf895e39', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80377) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:33:58.332798 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4ba9d13-37df-48f1-8150-2bed346a3310 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:33:58.333037 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4ba9d13-37df-48f1-8150-2bed346a3310 admin admin] SCRIPT_NAME: `/identity` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:33:58.333308 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4ba9d13-37df-48f1-8150-2bed346a3310 admin admin] PATH_INFO: `/v3/projects/d072cc2b72dc4b4995178109f3b584a7` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:33:58.334483 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4ba9d13-37df-48f1-8150-2bed346a3310 admin admin] RBAC: Authorizing `identity:delete_project(project_id=d072cc2b72dc4b4995178109f3b584a7)` {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:33:58.336382 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4ba9d13-37df-48f1-8150-2bed346a3310 admin admin] RBAC: Authorization granted {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:33:58.365417 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG dbcounter [-] [80377] Writing DB stats keystone:SELECT=91,keystone:DELETE=8,keystone:INSERT=1 {{(pid=80377) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 10:33:58.383584 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.notifications [None req-a4ba9d13-37df-48f1-8150-2bed346a3310 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'd072cc2b72dc4b4995178109f3b584a7', 'request_id': 'req-a4ba9d13-37df-48f1-8150-2bed346a3310'} {{(pid=80377) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 10:33:58.388993 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.notifications [None req-a4ba9d13-37df-48f1-8150-2bed346a3310 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'd072cc2b72dc4b4995178109f3b584a7', 'request_id': 'req-a4ba9d13-37df-48f1-8150-2bed346a3310'} {{(pid=80377) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 10:33:58.395040 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.notifications [None req-a4ba9d13-37df-48f1-8150-2bed346a3310 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'd072cc2b72dc4b4995178109f3b584a7', 'request_id': 'req-a4ba9d13-37df-48f1-8150-2bed346a3310'} {{(pid=80377) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 10:33:58.407169 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.notifications [None req-a4ba9d13-37df-48f1-8150-2bed346a3310 admin admin] The token cache is being invalidate because project d072cc2b72dc4b4995178109f3b584a7 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80377) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 02 10:33:58.407654 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.notifications [None req-a4ba9d13-37df-48f1-8150-2bed346a3310 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80377) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 10:33:58.408269 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.notifications [None req-a4ba9d13-37df-48f1-8150-2bed346a3310 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80377) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 10:33:58.409890 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: [pid: 80377|app: 0|req: 1414/2814] 10.222.0.82 () {58 vars in 1233 bytes} [Tue Aug 2 10:33:58 2022] DELETE /identity/v3/projects/d072cc2b72dc4b4995178109f3b584a7 => generated 0 bytes in 149 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 02 10:33:58.414931 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fd9f450-7e0a-4944-b916-bc3e4f3f5954 None None] REQUEST_METHOD: `POST` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:33:58.415394 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fd9f450-7e0a-4944-b916-bc3e4f3f5954 None None] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:33:58.415510 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fd9f450-7e0a-4944-b916-bc3e4f3f5954 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:33:58.435644 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.auth.core [None req-1fd9f450-7e0a-4944-b916-bc3e4f3f5954 None None] MFA Rules not processed for user `891005fa5ff74041b75747d16cc66798`. Rule list: `[]` (Enabled: `True`). {{(pid=80378) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 02 10:33:58.436827 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.fernet_utils [None req-1fd9f450-7e0a-4944-b916-bc3e4f3f5954 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80378) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 10:33:58.493845 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 1401/2815] 10.222.0.82 () {58 vars in 958 bytes} [Tue Aug 2 10:33:58 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 80 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 02 10:33:58.501272 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d8a3f8d-2e31-4e87-bd9d-bc63ae77d9f7 None None] REQUEST_METHOD: `POST` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:33:58.501559 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d8a3f8d-2e31-4e87-bd9d-bc63ae77d9f7 None None] SCRIPT_NAME: `/identity` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:33:58.501868 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d8a3f8d-2e31-4e87-bd9d-bc63ae77d9f7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:33:58.521371 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.auth.core [None req-8d8a3f8d-2e31-4e87-bd9d-bc63ae77d9f7 None None] MFA Rules not processed for user `891005fa5ff74041b75747d16cc66798`. Rule list: `[]` (Enabled: `True`). {{(pid=80377) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 02 10:33:58.522591 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.fernet_utils [None req-8d8a3f8d-2e31-4e87-bd9d-bc63ae77d9f7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80377) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 10:33:58.530315 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: [pid: 80377|app: 0|req: 1415/2816] 10.222.0.82 () {58 vars in 958 bytes} [Tue Aug 2 10:33:58 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 02 10:33:58.534335 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fd9f450-7e0a-4944-b916-bc3e4f3f5954 None None] Authenticating user token {{(pid=80378) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:33:58.544064 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fd9f450-7e0a-4944-b916-bc3e4f3f5954 None None] Validating token access rules against request {{(pid=80378) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:33:58.548636 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-034626e9-096e-47eb-8870-bf9357e84457 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '891005fa5ff74041b75747d16cc66798', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7047912c5acd4adeba5d87c3bf895e39', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80378) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:33:58.549247 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-034626e9-096e-47eb-8870-bf9357e84457 admin admin] REQUEST_METHOD: `GET` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:33:58.549528 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-034626e9-096e-47eb-8870-bf9357e84457 admin admin] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:33:58.550379 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-034626e9-096e-47eb-8870-bf9357e84457 admin admin] PATH_INFO: `/v3/domains` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:33:58.551147 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-034626e9-096e-47eb-8870-bf9357e84457 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80378) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 02 10:33:58.551454 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-034626e9-096e-47eb-8870-bf9357e84457 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:33:58.553047 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-034626e9-096e-47eb-8870-bf9357e84457 admin admin] RBAC: Authorization granted {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:33:58.564424 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 1402/2817] 10.222.0.82 () {58 vars in 1153 bytes} [Tue Aug 2 10:33:58 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 02 10:33:58.570439 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d8a3f8d-2e31-4e87-bd9d-bc63ae77d9f7 None None] Authenticating user token {{(pid=80377) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:33:58.580120 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d8a3f8d-2e31-4e87-bd9d-bc63ae77d9f7 None None] Validating token access rules against request {{(pid=80377) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:33:58.586541 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6b31f35-f865-493e-a515-a0aa0328ea8e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '891005fa5ff74041b75747d16cc66798', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7047912c5acd4adeba5d87c3bf895e39', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80377) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:33:58.588184 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6b31f35-f865-493e-a515-a0aa0328ea8e admin admin] REQUEST_METHOD: `POST` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:33:58.588184 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6b31f35-f865-493e-a515-a0aa0328ea8e admin admin] SCRIPT_NAME: `/identity` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:33:58.588184 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6b31f35-f865-493e-a515-a0aa0328ea8e admin admin] PATH_INFO: `/v3/projects` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:33:58.588850 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6b31f35-f865-493e-a515-a0aa0328ea8e admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:33:58.591534 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6b31f35-f865-493e-a515-a0aa0328ea8e admin admin] RBAC: Authorization granted {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:33:58.620073 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: [pid: 80377|app: 0|req: 1416/2818] 10.222.0.82 () {60 vars in 1153 bytes} [Tue Aug 2 10:33:58 2022] POST /identity/v3/projects => generated 365 bytes in 51 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 02 10:33:58.624132 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-034626e9-096e-47eb-8870-bf9357e84457 admin admin] Authenticating user token {{(pid=80378) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:33:58.634697 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-034626e9-096e-47eb-8870-bf9357e84457 admin admin] Validating token access rules against request {{(pid=80378) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:33:58.660572 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG dbcounter [-] [80378] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80378) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 10:33:58.693245 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cab7f7a0-90ff-4f6e-bbf4-6b732925262e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '891005fa5ff74041b75747d16cc66798', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7047912c5acd4adeba5d87c3bf895e39', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80378) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:33:58.693882 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cab7f7a0-90ff-4f6e-bbf4-6b732925262e admin admin] REQUEST_METHOD: `POST` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:33:58.694168 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cab7f7a0-90ff-4f6e-bbf4-6b732925262e admin admin] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:33:58.694449 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cab7f7a0-90ff-4f6e-bbf4-6b732925262e admin admin] PATH_INFO: `/v3/users` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:33:58.695119 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cab7f7a0-90ff-4f6e-bbf4-6b732925262e admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:33:58.697028 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cab7f7a0-90ff-4f6e-bbf4-6b732925262e admin admin] RBAC: Authorization granted {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:33:58.717749 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 1403/2819] 10.222.0.82 () {60 vars in 1144 bytes} [Tue Aug 2 10:33:58 2022] POST /identity/v3/users => generated 283 bytes in 94 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 02 10:33:58.722437 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6b31f35-f865-493e-a515-a0aa0328ea8e admin admin] Authenticating user token {{(pid=80377) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:33:58.733000 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6b31f35-f865-493e-a515-a0aa0328ea8e admin admin] Validating token access rules against request {{(pid=80377) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:33:58.738600 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e04c96cf-650e-43d4-a1ca-821aa6c5d5af admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '891005fa5ff74041b75747d16cc66798', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7047912c5acd4adeba5d87c3bf895e39', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80377) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:33:58.739204 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e04c96cf-650e-43d4-a1ca-821aa6c5d5af admin admin] REQUEST_METHOD: `GET` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:33:58.739485 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e04c96cf-650e-43d4-a1ca-821aa6c5d5af admin admin] SCRIPT_NAME: `/identity` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:33:58.739758 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e04c96cf-650e-43d4-a1ca-821aa6c5d5af admin admin] PATH_INFO: `/v3/roles` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:33:58.740453 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e04c96cf-650e-43d4-a1ca-821aa6c5d5af admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:33:58.742071 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e04c96cf-650e-43d4-a1ca-821aa6c5d5af admin admin] RBAC: Authorization granted {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:33:58.753294 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: [pid: 80377|app: 0|req: 1417/2820] 10.222.0.82 () {58 vars in 1122 bytes} [Tue Aug 2 10:33:58 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 02 10:33:58.757334 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cab7f7a0-90ff-4f6e-bbf4-6b732925262e admin admin] Authenticating user token {{(pid=80378) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:33:58.767240 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cab7f7a0-90ff-4f6e-bbf4-6b732925262e admin admin] Validating token access rules against request {{(pid=80378) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:33:58.771863 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-871fd09a-3f5b-4d38-9541-3cb8843bfa35 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '891005fa5ff74041b75747d16cc66798', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7047912c5acd4adeba5d87c3bf895e39', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80378) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:33:58.772470 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-871fd09a-3f5b-4d38-9541-3cb8843bfa35 admin admin] REQUEST_METHOD: `GET` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:33:58.772760 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-871fd09a-3f5b-4d38-9541-3cb8843bfa35 admin admin] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:33:58.773034 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-871fd09a-3f5b-4d38-9541-3cb8843bfa35 admin admin] PATH_INFO: `/v3/roles` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:33:58.773729 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-871fd09a-3f5b-4d38-9541-3cb8843bfa35 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:33:58.775351 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-871fd09a-3f5b-4d38-9541-3cb8843bfa35 admin admin] RBAC: Authorization granted {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:33:58.786230 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 1404/2821] 10.222.0.82 () {58 vars in 1122 bytes} [Tue Aug 2 10:33:58 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 02 10:33:58.790202 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e04c96cf-650e-43d4-a1ca-821aa6c5d5af admin admin] Authenticating user token {{(pid=80377) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:33:58.800712 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e04c96cf-650e-43d4-a1ca-821aa6c5d5af admin admin] Validating token access rules against request {{(pid=80377) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:33:58.805431 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d00cf80a-e368-43b2-a6a8-fd9fdb3c8ce8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '891005fa5ff74041b75747d16cc66798', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7047912c5acd4adeba5d87c3bf895e39', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80377) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:33:58.806120 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d00cf80a-e368-43b2-a6a8-fd9fdb3c8ce8 admin admin] REQUEST_METHOD: `PUT` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:33:58.806401 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d00cf80a-e368-43b2-a6a8-fd9fdb3c8ce8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:33:58.806702 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d00cf80a-e368-43b2-a6a8-fd9fdb3c8ce8 admin admin] PATH_INFO: `/v3/projects/aac48bec82654012b2d6dec95932571d/users/a04cb829a73441f6a5b24f08b73813c3/roles/e1c04b19fc4543b8b54f58bbcffbc598` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:33:58.821078 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d00cf80a-e368-43b2-a6a8-fd9fdb3c8ce8 admin admin] RBAC: Authorizing `identity:create_grant(project_id=aac48bec82654012b2d6dec95932571d, user_id=a04cb829a73441f6a5b24f08b73813c3, role_id=e1c04b19fc4543b8b54f58bbcffbc598)` {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:33:58.823737 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d00cf80a-e368-43b2-a6a8-fd9fdb3c8ce8 admin admin] RBAC: Authorization granted {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:33:58.836329 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: [pid: 80377|app: 0|req: 1418/2822] 10.222.0.82 () {60 vars in 1483 bytes} [Tue Aug 2 10:33:58 2022] PUT /identity/v3/projects/aac48bec82654012b2d6dec95932571d/users/a04cb829a73441f6a5b24f08b73813c3/roles/e1c04b19fc4543b8b54f58bbcffbc598 => generated 0 bytes in 47 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 02 10:33:58.848046 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc253055-c3b0-43e7-8511-58449c3f786e None None] REQUEST_METHOD: `POST` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:33:58.848337 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc253055-c3b0-43e7-8511-58449c3f786e None None] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:33:58.848608 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc253055-c3b0-43e7-8511-58449c3f786e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:33:58.876087 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.auth.core [None req-dc253055-c3b0-43e7-8511-58449c3f786e None None] MFA Rules not processed for user `a04cb829a73441f6a5b24f08b73813c3`. Rule list: `[]` (Enabled: `True`). {{(pid=80378) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 02 10:33:58.877214 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.fernet_utils [None req-dc253055-c3b0-43e7-8511-58449c3f786e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80378) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 10:33:58.938529 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 1405/2823] 10.222.0.82 () {58 vars in 958 bytes} [Tue Aug 2 10:33:58 2022] POST /identity/v3/auth/tokens => generated 3242 bytes in 92 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 02 10:33:58.943274 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d00cf80a-e368-43b2-a6a8-fd9fdb3c8ce8 admin admin] Authenticating user token {{(pid=80377) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:33:59.006082 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d00cf80a-e368-43b2-a6a8-fd9fdb3c8ce8 admin admin] Validating token access rules against request {{(pid=80377) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:33:59.010712 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80ad43ae-9e34-4d6a-8d4d-b96f93deb8f6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '891005fa5ff74041b75747d16cc66798', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7047912c5acd4adeba5d87c3bf895e39', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80377) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:33:59.011327 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80ad43ae-9e34-4d6a-8d4d-b96f93deb8f6 admin admin] REQUEST_METHOD: `POST` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:33:59.011609 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80ad43ae-9e34-4d6a-8d4d-b96f93deb8f6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:33:59.011883 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80ad43ae-9e34-4d6a-8d4d-b96f93deb8f6 admin admin] PATH_INFO: `/v3/projects` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:33:59.012543 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80ad43ae-9e34-4d6a-8d4d-b96f93deb8f6 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:33:59.014475 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80ad43ae-9e34-4d6a-8d4d-b96f93deb8f6 admin admin] RBAC: Authorization granted {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:33:59.044484 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: [pid: 80377|app: 0|req: 1419/2824] 10.222.0.82 () {60 vars in 1153 bytes} [Tue Aug 2 10:33:58 2022] POST /identity/v3/projects => generated 363 bytes in 102 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 02 10:33:59.048616 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc253055-c3b0-43e7-8511-58449c3f786e None None] Authenticating user token {{(pid=80378) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:33:59.059207 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc253055-c3b0-43e7-8511-58449c3f786e None None] Validating token access rules against request {{(pid=80378) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:33:59.114900 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef75d148-2dbf-4fed-956b-8bbc859ae5d9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '891005fa5ff74041b75747d16cc66798', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7047912c5acd4adeba5d87c3bf895e39', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80378) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:33:59.115518 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef75d148-2dbf-4fed-956b-8bbc859ae5d9 admin admin] REQUEST_METHOD: `POST` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:33:59.115806 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef75d148-2dbf-4fed-956b-8bbc859ae5d9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:33:59.116078 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef75d148-2dbf-4fed-956b-8bbc859ae5d9 admin admin] PATH_INFO: `/v3/users` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:33:59.116731 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef75d148-2dbf-4fed-956b-8bbc859ae5d9 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:33:59.118677 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef75d148-2dbf-4fed-956b-8bbc859ae5d9 admin admin] RBAC: Authorization granted {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:33:59.140109 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 1406/2825] 10.222.0.82 () {60 vars in 1144 bytes} [Tue Aug 2 10:33:59 2022] POST /identity/v3/users => generated 288 bytes in 92 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 02 10:33:59.144595 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80ad43ae-9e34-4d6a-8d4d-b96f93deb8f6 admin admin] Authenticating user token {{(pid=80377) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:33:59.154698 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80ad43ae-9e34-4d6a-8d4d-b96f93deb8f6 admin admin] Validating token access rules against request {{(pid=80377) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:33:59.159318 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74eadc17-9571-409b-9a13-a8ca80fa2b13 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '891005fa5ff74041b75747d16cc66798', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7047912c5acd4adeba5d87c3bf895e39', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80377) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:33:59.160508 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74eadc17-9571-409b-9a13-a8ca80fa2b13 admin admin] REQUEST_METHOD: `GET` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:33:59.160790 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74eadc17-9571-409b-9a13-a8ca80fa2b13 admin admin] SCRIPT_NAME: `/identity` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:33:59.161067 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74eadc17-9571-409b-9a13-a8ca80fa2b13 admin admin] PATH_INFO: `/v3/roles` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:33:59.161759 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74eadc17-9571-409b-9a13-a8ca80fa2b13 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:33:59.163371 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74eadc17-9571-409b-9a13-a8ca80fa2b13 admin admin] RBAC: Authorization granted {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:33:59.173691 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: [pid: 80377|app: 0|req: 1420/2826] 10.222.0.82 () {58 vars in 1122 bytes} [Tue Aug 2 10:33:59 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 02 10:33:59.177814 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef75d148-2dbf-4fed-956b-8bbc859ae5d9 admin admin] Authenticating user token {{(pid=80378) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:33:59.187985 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef75d148-2dbf-4fed-956b-8bbc859ae5d9 admin admin] Validating token access rules against request {{(pid=80378) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:33:59.192932 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-233b488e-0ed2-4fdf-bf2f-e83cda5e95fb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '891005fa5ff74041b75747d16cc66798', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7047912c5acd4adeba5d87c3bf895e39', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80378) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:33:59.193651 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-233b488e-0ed2-4fdf-bf2f-e83cda5e95fb admin admin] REQUEST_METHOD: `PUT` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:33:59.193963 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-233b488e-0ed2-4fdf-bf2f-e83cda5e95fb admin admin] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:33:59.194242 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-233b488e-0ed2-4fdf-bf2f-e83cda5e95fb admin admin] PATH_INFO: `/v3/projects/c9c5706fc0d046d9ba129baf5a6604d9/users/255aa8f483d14187a3c73ed70994eea1/roles/237966442f05411191105bf336bc14f3` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:33:59.210789 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-233b488e-0ed2-4fdf-bf2f-e83cda5e95fb admin admin] RBAC: Authorizing `identity:create_grant(project_id=c9c5706fc0d046d9ba129baf5a6604d9, user_id=255aa8f483d14187a3c73ed70994eea1, role_id=237966442f05411191105bf336bc14f3)` {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:33:59.213466 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-233b488e-0ed2-4fdf-bf2f-e83cda5e95fb admin admin] RBAC: Authorization granted {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:33:59.226354 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 1407/2827] 10.222.0.82 () {60 vars in 1483 bytes} [Tue Aug 2 10:33:59 2022] PUT /identity/v3/projects/c9c5706fc0d046d9ba129baf5a6604d9/users/255aa8f483d14187a3c73ed70994eea1/roles/237966442f05411191105bf336bc14f3 => generated 0 bytes in 49 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 02 10:33:59.230268 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74eadc17-9571-409b-9a13-a8ca80fa2b13 admin admin] Authenticating user token {{(pid=80377) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:33:59.240715 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74eadc17-9571-409b-9a13-a8ca80fa2b13 admin admin] Validating token access rules against request {{(pid=80377) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:33:59.260377 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG dbcounter [-] [80377] Writing DB stats keystone:DELETE=6,keystone:SELECT=90,keystone:INSERT=4 {{(pid=80377) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 10:33:59.298146 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7d7a877-e18e-4803-b47b-faffbf900bab admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '891005fa5ff74041b75747d16cc66798', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7047912c5acd4adeba5d87c3bf895e39', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80377) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:33:59.298761 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7d7a877-e18e-4803-b47b-faffbf900bab admin admin] REQUEST_METHOD: `GET` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:33:59.299043 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7d7a877-e18e-4803-b47b-faffbf900bab admin admin] SCRIPT_NAME: `/identity` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:33:59.299318 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7d7a877-e18e-4803-b47b-faffbf900bab admin admin] PATH_INFO: `/v3/roles` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:33:59.300048 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7d7a877-e18e-4803-b47b-faffbf900bab admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:33:59.301640 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7d7a877-e18e-4803-b47b-faffbf900bab admin admin] RBAC: Authorization granted {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:33:59.311229 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: [pid: 80377|app: 0|req: 1421/2828] 10.222.0.82 () {58 vars in 1122 bytes} [Tue Aug 2 10:33:59 2022] GET /identity/v3/roles => generated 1420 bytes in 81 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 02 10:33:59.315294 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-233b488e-0ed2-4fdf-bf2f-e83cda5e95fb admin admin] Authenticating user token {{(pid=80378) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:33:59.320938 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG dbcounter [-] [80378] Writing DB stats keystone:SELECT=93,keystone:INSERT=7 {{(pid=80378) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 10:33:59.327375 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-233b488e-0ed2-4fdf-bf2f-e83cda5e95fb admin admin] Validating token access rules against request {{(pid=80378) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:33:59.333061 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24ef73bd-e969-4d09-af99-8b025e0d3251 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '891005fa5ff74041b75747d16cc66798', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7047912c5acd4adeba5d87c3bf895e39', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80378) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:33:59.333727 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24ef73bd-e969-4d09-af99-8b025e0d3251 admin admin] REQUEST_METHOD: `PUT` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:33:59.334055 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24ef73bd-e969-4d09-af99-8b025e0d3251 admin admin] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:33:59.334334 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24ef73bd-e969-4d09-af99-8b025e0d3251 admin admin] PATH_INFO: `/v3/projects/c9c5706fc0d046d9ba129baf5a6604d9/users/255aa8f483d14187a3c73ed70994eea1/roles/237966442f05411191105bf336bc14f3` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:33:59.336394 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24ef73bd-e969-4d09-af99-8b025e0d3251 admin admin] RBAC: Authorizing `identity:create_grant(project_id=c9c5706fc0d046d9ba129baf5a6604d9, user_id=255aa8f483d14187a3c73ed70994eea1, role_id=237966442f05411191105bf336bc14f3)` {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:33:59.339271 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24ef73bd-e969-4d09-af99-8b025e0d3251 admin admin] RBAC: Authorization granted {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:33:59.350068 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 1408/2829] 10.222.0.82 () {60 vars in 1483 bytes} [Tue Aug 2 10:33:59 2022] PUT /identity/v3/projects/c9c5706fc0d046d9ba129baf5a6604d9/users/255aa8f483d14187a3c73ed70994eea1/roles/237966442f05411191105bf336bc14f3 => generated 0 bytes in 35 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 02 10:33:59.360180 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e4e1658-ba30-4a83-bd19-17d3779eaa59 None None] REQUEST_METHOD: `POST` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:33:59.360661 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e4e1658-ba30-4a83-bd19-17d3779eaa59 None None] SCRIPT_NAME: `/identity` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:33:59.360802 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e4e1658-ba30-4a83-bd19-17d3779eaa59 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:33:59.388076 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.auth.core [None req-5e4e1658-ba30-4a83-bd19-17d3779eaa59 None None] MFA Rules not processed for user `255aa8f483d14187a3c73ed70994eea1`. Rule list: `[]` (Enabled: `True`). {{(pid=80377) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 02 10:33:59.389277 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.fernet_utils [None req-5e4e1658-ba30-4a83-bd19-17d3779eaa59 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80377) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 10:33:59.455485 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: [pid: 80377|app: 0|req: 1422/2830] 10.222.0.82 () {58 vars in 958 bytes} [Tue Aug 2 10:33:59 2022] POST /identity/v3/auth/tokens => generated 3307 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 02 10:33:59.468197 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24ef73bd-e969-4d09-af99-8b025e0d3251 admin admin] Authenticating user token {{(pid=80378) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:33:59.470040 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.fernet_utils [None req-24ef73bd-e969-4d09-af99-8b025e0d3251 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80378) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 10:33:59.526378 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24ef73bd-e969-4d09-af99-8b025e0d3251 admin admin] Validating token access rules against request {{(pid=80378) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:33:59.529942 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc4879b0-7bd7-45c8-9473-0e1800e7457c service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1023de281ae74e9d8a6d6c312c1c97a7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f1b38ebb804c41c69772c998aae72c41', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80378) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:33:59.530543 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc4879b0-7bd7-45c8-9473-0e1800e7457c service cinder] REQUEST_METHOD: `GET` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:33:59.530821 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc4879b0-7bd7-45c8-9473-0e1800e7457c service cinder] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:33:59.531091 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc4879b0-7bd7-45c8-9473-0e1800e7457c service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:33:59.539967 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc4879b0-7bd7-45c8-9473-0e1800e7457c service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:33:59.541376 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc4879b0-7bd7-45c8-9473-0e1800e7457c service cinder] RBAC: Authorization granted {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:33:59.550682 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 1409/2831] 10.222.0.82 () {60 vars in 1365 bytes} [Tue Aug 2 10:33:59 2022] GET /identity/v3/auth/tokens => generated 3307 bytes in 83 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 02 10:33:59.626520 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e4e1658-ba30-4a83-bd19-17d3779eaa59 None None] Authenticating user token {{(pid=80377) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:33:59.636808 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e4e1658-ba30-4a83-bd19-17d3779eaa59 None None] Validating token access rules against request {{(pid=80377) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:33:59.640498 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c7e7a30-33dd-484e-b942-742911b86b3f service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1023de281ae74e9d8a6d6c312c1c97a7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f1b38ebb804c41c69772c998aae72c41', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80377) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:33:59.641113 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c7e7a30-33dd-484e-b942-742911b86b3f service cinder] REQUEST_METHOD: `GET` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:33:59.641399 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c7e7a30-33dd-484e-b942-742911b86b3f service cinder] SCRIPT_NAME: `/identity` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:33:59.641677 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c7e7a30-33dd-484e-b942-742911b86b3f service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:33:59.698173 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c7e7a30-33dd-484e-b942-742911b86b3f service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:33:59.699632 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c7e7a30-33dd-484e-b942-742911b86b3f service cinder] RBAC: Authorization granted {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:33:59.709100 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: [pid: 80377|app: 0|req: 1423/2832] 10.222.0.82 () {60 vars in 1365 bytes} [Tue Aug 2 10:33:59 2022] GET /identity/v3/auth/tokens => generated 3242 bytes in 83 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 02 10:34:09.546504 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG dbcounter [-] [80378] Writing DB stats keystone:SELECT=30,keystone:INSERT=1 {{(pid=80378) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 10:34:09.704367 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG dbcounter [-] [80377] Writing DB stats keystone:SELECT=80 {{(pid=80377) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 10:34:10.982124 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc4879b0-7bd7-45c8-9473-0e1800e7457c service cinder] Authenticating user token {{(pid=80378) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:34:11.043183 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc4879b0-7bd7-45c8-9473-0e1800e7457c service cinder] Validating token access rules against request {{(pid=80378) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:34:11.047587 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25ae693d-9602-4192-8e62-25fb2889cc6f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '891005fa5ff74041b75747d16cc66798', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7047912c5acd4adeba5d87c3bf895e39', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80378) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:34:11.048333 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25ae693d-9602-4192-8e62-25fb2889cc6f admin admin] REQUEST_METHOD: `DELETE` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:34:11.048574 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25ae693d-9602-4192-8e62-25fb2889cc6f admin admin] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:34:11.048924 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25ae693d-9602-4192-8e62-25fb2889cc6f admin admin] PATH_INFO: `/v3/users/a04cb829a73441f6a5b24f08b73813c3` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:34:11.050655 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25ae693d-9602-4192-8e62-25fb2889cc6f admin admin] RBAC: Authorizing `identity:delete_user(user_id=a04cb829a73441f6a5b24f08b73813c3)` {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:34:11.052584 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25ae693d-9602-4192-8e62-25fb2889cc6f admin admin] RBAC: Authorization granted {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:34:11.099076 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.notifications [None req-25ae693d-9602-4192-8e62-25fb2889cc6f admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'a04cb829a73441f6a5b24f08b73813c3'} {{(pid=80378) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 10:34:11.106607 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.notifications [None req-25ae693d-9602-4192-8e62-25fb2889cc6f admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'a04cb829a73441f6a5b24f08b73813c3'} {{(pid=80378) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 10:34:11.114130 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.notifications [None req-25ae693d-9602-4192-8e62-25fb2889cc6f admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'a04cb829a73441f6a5b24f08b73813c3'} {{(pid=80378) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 10:34:11.114777 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.notifications [None req-25ae693d-9602-4192-8e62-25fb2889cc6f admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'a04cb829a73441f6a5b24f08b73813c3'} {{(pid=80378) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 10:34:11.127651 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.notifications [None req-25ae693d-9602-4192-8e62-25fb2889cc6f admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'a04cb829a73441f6a5b24f08b73813c3'} {{(pid=80378) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 10:34:11.136315 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 1410/2833] 10.222.0.82 () {58 vars in 1224 bytes} [Tue Aug 2 10:34:10 2022] DELETE /identity/v3/users/a04cb829a73441f6a5b24f08b73813c3 => generated 0 bytes in 155 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 02 10:34:11.140839 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b8d4f7e-f44a-4d0a-8512-4dd45350af09 None None] REQUEST_METHOD: `POST` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:34:11.141357 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b8d4f7e-f44a-4d0a-8512-4dd45350af09 None None] SCRIPT_NAME: `/identity` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:34:11.141471 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b8d4f7e-f44a-4d0a-8512-4dd45350af09 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:34:11.161108 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.auth.core [None req-9b8d4f7e-f44a-4d0a-8512-4dd45350af09 None None] MFA Rules not processed for user `891005fa5ff74041b75747d16cc66798`. Rule list: `[]` (Enabled: `True`). {{(pid=80377) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 02 10:34:11.162357 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.fernet_utils [None req-9b8d4f7e-f44a-4d0a-8512-4dd45350af09 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80377) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 10:34:11.223083 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: [pid: 80377|app: 0|req: 1424/2834] 10.222.0.82 () {58 vars in 958 bytes} [Tue Aug 2 10:34:11 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 83 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 02 10:34:11.232792 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25ae693d-9602-4192-8e62-25fb2889cc6f admin admin] Authenticating user token {{(pid=80378) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:34:11.234215 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.fernet_utils [None req-25ae693d-9602-4192-8e62-25fb2889cc6f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80378) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 10:34:11.290830 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25ae693d-9602-4192-8e62-25fb2889cc6f admin admin] Validating token access rules against request {{(pid=80378) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:34:11.294549 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-417e9a59-6a8f-4a1c-a1e2-afb8a2d843ea service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '33b87170f0634b67bce60750847b5cbe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f1b38ebb804c41c69772c998aae72c41', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80378) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:34:11.295112 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-417e9a59-6a8f-4a1c-a1e2-afb8a2d843ea service neutron] REQUEST_METHOD: `GET` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:34:11.295397 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-417e9a59-6a8f-4a1c-a1e2-afb8a2d843ea service neutron] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:34:11.295673 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-417e9a59-6a8f-4a1c-a1e2-afb8a2d843ea service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:34:11.305037 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-417e9a59-6a8f-4a1c-a1e2-afb8a2d843ea service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:34:11.306564 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-417e9a59-6a8f-4a1c-a1e2-afb8a2d843ea service neutron] RBAC: Authorization granted {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:34:11.316859 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 1411/2835] 10.222.0.82 () {60 vars in 1365 bytes} [Tue Aug 2 10:34:11 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 85 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 02 10:34:11.582809 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b8d4f7e-f44a-4d0a-8512-4dd45350af09 None None] Authenticating user token {{(pid=80377) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:34:11.584235 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.fernet_utils [None req-9b8d4f7e-f44a-4d0a-8512-4dd45350af09 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80377) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 10:34:11.596362 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b8d4f7e-f44a-4d0a-8512-4dd45350af09 None None] Validating token access rules against request {{(pid=80377) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:34:11.601600 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37f9aedd-f425-4c27-b132-6fcef6f71950 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '891005fa5ff74041b75747d16cc66798', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7047912c5acd4adeba5d87c3bf895e39', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80377) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:34:11.602279 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37f9aedd-f425-4c27-b132-6fcef6f71950 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:34:11.602562 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37f9aedd-f425-4c27-b132-6fcef6f71950 admin admin] SCRIPT_NAME: `/identity` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:34:11.602841 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37f9aedd-f425-4c27-b132-6fcef6f71950 admin admin] PATH_INFO: `/v3/projects/aac48bec82654012b2d6dec95932571d` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:34:11.604008 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37f9aedd-f425-4c27-b132-6fcef6f71950 admin admin] RBAC: Authorizing `identity:delete_project(project_id=aac48bec82654012b2d6dec95932571d)` {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:34:11.605967 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37f9aedd-f425-4c27-b132-6fcef6f71950 admin admin] RBAC: Authorization granted {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:34:11.652898 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.notifications [None req-37f9aedd-f425-4c27-b132-6fcef6f71950 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'aac48bec82654012b2d6dec95932571d', 'request_id': 'req-37f9aedd-f425-4c27-b132-6fcef6f71950'} {{(pid=80377) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 10:34:11.658323 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.notifications [None req-37f9aedd-f425-4c27-b132-6fcef6f71950 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'aac48bec82654012b2d6dec95932571d', 'request_id': 'req-37f9aedd-f425-4c27-b132-6fcef6f71950'} {{(pid=80377) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 10:34:11.665609 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.notifications [None req-37f9aedd-f425-4c27-b132-6fcef6f71950 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'aac48bec82654012b2d6dec95932571d', 'request_id': 'req-37f9aedd-f425-4c27-b132-6fcef6f71950'} {{(pid=80377) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 10:34:11.678380 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.notifications [None req-37f9aedd-f425-4c27-b132-6fcef6f71950 admin admin] The token cache is being invalidate because project aac48bec82654012b2d6dec95932571d was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80377) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 02 10:34:11.678670 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.notifications [None req-37f9aedd-f425-4c27-b132-6fcef6f71950 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80377) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 10:34:11.679294 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.notifications [None req-37f9aedd-f425-4c27-b132-6fcef6f71950 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80377) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 10:34:11.680799 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: [pid: 80377|app: 0|req: 1425/2836] 10.222.0.82 () {58 vars in 1233 bytes} [Tue Aug 2 10:34:11 2022] DELETE /identity/v3/projects/aac48bec82654012b2d6dec95932571d => generated 0 bytes in 99 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 02 10:34:11.684761 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-417e9a59-6a8f-4a1c-a1e2-afb8a2d843ea service neutron] Authenticating user token {{(pid=80378) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:34:11.694636 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-417e9a59-6a8f-4a1c-a1e2-afb8a2d843ea service neutron] Validating token access rules against request {{(pid=80378) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:34:11.711745 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG dbcounter [-] [80378] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80378) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 10:34:11.752913 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f05f814a-f181-4e90-8e83-839513df1bca admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '891005fa5ff74041b75747d16cc66798', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7047912c5acd4adeba5d87c3bf895e39', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80378) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:34:11.753551 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f05f814a-f181-4e90-8e83-839513df1bca admin admin] REQUEST_METHOD: `DELETE` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:34:11.753864 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f05f814a-f181-4e90-8e83-839513df1bca admin admin] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:34:11.754138 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f05f814a-f181-4e90-8e83-839513df1bca admin admin] PATH_INFO: `/v3/users/255aa8f483d14187a3c73ed70994eea1` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:34:11.755317 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f05f814a-f181-4e90-8e83-839513df1bca admin admin] RBAC: Authorizing `identity:delete_user(user_id=255aa8f483d14187a3c73ed70994eea1)` {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:34:11.757254 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f05f814a-f181-4e90-8e83-839513df1bca admin admin] RBAC: Authorization granted {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:34:11.802939 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.notifications [None req-f05f814a-f181-4e90-8e83-839513df1bca admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '255aa8f483d14187a3c73ed70994eea1'} {{(pid=80378) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 10:34:11.810036 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.notifications [None req-f05f814a-f181-4e90-8e83-839513df1bca admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '255aa8f483d14187a3c73ed70994eea1'} {{(pid=80378) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 10:34:11.817153 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.notifications [None req-f05f814a-f181-4e90-8e83-839513df1bca admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '255aa8f483d14187a3c73ed70994eea1'} {{(pid=80378) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 10:34:11.817817 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.notifications [None req-f05f814a-f181-4e90-8e83-839513df1bca admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '255aa8f483d14187a3c73ed70994eea1'} {{(pid=80378) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 10:34:11.830039 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.notifications [None req-f05f814a-f181-4e90-8e83-839513df1bca admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '255aa8f483d14187a3c73ed70994eea1'} {{(pid=80378) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 10:34:11.836360 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 1412/2837] 10.222.0.82 () {58 vars in 1224 bytes} [Tue Aug 2 10:34:11 2022] DELETE /identity/v3/users/255aa8f483d14187a3c73ed70994eea1 => generated 0 bytes in 152 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 02 10:34:12.109921 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37f9aedd-f425-4c27-b132-6fcef6f71950 admin admin] Authenticating user token {{(pid=80377) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:34:12.111385 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.fernet_utils [None req-37f9aedd-f425-4c27-b132-6fcef6f71950 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80377) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 10:34:12.124934 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37f9aedd-f425-4c27-b132-6fcef6f71950 admin admin] Validating token access rules against request {{(pid=80377) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:34:12.182343 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7266f01c-8a66-4711-85bc-1663c40a9bc7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '891005fa5ff74041b75747d16cc66798', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7047912c5acd4adeba5d87c3bf895e39', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80377) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:34:12.183032 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7266f01c-8a66-4711-85bc-1663c40a9bc7 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:34:12.183260 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7266f01c-8a66-4711-85bc-1663c40a9bc7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:34:12.183535 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7266f01c-8a66-4711-85bc-1663c40a9bc7 admin admin] PATH_INFO: `/v3/projects/c9c5706fc0d046d9ba129baf5a6604d9` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:34:12.184704 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7266f01c-8a66-4711-85bc-1663c40a9bc7 admin admin] RBAC: Authorizing `identity:delete_project(project_id=c9c5706fc0d046d9ba129baf5a6604d9)` {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:34:12.186660 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7266f01c-8a66-4711-85bc-1663c40a9bc7 admin admin] RBAC: Authorization granted {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:34:12.218458 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG dbcounter [-] [80377] Writing DB stats keystone:SELECT=91,keystone:DELETE=8,keystone:INSERT=1 {{(pid=80377) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 10:34:12.235764 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.notifications [None req-7266f01c-8a66-4711-85bc-1663c40a9bc7 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'c9c5706fc0d046d9ba129baf5a6604d9', 'request_id': 'req-7266f01c-8a66-4711-85bc-1663c40a9bc7'} {{(pid=80377) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 10:34:12.241498 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.notifications [None req-7266f01c-8a66-4711-85bc-1663c40a9bc7 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'c9c5706fc0d046d9ba129baf5a6604d9', 'request_id': 'req-7266f01c-8a66-4711-85bc-1663c40a9bc7'} {{(pid=80377) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 10:34:12.247781 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.notifications [None req-7266f01c-8a66-4711-85bc-1663c40a9bc7 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'c9c5706fc0d046d9ba129baf5a6604d9', 'request_id': 'req-7266f01c-8a66-4711-85bc-1663c40a9bc7'} {{(pid=80377) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 10:34:12.261994 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.notifications [None req-7266f01c-8a66-4711-85bc-1663c40a9bc7 admin admin] The token cache is being invalidate because project c9c5706fc0d046d9ba129baf5a6604d9 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80377) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 02 10:34:12.262287 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.notifications [None req-7266f01c-8a66-4711-85bc-1663c40a9bc7 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80377) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 10:34:12.262927 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.notifications [None req-7266f01c-8a66-4711-85bc-1663c40a9bc7 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80377) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 10:34:12.264441 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: [pid: 80377|app: 0|req: 1426/2838] 10.222.0.82 () {58 vars in 1233 bytes} [Tue Aug 2 10:34:12 2022] DELETE /identity/v3/projects/c9c5706fc0d046d9ba129baf5a6604d9 => generated 0 bytes in 155 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 02 10:34:12.269849 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9916f944-e9e8-47cd-a9dc-26ec48979ee7 None None] REQUEST_METHOD: `POST` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:34:12.270341 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9916f944-e9e8-47cd-a9dc-26ec48979ee7 None None] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:34:12.270457 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9916f944-e9e8-47cd-a9dc-26ec48979ee7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:34:12.289916 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.auth.core [None req-9916f944-e9e8-47cd-a9dc-26ec48979ee7 None None] MFA Rules not processed for user `891005fa5ff74041b75747d16cc66798`. Rule list: `[]` (Enabled: `True`). {{(pid=80378) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 02 10:34:12.291099 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.fernet_utils [None req-9916f944-e9e8-47cd-a9dc-26ec48979ee7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80378) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 10:34:12.349896 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 1413/2839] 10.222.0.82 () {58 vars in 958 bytes} [Tue Aug 2 10:34:12 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 82 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 02 10:34:12.357826 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3462f6af-04fe-4f1c-bda0-afc24c258749 None None] REQUEST_METHOD: `POST` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:34:12.358163 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3462f6af-04fe-4f1c-bda0-afc24c258749 None None] SCRIPT_NAME: `/identity` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:34:12.358445 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3462f6af-04fe-4f1c-bda0-afc24c258749 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:34:12.378522 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.auth.core [None req-3462f6af-04fe-4f1c-bda0-afc24c258749 None None] MFA Rules not processed for user `891005fa5ff74041b75747d16cc66798`. Rule list: `[]` (Enabled: `True`). {{(pid=80377) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 02 10:34:12.379723 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.fernet_utils [None req-3462f6af-04fe-4f1c-bda0-afc24c258749 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80377) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 10:34:12.388730 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: [pid: 80377|app: 0|req: 1427/2840] 10.222.0.82 () {58 vars in 958 bytes} [Tue Aug 2 10:34:12 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 02 10:34:12.392911 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9916f944-e9e8-47cd-a9dc-26ec48979ee7 None None] Authenticating user token {{(pid=80378) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:34:12.402886 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9916f944-e9e8-47cd-a9dc-26ec48979ee7 None None] Validating token access rules against request {{(pid=80378) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:34:12.407360 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c22ceff8-ef16-4876-9f63-ec74ae5290c3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '891005fa5ff74041b75747d16cc66798', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7047912c5acd4adeba5d87c3bf895e39', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80378) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:34:12.407961 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c22ceff8-ef16-4876-9f63-ec74ae5290c3 admin admin] REQUEST_METHOD: `GET` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:34:12.408242 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c22ceff8-ef16-4876-9f63-ec74ae5290c3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:34:12.408514 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c22ceff8-ef16-4876-9f63-ec74ae5290c3 admin admin] PATH_INFO: `/v3/domains` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:34:12.409200 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c22ceff8-ef16-4876-9f63-ec74ae5290c3 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80378) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 02 10:34:12.409531 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c22ceff8-ef16-4876-9f63-ec74ae5290c3 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:34:12.411177 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c22ceff8-ef16-4876-9f63-ec74ae5290c3 admin admin] RBAC: Authorization granted {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:34:12.422949 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 1414/2841] 10.222.0.82 () {58 vars in 1153 bytes} [Tue Aug 2 10:34:12 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 02 10:34:12.427166 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3462f6af-04fe-4f1c-bda0-afc24c258749 None None] Authenticating user token {{(pid=80377) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:34:12.436375 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3462f6af-04fe-4f1c-bda0-afc24c258749 None None] Validating token access rules against request {{(pid=80377) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:34:12.441870 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf2f4a87-975b-4312-8dda-a174312588d1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '891005fa5ff74041b75747d16cc66798', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7047912c5acd4adeba5d87c3bf895e39', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80377) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:34:12.442470 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf2f4a87-975b-4312-8dda-a174312588d1 admin admin] REQUEST_METHOD: `POST` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:34:12.442764 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf2f4a87-975b-4312-8dda-a174312588d1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:34:12.443039 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf2f4a87-975b-4312-8dda-a174312588d1 admin admin] PATH_INFO: `/v3/projects` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:34:12.443696 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf2f4a87-975b-4312-8dda-a174312588d1 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:34:12.445575 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf2f4a87-975b-4312-8dda-a174312588d1 admin admin] RBAC: Authorization granted {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:34:12.473643 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: [pid: 80377|app: 0|req: 1428/2842] 10.222.0.82 () {60 vars in 1153 bytes} [Tue Aug 2 10:34:12 2022] POST /identity/v3/projects => generated 365 bytes in 47 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 02 10:34:12.477703 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c22ceff8-ef16-4876-9f63-ec74ae5290c3 admin admin] Authenticating user token {{(pid=80378) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:34:12.487753 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c22ceff8-ef16-4876-9f63-ec74ae5290c3 admin admin] Validating token access rules against request {{(pid=80378) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:34:12.513016 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG dbcounter [-] [80378] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80378) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 10:34:12.550750 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a0a07c1-b566-4d78-9a65-19f49c1b5230 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '891005fa5ff74041b75747d16cc66798', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7047912c5acd4adeba5d87c3bf895e39', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80378) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:34:12.551374 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a0a07c1-b566-4d78-9a65-19f49c1b5230 admin admin] REQUEST_METHOD: `POST` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:34:12.551658 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a0a07c1-b566-4d78-9a65-19f49c1b5230 admin admin] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:34:12.551936 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a0a07c1-b566-4d78-9a65-19f49c1b5230 admin admin] PATH_INFO: `/v3/users` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:34:12.552622 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a0a07c1-b566-4d78-9a65-19f49c1b5230 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:34:12.554571 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a0a07c1-b566-4d78-9a65-19f49c1b5230 admin admin] RBAC: Authorization granted {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:34:12.575843 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 1415/2843] 10.222.0.82 () {60 vars in 1144 bytes} [Tue Aug 2 10:34:12 2022] POST /identity/v3/users => generated 283 bytes in 99 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 02 10:34:12.579969 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf2f4a87-975b-4312-8dda-a174312588d1 admin admin] Authenticating user token {{(pid=80377) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:34:12.590183 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf2f4a87-975b-4312-8dda-a174312588d1 admin admin] Validating token access rules against request {{(pid=80377) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:34:12.594833 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-545d513c-4966-405a-b178-485765691aef admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '891005fa5ff74041b75747d16cc66798', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7047912c5acd4adeba5d87c3bf895e39', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80377) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:34:12.595442 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-545d513c-4966-405a-b178-485765691aef admin admin] REQUEST_METHOD: `GET` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:34:12.595726 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-545d513c-4966-405a-b178-485765691aef admin admin] SCRIPT_NAME: `/identity` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:34:12.596002 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-545d513c-4966-405a-b178-485765691aef admin admin] PATH_INFO: `/v3/roles` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:34:12.596722 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-545d513c-4966-405a-b178-485765691aef admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:34:12.598362 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-545d513c-4966-405a-b178-485765691aef admin admin] RBAC: Authorization granted {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:34:12.608202 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: [pid: 80377|app: 0|req: 1429/2844] 10.222.0.82 () {58 vars in 1122 bytes} [Tue Aug 2 10:34:12 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 02 10:34:12.611823 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a0a07c1-b566-4d78-9a65-19f49c1b5230 admin admin] Authenticating user token {{(pid=80378) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:34:12.622073 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a0a07c1-b566-4d78-9a65-19f49c1b5230 admin admin] Validating token access rules against request {{(pid=80378) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:34:12.626668 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b24083e-ede8-4c92-a392-7f27941537da admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '891005fa5ff74041b75747d16cc66798', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7047912c5acd4adeba5d87c3bf895e39', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80378) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:34:12.627263 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b24083e-ede8-4c92-a392-7f27941537da admin admin] REQUEST_METHOD: `GET` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:34:12.627550 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b24083e-ede8-4c92-a392-7f27941537da admin admin] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:34:12.628388 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b24083e-ede8-4c92-a392-7f27941537da admin admin] PATH_INFO: `/v3/roles` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:34:12.629071 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b24083e-ede8-4c92-a392-7f27941537da admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:34:12.630709 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b24083e-ede8-4c92-a392-7f27941537da admin admin] RBAC: Authorization granted {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:34:12.641079 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 1416/2845] 10.222.0.82 () {58 vars in 1122 bytes} [Tue Aug 2 10:34:12 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 02 10:34:12.645668 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-545d513c-4966-405a-b178-485765691aef admin admin] Authenticating user token {{(pid=80377) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:34:12.656244 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-545d513c-4966-405a-b178-485765691aef admin admin] Validating token access rules against request {{(pid=80377) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:34:12.661474 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b2c631b-48b5-4026-952e-d10fd915857a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '891005fa5ff74041b75747d16cc66798', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7047912c5acd4adeba5d87c3bf895e39', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80377) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:34:12.662267 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b2c631b-48b5-4026-952e-d10fd915857a admin admin] REQUEST_METHOD: `PUT` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:34:12.662554 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b2c631b-48b5-4026-952e-d10fd915857a admin admin] SCRIPT_NAME: `/identity` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:34:12.662836 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b2c631b-48b5-4026-952e-d10fd915857a admin admin] PATH_INFO: `/v3/projects/c274ce0e9703454594fc62a1ff33a030/users/14b83284af034be38a30ba54867ad0c2/roles/e1c04b19fc4543b8b54f58bbcffbc598` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:34:12.677306 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b2c631b-48b5-4026-952e-d10fd915857a admin admin] RBAC: Authorizing `identity:create_grant(project_id=c274ce0e9703454594fc62a1ff33a030, user_id=14b83284af034be38a30ba54867ad0c2, role_id=e1c04b19fc4543b8b54f58bbcffbc598)` {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:34:12.680025 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b2c631b-48b5-4026-952e-d10fd915857a admin admin] RBAC: Authorization granted {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:34:12.692635 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: [pid: 80377|app: 0|req: 1430/2846] 10.222.0.82 () {60 vars in 1483 bytes} [Tue Aug 2 10:34:12 2022] PUT /identity/v3/projects/c274ce0e9703454594fc62a1ff33a030/users/14b83284af034be38a30ba54867ad0c2/roles/e1c04b19fc4543b8b54f58bbcffbc598 => generated 0 bytes in 47 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 02 10:34:12.704039 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1094a50-ff7d-4a4d-b3ac-b98c1035fe13 None None] REQUEST_METHOD: `POST` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:34:12.704334 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1094a50-ff7d-4a4d-b3ac-b98c1035fe13 None None] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:34:12.704613 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1094a50-ff7d-4a4d-b3ac-b98c1035fe13 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:34:12.733978 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.auth.core [None req-c1094a50-ff7d-4a4d-b3ac-b98c1035fe13 None None] MFA Rules not processed for user `14b83284af034be38a30ba54867ad0c2`. Rule list: `[]` (Enabled: `True`). {{(pid=80378) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 02 10:34:12.735228 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.fernet_utils [None req-c1094a50-ff7d-4a4d-b3ac-b98c1035fe13 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80378) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 10:34:12.798248 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 1417/2847] 10.222.0.82 () {58 vars in 958 bytes} [Tue Aug 2 10:34:12 2022] POST /identity/v3/auth/tokens => generated 3242 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 02 10:34:12.802717 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b2c631b-48b5-4026-952e-d10fd915857a admin admin] Authenticating user token {{(pid=80377) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:34:12.864388 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b2c631b-48b5-4026-952e-d10fd915857a admin admin] Validating token access rules against request {{(pid=80377) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:34:12.868985 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6090255-ab0b-4fa3-a8ad-50d097635b64 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '891005fa5ff74041b75747d16cc66798', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7047912c5acd4adeba5d87c3bf895e39', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80377) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:34:12.869598 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6090255-ab0b-4fa3-a8ad-50d097635b64 admin admin] REQUEST_METHOD: `POST` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:34:12.869924 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6090255-ab0b-4fa3-a8ad-50d097635b64 admin admin] SCRIPT_NAME: `/identity` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:34:12.870201 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6090255-ab0b-4fa3-a8ad-50d097635b64 admin admin] PATH_INFO: `/v3/projects` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:34:12.870870 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6090255-ab0b-4fa3-a8ad-50d097635b64 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:34:12.872776 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6090255-ab0b-4fa3-a8ad-50d097635b64 admin admin] RBAC: Authorization granted {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:34:12.899786 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: [pid: 80377|app: 0|req: 1431/2848] 10.222.0.82 () {60 vars in 1153 bytes} [Tue Aug 2 10:34:12 2022] POST /identity/v3/projects => generated 363 bytes in 98 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 02 10:34:12.903992 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1094a50-ff7d-4a4d-b3ac-b98c1035fe13 None None] Authenticating user token {{(pid=80378) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:34:12.915021 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1094a50-ff7d-4a4d-b3ac-b98c1035fe13 None None] Validating token access rules against request {{(pid=80378) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:34:12.969537 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08c653c9-ba34-444c-85f6-6d9a91c0905e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '891005fa5ff74041b75747d16cc66798', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7047912c5acd4adeba5d87c3bf895e39', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80378) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:34:12.970184 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08c653c9-ba34-444c-85f6-6d9a91c0905e admin admin] REQUEST_METHOD: `POST` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:34:12.970470 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08c653c9-ba34-444c-85f6-6d9a91c0905e admin admin] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:34:12.970750 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08c653c9-ba34-444c-85f6-6d9a91c0905e admin admin] PATH_INFO: `/v3/users` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:34:12.971409 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08c653c9-ba34-444c-85f6-6d9a91c0905e admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:34:12.973863 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08c653c9-ba34-444c-85f6-6d9a91c0905e admin admin] RBAC: Authorization granted {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:34:12.995378 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 1418/2849] 10.222.0.82 () {60 vars in 1144 bytes} [Tue Aug 2 10:34:12 2022] POST /identity/v3/users => generated 288 bytes in 92 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 02 10:34:12.999325 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6090255-ab0b-4fa3-a8ad-50d097635b64 admin admin] Authenticating user token {{(pid=80377) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:34:13.009337 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6090255-ab0b-4fa3-a8ad-50d097635b64 admin admin] Validating token access rules against request {{(pid=80377) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:34:13.013804 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c60e364d-6b4c-41ca-9b56-4a83c1a8ec6f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '891005fa5ff74041b75747d16cc66798', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7047912c5acd4adeba5d87c3bf895e39', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80377) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:34:13.014473 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c60e364d-6b4c-41ca-9b56-4a83c1a8ec6f admin admin] REQUEST_METHOD: `GET` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:34:13.014758 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c60e364d-6b4c-41ca-9b56-4a83c1a8ec6f admin admin] SCRIPT_NAME: `/identity` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:34:13.015034 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c60e364d-6b4c-41ca-9b56-4a83c1a8ec6f admin admin] PATH_INFO: `/v3/roles` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:34:13.015721 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c60e364d-6b4c-41ca-9b56-4a83c1a8ec6f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:34:13.017297 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c60e364d-6b4c-41ca-9b56-4a83c1a8ec6f admin admin] RBAC: Authorization granted {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:34:13.027535 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: [pid: 80377|app: 0|req: 1432/2850] 10.222.0.82 () {58 vars in 1122 bytes} [Tue Aug 2 10:34:12 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 02 10:34:13.031446 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08c653c9-ba34-444c-85f6-6d9a91c0905e admin admin] Authenticating user token {{(pid=80378) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:34:13.041880 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08c653c9-ba34-444c-85f6-6d9a91c0905e admin admin] Validating token access rules against request {{(pid=80378) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:34:13.046509 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab8c8f88-4239-4cac-b35c-89677c17fbff admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '891005fa5ff74041b75747d16cc66798', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7047912c5acd4adeba5d87c3bf895e39', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80378) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:34:13.047158 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab8c8f88-4239-4cac-b35c-89677c17fbff admin admin] REQUEST_METHOD: `PUT` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:34:13.047443 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab8c8f88-4239-4cac-b35c-89677c17fbff admin admin] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:34:13.047722 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab8c8f88-4239-4cac-b35c-89677c17fbff admin admin] PATH_INFO: `/v3/projects/a8e8cb5c30d041e4b5b84a9e4c70b420/users/f6a7d6f023c5435a9bd6c3ac2c294b59/roles/237966442f05411191105bf336bc14f3` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:34:13.065025 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab8c8f88-4239-4cac-b35c-89677c17fbff admin admin] RBAC: Authorizing `identity:create_grant(project_id=a8e8cb5c30d041e4b5b84a9e4c70b420, user_id=f6a7d6f023c5435a9bd6c3ac2c294b59, role_id=237966442f05411191105bf336bc14f3)` {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:34:13.068383 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab8c8f88-4239-4cac-b35c-89677c17fbff admin admin] RBAC: Authorization granted {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:34:13.084497 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 1419/2851] 10.222.0.82 () {60 vars in 1483 bytes} [Tue Aug 2 10:34:13 2022] PUT /identity/v3/projects/a8e8cb5c30d041e4b5b84a9e4c70b420/users/f6a7d6f023c5435a9bd6c3ac2c294b59/roles/237966442f05411191105bf336bc14f3 => generated 0 bytes in 53 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 02 10:34:13.088500 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c60e364d-6b4c-41ca-9b56-4a83c1a8ec6f admin admin] Authenticating user token {{(pid=80377) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:34:13.098831 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c60e364d-6b4c-41ca-9b56-4a83c1a8ec6f admin admin] Validating token access rules against request {{(pid=80377) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:34:13.118932 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG dbcounter [-] [80377] Writing DB stats keystone:DELETE=6,keystone:SELECT=90,keystone:INSERT=4 {{(pid=80377) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 10:34:13.158819 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-788cc290-2890-4bb1-b617-e91eb844b060 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '891005fa5ff74041b75747d16cc66798', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7047912c5acd4adeba5d87c3bf895e39', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80377) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:34:13.159953 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-788cc290-2890-4bb1-b617-e91eb844b060 admin admin] REQUEST_METHOD: `GET` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:34:13.160237 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-788cc290-2890-4bb1-b617-e91eb844b060 admin admin] SCRIPT_NAME: `/identity` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:34:13.160514 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-788cc290-2890-4bb1-b617-e91eb844b060 admin admin] PATH_INFO: `/v3/roles` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:34:13.161215 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-788cc290-2890-4bb1-b617-e91eb844b060 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:34:13.162848 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-788cc290-2890-4bb1-b617-e91eb844b060 admin admin] RBAC: Authorization granted {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:34:13.172768 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: [pid: 80377|app: 0|req: 1433/2852] 10.222.0.82 () {58 vars in 1122 bytes} [Tue Aug 2 10:34:13 2022] GET /identity/v3/roles => generated 1420 bytes in 85 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 02 10:34:13.177650 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab8c8f88-4239-4cac-b35c-89677c17fbff admin admin] Authenticating user token {{(pid=80378) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:34:13.183122 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG dbcounter [-] [80378] Writing DB stats keystone:SELECT=93,keystone:INSERT=7 {{(pid=80378) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 10:34:13.189668 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab8c8f88-4239-4cac-b35c-89677c17fbff admin admin] Validating token access rules against request {{(pid=80378) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:34:13.195076 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ac21f54-9e4c-4d91-a92a-5d8543593ec4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '891005fa5ff74041b75747d16cc66798', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7047912c5acd4adeba5d87c3bf895e39', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80378) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:34:13.195752 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ac21f54-9e4c-4d91-a92a-5d8543593ec4 admin admin] REQUEST_METHOD: `PUT` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:34:13.196036 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ac21f54-9e4c-4d91-a92a-5d8543593ec4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:34:13.196315 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ac21f54-9e4c-4d91-a92a-5d8543593ec4 admin admin] PATH_INFO: `/v3/projects/a8e8cb5c30d041e4b5b84a9e4c70b420/users/f6a7d6f023c5435a9bd6c3ac2c294b59/roles/237966442f05411191105bf336bc14f3` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:34:13.198336 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ac21f54-9e4c-4d91-a92a-5d8543593ec4 admin admin] RBAC: Authorizing `identity:create_grant(project_id=a8e8cb5c30d041e4b5b84a9e4c70b420, user_id=f6a7d6f023c5435a9bd6c3ac2c294b59, role_id=237966442f05411191105bf336bc14f3)` {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:34:13.200996 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ac21f54-9e4c-4d91-a92a-5d8543593ec4 admin admin] RBAC: Authorization granted {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:34:13.211325 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 1420/2853] 10.222.0.82 () {60 vars in 1483 bytes} [Tue Aug 2 10:34:13 2022] PUT /identity/v3/projects/a8e8cb5c30d041e4b5b84a9e4c70b420/users/f6a7d6f023c5435a9bd6c3ac2c294b59/roles/237966442f05411191105bf336bc14f3 => generated 0 bytes in 34 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 02 10:34:13.222024 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d624de0-ac39-4aa7-a954-966d1ce658d1 None None] REQUEST_METHOD: `POST` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:34:13.222526 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d624de0-ac39-4aa7-a954-966d1ce658d1 None None] SCRIPT_NAME: `/identity` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:34:13.222632 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d624de0-ac39-4aa7-a954-966d1ce658d1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:34:13.258053 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.auth.core [None req-8d624de0-ac39-4aa7-a954-966d1ce658d1 None None] MFA Rules not processed for user `f6a7d6f023c5435a9bd6c3ac2c294b59`. Rule list: `[]` (Enabled: `True`). {{(pid=80377) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 02 10:34:13.259262 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.fernet_utils [None req-8d624de0-ac39-4aa7-a954-966d1ce658d1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80377) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 10:34:13.325736 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: [pid: 80377|app: 0|req: 1434/2854] 10.222.0.82 () {58 vars in 958 bytes} [Tue Aug 2 10:34:13 2022] POST /identity/v3/auth/tokens => generated 3307 bytes in 105 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 02 10:34:13.337294 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ac21f54-9e4c-4d91-a92a-5d8543593ec4 admin admin] Authenticating user token {{(pid=80378) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:34:13.339071 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.fernet_utils [None req-1ac21f54-9e4c-4d91-a92a-5d8543593ec4 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80378) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 10:34:13.396565 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ac21f54-9e4c-4d91-a92a-5d8543593ec4 admin admin] Validating token access rules against request {{(pid=80378) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:34:13.400143 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c707a25-52fe-4015-aad4-9b6b7a19e588 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1023de281ae74e9d8a6d6c312c1c97a7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f1b38ebb804c41c69772c998aae72c41', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80378) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:34:13.400751 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c707a25-52fe-4015-aad4-9b6b7a19e588 service cinder] REQUEST_METHOD: `GET` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:34:13.401033 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c707a25-52fe-4015-aad4-9b6b7a19e588 service cinder] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:34:13.401302 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c707a25-52fe-4015-aad4-9b6b7a19e588 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:34:13.410919 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c707a25-52fe-4015-aad4-9b6b7a19e588 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:34:13.412344 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c707a25-52fe-4015-aad4-9b6b7a19e588 service cinder] RBAC: Authorization granted {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:34:13.421716 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 1421/2855] 10.222.0.82 () {60 vars in 1365 bytes} [Tue Aug 2 10:34:13 2022] GET /identity/v3/auth/tokens => generated 3307 bytes in 85 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 02 10:34:13.497817 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d624de0-ac39-4aa7-a954-966d1ce658d1 None None] Authenticating user token {{(pid=80377) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:34:13.507516 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d624de0-ac39-4aa7-a954-966d1ce658d1 None None] Validating token access rules against request {{(pid=80377) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:34:13.511045 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d329ed73-0c51-4b95-b6aa-ae1f9040feac service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1023de281ae74e9d8a6d6c312c1c97a7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f1b38ebb804c41c69772c998aae72c41', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80377) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:34:13.511650 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d329ed73-0c51-4b95-b6aa-ae1f9040feac service cinder] REQUEST_METHOD: `GET` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:34:13.511928 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d329ed73-0c51-4b95-b6aa-ae1f9040feac service cinder] SCRIPT_NAME: `/identity` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:34:13.512200 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d329ed73-0c51-4b95-b6aa-ae1f9040feac service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:34:13.568601 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d329ed73-0c51-4b95-b6aa-ae1f9040feac service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:34:13.570072 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d329ed73-0c51-4b95-b6aa-ae1f9040feac service cinder] RBAC: Authorization granted {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:34:13.579062 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: [pid: 80377|app: 0|req: 1435/2856] 10.222.0.82 () {60 vars in 1365 bytes} [Tue Aug 2 10:34:13 2022] GET /identity/v3/auth/tokens => generated 3242 bytes in 82 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 02 10:34:16.508651 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c707a25-52fe-4015-aad4-9b6b7a19e588 service cinder] Authenticating user token {{(pid=80378) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:34:16.570432 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c707a25-52fe-4015-aad4-9b6b7a19e588 service cinder] Validating token access rules against request {{(pid=80378) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:34:16.574938 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffc9784f-27b8-4f08-9165-379fb9dc3bdb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '891005fa5ff74041b75747d16cc66798', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7047912c5acd4adeba5d87c3bf895e39', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80378) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:34:16.575575 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffc9784f-27b8-4f08-9165-379fb9dc3bdb admin admin] REQUEST_METHOD: `DELETE` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:34:16.575859 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffc9784f-27b8-4f08-9165-379fb9dc3bdb admin admin] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:34:16.576137 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffc9784f-27b8-4f08-9165-379fb9dc3bdb admin admin] PATH_INFO: `/v3/users/14b83284af034be38a30ba54867ad0c2` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:34:16.577329 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ffc9784f-27b8-4f08-9165-379fb9dc3bdb admin admin] RBAC: Authorizing `identity:delete_user(user_id=14b83284af034be38a30ba54867ad0c2)` {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:34:16.579277 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ffc9784f-27b8-4f08-9165-379fb9dc3bdb admin admin] RBAC: Authorization granted {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:34:16.623498 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.notifications [None req-ffc9784f-27b8-4f08-9165-379fb9dc3bdb admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '14b83284af034be38a30ba54867ad0c2'} {{(pid=80378) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 10:34:16.631284 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.notifications [None req-ffc9784f-27b8-4f08-9165-379fb9dc3bdb admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '14b83284af034be38a30ba54867ad0c2'} {{(pid=80378) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 10:34:16.638133 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.notifications [None req-ffc9784f-27b8-4f08-9165-379fb9dc3bdb admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '14b83284af034be38a30ba54867ad0c2'} {{(pid=80378) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 10:34:16.638752 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.notifications [None req-ffc9784f-27b8-4f08-9165-379fb9dc3bdb admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '14b83284af034be38a30ba54867ad0c2'} {{(pid=80378) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 10:34:16.650229 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.notifications [None req-ffc9784f-27b8-4f08-9165-379fb9dc3bdb admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '14b83284af034be38a30ba54867ad0c2'} {{(pid=80378) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 10:34:16.656147 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 1422/2857] 10.222.0.82 () {58 vars in 1224 bytes} [Tue Aug 2 10:34:16 2022] DELETE /identity/v3/users/14b83284af034be38a30ba54867ad0c2 => generated 0 bytes in 148 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 02 10:34:16.661328 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cadadc49-22ef-4513-90a9-a092b380be10 None None] REQUEST_METHOD: `POST` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:34:16.661843 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cadadc49-22ef-4513-90a9-a092b380be10 None None] SCRIPT_NAME: `/identity` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:34:16.661966 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cadadc49-22ef-4513-90a9-a092b380be10 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:34:16.681394 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.auth.core [None req-cadadc49-22ef-4513-90a9-a092b380be10 None None] MFA Rules not processed for user `891005fa5ff74041b75747d16cc66798`. Rule list: `[]` (Enabled: `True`). {{(pid=80377) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 02 10:34:16.682729 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.fernet_utils [None req-cadadc49-22ef-4513-90a9-a092b380be10 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80377) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 10:34:16.726237 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG dbcounter [-] [80377] Writing DB stats keystone:SELECT=100 {{(pid=80377) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 10:34:16.745706 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: [pid: 80377|app: 0|req: 1436/2858] 10.222.0.82 () {58 vars in 958 bytes} [Tue Aug 2 10:34:16 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 02 10:34:16.755289 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffc9784f-27b8-4f08-9165-379fb9dc3bdb admin admin] Authenticating user token {{(pid=80378) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:34:16.756732 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.fernet_utils [None req-ffc9784f-27b8-4f08-9165-379fb9dc3bdb admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80378) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 10:34:16.770778 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG dbcounter [-] [80378] Writing DB stats keystone:SELECT=88,keystone:INSERT=2,keystone:DELETE=10 {{(pid=80378) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 10:34:16.816020 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffc9784f-27b8-4f08-9165-379fb9dc3bdb admin admin] Validating token access rules against request {{(pid=80378) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:34:16.819942 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d687b267-f9ef-40f5-b6d7-9f5363f85cf2 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '33b87170f0634b67bce60750847b5cbe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f1b38ebb804c41c69772c998aae72c41', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80378) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:34:16.820561 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d687b267-f9ef-40f5-b6d7-9f5363f85cf2 service neutron] REQUEST_METHOD: `GET` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:34:16.820846 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d687b267-f9ef-40f5-b6d7-9f5363f85cf2 service neutron] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:34:16.821124 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d687b267-f9ef-40f5-b6d7-9f5363f85cf2 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:34:16.830108 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d687b267-f9ef-40f5-b6d7-9f5363f85cf2 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:34:16.831537 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d687b267-f9ef-40f5-b6d7-9f5363f85cf2 service neutron] RBAC: Authorization granted {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:34:16.840811 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 1423/2859] 10.222.0.82 () {60 vars in 1365 bytes} [Tue Aug 2 10:34:16 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 86 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 02 10:34:17.115840 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cadadc49-22ef-4513-90a9-a092b380be10 None None] Authenticating user token {{(pid=80377) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:34:17.117524 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.fernet_utils [None req-cadadc49-22ef-4513-90a9-a092b380be10 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80377) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 10:34:17.129923 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cadadc49-22ef-4513-90a9-a092b380be10 None None] Validating token access rules against request {{(pid=80377) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:34:17.134605 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a0737dc-1511-4156-94cd-311b9bdebc7b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '891005fa5ff74041b75747d16cc66798', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7047912c5acd4adeba5d87c3bf895e39', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80377) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:34:17.135311 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a0737dc-1511-4156-94cd-311b9bdebc7b admin admin] REQUEST_METHOD: `DELETE` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:34:17.135594 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a0737dc-1511-4156-94cd-311b9bdebc7b admin admin] SCRIPT_NAME: `/identity` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:34:17.135873 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a0737dc-1511-4156-94cd-311b9bdebc7b admin admin] PATH_INFO: `/v3/projects/c274ce0e9703454594fc62a1ff33a030` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:34:17.137050 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a0737dc-1511-4156-94cd-311b9bdebc7b admin admin] RBAC: Authorizing `identity:delete_project(project_id=c274ce0e9703454594fc62a1ff33a030)` {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:34:17.139015 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a0737dc-1511-4156-94cd-311b9bdebc7b admin admin] RBAC: Authorization granted {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:34:17.187269 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.notifications [None req-9a0737dc-1511-4156-94cd-311b9bdebc7b admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'c274ce0e9703454594fc62a1ff33a030', 'request_id': 'req-9a0737dc-1511-4156-94cd-311b9bdebc7b'} {{(pid=80377) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 10:34:17.192936 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.notifications [None req-9a0737dc-1511-4156-94cd-311b9bdebc7b admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'c274ce0e9703454594fc62a1ff33a030', 'request_id': 'req-9a0737dc-1511-4156-94cd-311b9bdebc7b'} {{(pid=80377) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 10:34:17.199046 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.notifications [None req-9a0737dc-1511-4156-94cd-311b9bdebc7b admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'c274ce0e9703454594fc62a1ff33a030', 'request_id': 'req-9a0737dc-1511-4156-94cd-311b9bdebc7b'} {{(pid=80377) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 10:34:17.214948 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.notifications [None req-9a0737dc-1511-4156-94cd-311b9bdebc7b admin admin] The token cache is being invalidate because project c274ce0e9703454594fc62a1ff33a030 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80377) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 02 10:34:17.215238 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.notifications [None req-9a0737dc-1511-4156-94cd-311b9bdebc7b admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80377) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 10:34:17.217227 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.notifications [None req-9a0737dc-1511-4156-94cd-311b9bdebc7b admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80377) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 10:34:17.218832 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: [pid: 80377|app: 0|req: 1437/2860] 10.222.0.82 () {58 vars in 1233 bytes} [Tue Aug 2 10:34:17 2022] DELETE /identity/v3/projects/c274ce0e9703454594fc62a1ff33a030 => generated 0 bytes in 104 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 02 10:34:17.223456 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d687b267-f9ef-40f5-b6d7-9f5363f85cf2 service neutron] Authenticating user token {{(pid=80378) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:34:17.233413 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d687b267-f9ef-40f5-b6d7-9f5363f85cf2 service neutron] Validating token access rules against request {{(pid=80378) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:34:17.289611 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f25fbf8a-dded-4040-98eb-58ec592d9e64 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '891005fa5ff74041b75747d16cc66798', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7047912c5acd4adeba5d87c3bf895e39', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80378) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:34:17.290438 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f25fbf8a-dded-4040-98eb-58ec592d9e64 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:34:17.291180 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f25fbf8a-dded-4040-98eb-58ec592d9e64 admin admin] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:34:17.291180 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f25fbf8a-dded-4040-98eb-58ec592d9e64 admin admin] PATH_INFO: `/v3/users/f6a7d6f023c5435a9bd6c3ac2c294b59` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:34:17.292260 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f25fbf8a-dded-4040-98eb-58ec592d9e64 admin admin] RBAC: Authorizing `identity:delete_user(user_id=f6a7d6f023c5435a9bd6c3ac2c294b59)` {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:34:17.294249 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f25fbf8a-dded-4040-98eb-58ec592d9e64 admin admin] RBAC: Authorization granted {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:34:17.340081 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.notifications [None req-f25fbf8a-dded-4040-98eb-58ec592d9e64 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'f6a7d6f023c5435a9bd6c3ac2c294b59'} {{(pid=80378) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 10:34:17.347703 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.notifications [None req-f25fbf8a-dded-4040-98eb-58ec592d9e64 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'f6a7d6f023c5435a9bd6c3ac2c294b59'} {{(pid=80378) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 10:34:17.354650 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.notifications [None req-f25fbf8a-dded-4040-98eb-58ec592d9e64 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'f6a7d6f023c5435a9bd6c3ac2c294b59'} {{(pid=80378) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 10:34:17.355280 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.notifications [None req-f25fbf8a-dded-4040-98eb-58ec592d9e64 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'f6a7d6f023c5435a9bd6c3ac2c294b59'} {{(pid=80378) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 10:34:17.367189 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.notifications [None req-f25fbf8a-dded-4040-98eb-58ec592d9e64 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'f6a7d6f023c5435a9bd6c3ac2c294b59'} {{(pid=80378) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 10:34:17.373005 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 1424/2861] 10.222.0.82 () {58 vars in 1224 bytes} [Tue Aug 2 10:34:17 2022] DELETE /identity/v3/users/f6a7d6f023c5435a9bd6c3ac2c294b59 => generated 0 bytes in 150 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 02 10:34:17.643976 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a0737dc-1511-4156-94cd-311b9bdebc7b admin admin] Authenticating user token {{(pid=80377) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:34:17.645506 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.fernet_utils [None req-9a0737dc-1511-4156-94cd-311b9bdebc7b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80377) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 10:34:17.661360 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a0737dc-1511-4156-94cd-311b9bdebc7b admin admin] Validating token access rules against request {{(pid=80377) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:34:17.720630 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30804632-6593-4430-a48e-ab631462eef6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '891005fa5ff74041b75747d16cc66798', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7047912c5acd4adeba5d87c3bf895e39', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80377) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:34:17.721203 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30804632-6593-4430-a48e-ab631462eef6 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:34:17.721512 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30804632-6593-4430-a48e-ab631462eef6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:34:17.722100 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30804632-6593-4430-a48e-ab631462eef6 admin admin] PATH_INFO: `/v3/projects/a8e8cb5c30d041e4b5b84a9e4c70b420` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:34:17.723270 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30804632-6593-4430-a48e-ab631462eef6 admin admin] RBAC: Authorizing `identity:delete_project(project_id=a8e8cb5c30d041e4b5b84a9e4c70b420)` {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:34:17.725176 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30804632-6593-4430-a48e-ab631462eef6 admin admin] RBAC: Authorization granted {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:34:17.775237 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.notifications [None req-30804632-6593-4430-a48e-ab631462eef6 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'a8e8cb5c30d041e4b5b84a9e4c70b420', 'request_id': 'req-30804632-6593-4430-a48e-ab631462eef6'} {{(pid=80377) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 10:34:17.780646 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.notifications [None req-30804632-6593-4430-a48e-ab631462eef6 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'a8e8cb5c30d041e4b5b84a9e4c70b420', 'request_id': 'req-30804632-6593-4430-a48e-ab631462eef6'} {{(pid=80377) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 10:34:17.786943 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.notifications [None req-30804632-6593-4430-a48e-ab631462eef6 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'a8e8cb5c30d041e4b5b84a9e4c70b420', 'request_id': 'req-30804632-6593-4430-a48e-ab631462eef6'} {{(pid=80377) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 10:34:17.795167 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG dbcounter [-] [80377] Writing DB stats keystone:SELECT=84,keystone:DELETE=14,keystone:INSERT=2 {{(pid=80377) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 10:34:17.800329 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.notifications [None req-30804632-6593-4430-a48e-ab631462eef6 admin admin] The token cache is being invalidate because project a8e8cb5c30d041e4b5b84a9e4c70b420 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80377) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 02 10:34:17.800843 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.notifications [None req-30804632-6593-4430-a48e-ab631462eef6 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80377) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 10:34:17.801767 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.notifications [None req-30804632-6593-4430-a48e-ab631462eef6 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80377) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 10:34:17.803814 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: [pid: 80377|app: 0|req: 1438/2862] 10.222.0.82 () {58 vars in 1233 bytes} [Tue Aug 2 10:34:17 2022] DELETE /identity/v3/projects/a8e8cb5c30d041e4b5b84a9e4c70b420 => generated 0 bytes in 161 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 02 10:34:17.809045 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b6ec6d6-9cb4-4285-a821-2cd020afdbba None None] REQUEST_METHOD: `POST` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:34:17.809544 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b6ec6d6-9cb4-4285-a821-2cd020afdbba None None] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:34:17.809715 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b6ec6d6-9cb4-4285-a821-2cd020afdbba None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:34:17.829789 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.auth.core [None req-5b6ec6d6-9cb4-4285-a821-2cd020afdbba None None] MFA Rules not processed for user `891005fa5ff74041b75747d16cc66798`. Rule list: `[]` (Enabled: `True`). {{(pid=80378) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 02 10:34:17.831007 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.fernet_utils [None req-5b6ec6d6-9cb4-4285-a821-2cd020afdbba None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80378) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 10:34:17.860637 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG dbcounter [-] [80378] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80378) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 10:34:17.893830 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 1425/2863] 10.222.0.82 () {58 vars in 958 bytes} [Tue Aug 2 10:34:17 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 02 10:34:17.903084 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fdf43ae-44c8-4578-89df-ae40631476d4 None None] REQUEST_METHOD: `POST` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:34:17.903376 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fdf43ae-44c8-4578-89df-ae40631476d4 None None] SCRIPT_NAME: `/identity` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:34:17.903656 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fdf43ae-44c8-4578-89df-ae40631476d4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:34:17.922859 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.auth.core [None req-4fdf43ae-44c8-4578-89df-ae40631476d4 None None] MFA Rules not processed for user `891005fa5ff74041b75747d16cc66798`. Rule list: `[]` (Enabled: `True`). {{(pid=80377) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 02 10:34:17.924337 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.fernet_utils [None req-4fdf43ae-44c8-4578-89df-ae40631476d4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80377) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 10:34:17.932608 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: [pid: 80377|app: 0|req: 1439/2864] 10.222.0.82 () {58 vars in 958 bytes} [Tue Aug 2 10:34:17 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 02 10:34:17.936694 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b6ec6d6-9cb4-4285-a821-2cd020afdbba None None] Authenticating user token {{(pid=80378) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:34:17.947270 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b6ec6d6-9cb4-4285-a821-2cd020afdbba None None] Validating token access rules against request {{(pid=80378) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:34:17.951866 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2a077f4-32cb-4ae4-ab7f-c0ca26116696 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '891005fa5ff74041b75747d16cc66798', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7047912c5acd4adeba5d87c3bf895e39', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80378) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:34:17.952471 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2a077f4-32cb-4ae4-ab7f-c0ca26116696 admin admin] REQUEST_METHOD: `GET` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:34:17.952755 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2a077f4-32cb-4ae4-ab7f-c0ca26116696 admin admin] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:34:17.953032 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2a077f4-32cb-4ae4-ab7f-c0ca26116696 admin admin] PATH_INFO: `/v3/domains` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:34:17.953723 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2a077f4-32cb-4ae4-ab7f-c0ca26116696 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80378) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 02 10:34:17.954062 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2a077f4-32cb-4ae4-ab7f-c0ca26116696 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:34:17.955662 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2a077f4-32cb-4ae4-ab7f-c0ca26116696 admin admin] RBAC: Authorization granted {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:34:17.966965 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 1426/2865] 10.222.0.82 () {58 vars in 1153 bytes} [Tue Aug 2 10:34:17 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 02 10:34:17.971098 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fdf43ae-44c8-4578-89df-ae40631476d4 None None] Authenticating user token {{(pid=80377) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:34:17.979996 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fdf43ae-44c8-4578-89df-ae40631476d4 None None] Validating token access rules against request {{(pid=80377) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:34:17.984508 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2aba0fb8-ec12-4788-97c3-a475a1018d3d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '891005fa5ff74041b75747d16cc66798', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7047912c5acd4adeba5d87c3bf895e39', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80377) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:34:17.985104 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2aba0fb8-ec12-4788-97c3-a475a1018d3d admin admin] REQUEST_METHOD: `POST` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:34:17.985385 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2aba0fb8-ec12-4788-97c3-a475a1018d3d admin admin] SCRIPT_NAME: `/identity` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:34:17.985664 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2aba0fb8-ec12-4788-97c3-a475a1018d3d admin admin] PATH_INFO: `/v3/projects` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:34:17.986362 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2aba0fb8-ec12-4788-97c3-a475a1018d3d admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:34:17.988740 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2aba0fb8-ec12-4788-97c3-a475a1018d3d admin admin] RBAC: Authorization granted {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:34:18.015347 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: [pid: 80377|app: 0|req: 1440/2866] 10.222.0.82 () {60 vars in 1153 bytes} [Tue Aug 2 10:34:17 2022] POST /identity/v3/projects => generated 371 bytes in 45 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 02 10:34:18.019540 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2a077f4-32cb-4ae4-ab7f-c0ca26116696 admin admin] Authenticating user token {{(pid=80378) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:34:18.029723 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2a077f4-32cb-4ae4-ab7f-c0ca26116696 admin admin] Validating token access rules against request {{(pid=80378) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:34:18.084445 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8be854df-71c1-423e-98c5-83f35cb7d108 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '891005fa5ff74041b75747d16cc66798', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7047912c5acd4adeba5d87c3bf895e39', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80378) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:34:18.085062 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8be854df-71c1-423e-98c5-83f35cb7d108 admin admin] REQUEST_METHOD: `POST` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:34:18.085349 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8be854df-71c1-423e-98c5-83f35cb7d108 admin admin] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:34:18.085627 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8be854df-71c1-423e-98c5-83f35cb7d108 admin admin] PATH_INFO: `/v3/users` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:34:18.086316 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8be854df-71c1-423e-98c5-83f35cb7d108 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:34:18.088258 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8be854df-71c1-423e-98c5-83f35cb7d108 admin admin] RBAC: Authorization granted {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:34:18.108561 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 1427/2867] 10.222.0.82 () {60 vars in 1144 bytes} [Tue Aug 2 10:34:18 2022] POST /identity/v3/users => generated 286 bytes in 89 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 02 10:34:18.113324 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2aba0fb8-ec12-4788-97c3-a475a1018d3d admin admin] Authenticating user token {{(pid=80377) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:34:18.123424 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2aba0fb8-ec12-4788-97c3-a475a1018d3d admin admin] Validating token access rules against request {{(pid=80377) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:34:18.129492 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2fe8313-3388-48d2-8574-a1dbbd3bd8e0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '891005fa5ff74041b75747d16cc66798', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7047912c5acd4adeba5d87c3bf895e39', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80377) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:34:18.130147 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2fe8313-3388-48d2-8574-a1dbbd3bd8e0 admin admin] REQUEST_METHOD: `GET` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:34:18.130435 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2fe8313-3388-48d2-8574-a1dbbd3bd8e0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:34:18.130713 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2fe8313-3388-48d2-8574-a1dbbd3bd8e0 admin admin] PATH_INFO: `/v3/roles` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:34:18.131418 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2fe8313-3388-48d2-8574-a1dbbd3bd8e0 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:34:18.133026 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2fe8313-3388-48d2-8574-a1dbbd3bd8e0 admin admin] RBAC: Authorization granted {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:34:18.142843 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: [pid: 80377|app: 0|req: 1441/2868] 10.222.0.82 () {58 vars in 1122 bytes} [Tue Aug 2 10:34:18 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 02 10:34:18.146827 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8be854df-71c1-423e-98c5-83f35cb7d108 admin admin] Authenticating user token {{(pid=80378) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:34:18.156999 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8be854df-71c1-423e-98c5-83f35cb7d108 admin admin] Validating token access rules against request {{(pid=80378) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:34:18.162163 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6912bdcb-726a-4fb5-8563-1a92e3b66e45 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '891005fa5ff74041b75747d16cc66798', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7047912c5acd4adeba5d87c3bf895e39', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80378) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:34:18.162765 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6912bdcb-726a-4fb5-8563-1a92e3b66e45 admin admin] REQUEST_METHOD: `GET` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:34:18.163048 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6912bdcb-726a-4fb5-8563-1a92e3b66e45 admin admin] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:34:18.163329 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6912bdcb-726a-4fb5-8563-1a92e3b66e45 admin admin] PATH_INFO: `/v3/roles` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:34:18.164029 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6912bdcb-726a-4fb5-8563-1a92e3b66e45 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:34:18.165627 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6912bdcb-726a-4fb5-8563-1a92e3b66e45 admin admin] RBAC: Authorization granted {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:34:18.176644 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 1428/2869] 10.222.0.82 () {58 vars in 1122 bytes} [Tue Aug 2 10:34:18 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 02 10:34:18.180660 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2fe8313-3388-48d2-8574-a1dbbd3bd8e0 admin admin] Authenticating user token {{(pid=80377) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:34:18.190385 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2fe8313-3388-48d2-8574-a1dbbd3bd8e0 admin admin] Validating token access rules against request {{(pid=80377) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:34:18.195164 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-301b0ac2-74fb-4513-9b2e-20f8446e83a1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '891005fa5ff74041b75747d16cc66798', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7047912c5acd4adeba5d87c3bf895e39', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80377) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:34:18.195835 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-301b0ac2-74fb-4513-9b2e-20f8446e83a1 admin admin] REQUEST_METHOD: `PUT` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:34:18.196090 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-301b0ac2-74fb-4513-9b2e-20f8446e83a1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:34:18.196363 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-301b0ac2-74fb-4513-9b2e-20f8446e83a1 admin admin] PATH_INFO: `/v3/projects/984f14e49b2e4d04a235fdb94d34462b/users/51278249ade046eb8f80d0f5c7743a62/roles/e1c04b19fc4543b8b54f58bbcffbc598` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:34:18.210372 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-301b0ac2-74fb-4513-9b2e-20f8446e83a1 admin admin] RBAC: Authorizing `identity:create_grant(project_id=984f14e49b2e4d04a235fdb94d34462b, user_id=51278249ade046eb8f80d0f5c7743a62, role_id=e1c04b19fc4543b8b54f58bbcffbc598)` {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:34:18.212987 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-301b0ac2-74fb-4513-9b2e-20f8446e83a1 admin admin] RBAC: Authorization granted {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:34:18.226165 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: [pid: 80377|app: 0|req: 1442/2870] 10.222.0.82 () {60 vars in 1483 bytes} [Tue Aug 2 10:34:18 2022] PUT /identity/v3/projects/984f14e49b2e4d04a235fdb94d34462b/users/51278249ade046eb8f80d0f5c7743a62/roles/e1c04b19fc4543b8b54f58bbcffbc598 => generated 0 bytes in 46 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 02 10:34:18.231238 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b01061e7-5068-4356-b90b-153a5980c0c2 None None] REQUEST_METHOD: `POST` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:34:18.231529 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b01061e7-5068-4356-b90b-153a5980c0c2 None None] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:34:18.231806 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b01061e7-5068-4356-b90b-153a5980c0c2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:34:18.250582 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.auth.core [None req-b01061e7-5068-4356-b90b-153a5980c0c2 None None] MFA Rules not processed for user `891005fa5ff74041b75747d16cc66798`. Rule list: `[]` (Enabled: `True`). {{(pid=80378) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 02 10:34:18.251759 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.fernet_utils [None req-b01061e7-5068-4356-b90b-153a5980c0c2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80378) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 10:34:18.309048 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 1429/2871] 10.222.0.82 () {58 vars in 958 bytes} [Tue Aug 2 10:34:18 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 79 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 02 10:34:18.318872 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-301b0ac2-74fb-4513-9b2e-20f8446e83a1 admin admin] Authenticating user token {{(pid=80377) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:34:18.320587 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.fernet_utils [None req-301b0ac2-74fb-4513-9b2e-20f8446e83a1 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80377) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 10:34:18.376533 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-301b0ac2-74fb-4513-9b2e-20f8446e83a1 admin admin] Validating token access rules against request {{(pid=80377) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:34:18.380143 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-618cb247-8fac-4034-8aeb-7901f8920468 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '33b87170f0634b67bce60750847b5cbe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f1b38ebb804c41c69772c998aae72c41', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80377) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:34:18.380766 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-618cb247-8fac-4034-8aeb-7901f8920468 service neutron] REQUEST_METHOD: `GET` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:34:18.381050 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-618cb247-8fac-4034-8aeb-7901f8920468 service neutron] SCRIPT_NAME: `/identity` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:34:18.381328 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-618cb247-8fac-4034-8aeb-7901f8920468 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:34:18.390189 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-618cb247-8fac-4034-8aeb-7901f8920468 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:34:18.391629 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-618cb247-8fac-4034-8aeb-7901f8920468 service neutron] RBAC: Authorization granted {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:34:18.400957 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: [pid: 80377|app: 0|req: 1443/2872] 10.222.0.82 () {60 vars in 1365 bytes} [Tue Aug 2 10:34:18 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 83 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 02 10:34:19.016337 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e52e0fb-9815-47f4-8236-37ee7c38befa None None] REQUEST_METHOD: `POST` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:34:19.016836 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e52e0fb-9815-47f4-8236-37ee7c38befa None None] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:34:19.016968 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e52e0fb-9815-47f4-8236-37ee7c38befa None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:34:19.044900 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.auth.core [None req-5e52e0fb-9815-47f4-8236-37ee7c38befa None None] MFA Rules not processed for user `51278249ade046eb8f80d0f5c7743a62`. Rule list: `[]` (Enabled: `True`). {{(pid=80378) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 02 10:34:19.046164 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.fernet_utils [None req-5e52e0fb-9815-47f4-8236-37ee7c38befa None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80378) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 10:34:19.077112 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG dbcounter [-] [80378] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80378) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 10:34:19.115808 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 1430/2873] 10.222.0.82 () {58 vars in 958 bytes} [Tue Aug 2 10:34:19 2022] POST /identity/v3/auth/tokens => generated 3248 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 02 10:34:19.120177 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-618cb247-8fac-4034-8aeb-7901f8920468 service neutron] Authenticating user token {{(pid=80377) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:34:19.130857 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-618cb247-8fac-4034-8aeb-7901f8920468 service neutron] Validating token access rules against request {{(pid=80377) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:34:19.135489 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e835d8fb-a12e-480e-a5e9-67a54aeb08f3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '891005fa5ff74041b75747d16cc66798', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7047912c5acd4adeba5d87c3bf895e39', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80377) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:34:19.136085 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e835d8fb-a12e-480e-a5e9-67a54aeb08f3 admin admin] REQUEST_METHOD: `POST` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:34:19.136376 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e835d8fb-a12e-480e-a5e9-67a54aeb08f3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:34:19.136650 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e835d8fb-a12e-480e-a5e9-67a54aeb08f3 admin admin] PATH_INFO: `/v3/projects` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:34:19.137327 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e835d8fb-a12e-480e-a5e9-67a54aeb08f3 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:34:19.139273 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e835d8fb-a12e-480e-a5e9-67a54aeb08f3 admin admin] RBAC: Authorization granted {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:34:19.172129 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: [pid: 80377|app: 0|req: 1444/2874] 10.222.0.82 () {60 vars in 1153 bytes} [Tue Aug 2 10:34:19 2022] POST /identity/v3/projects => generated 373 bytes in 49 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 02 10:34:19.173422 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e52e0fb-9815-47f4-8236-37ee7c38befa None None] Authenticating user token {{(pid=80378) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:34:19.184577 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e52e0fb-9815-47f4-8236-37ee7c38befa None None] Validating token access rules against request {{(pid=80378) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:34:19.242432 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e41da6d-0e51-499d-8f55-6b78c9229aec admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '891005fa5ff74041b75747d16cc66798', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7047912c5acd4adeba5d87c3bf895e39', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80378) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:34:19.243460 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e41da6d-0e51-499d-8f55-6b78c9229aec admin admin] REQUEST_METHOD: `POST` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:34:19.243874 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e41da6d-0e51-499d-8f55-6b78c9229aec admin admin] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:34:19.244157 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e41da6d-0e51-499d-8f55-6b78c9229aec admin admin] PATH_INFO: `/v3/users` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:34:19.245002 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e41da6d-0e51-499d-8f55-6b78c9229aec admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:34:19.247448 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e41da6d-0e51-499d-8f55-6b78c9229aec admin admin] RBAC: Authorization granted {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:34:19.269391 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 1431/2875] 10.222.0.82 () {60 vars in 1144 bytes} [Tue Aug 2 10:34:19 2022] POST /identity/v3/users => generated 293 bytes in 96 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 02 10:34:19.273350 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e835d8fb-a12e-480e-a5e9-67a54aeb08f3 admin admin] Authenticating user token {{(pid=80377) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:34:19.283536 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e835d8fb-a12e-480e-a5e9-67a54aeb08f3 admin admin] Validating token access rules against request {{(pid=80377) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:34:19.288218 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45bf25a8-aac0-4c23-b6b4-a714157abc19 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '891005fa5ff74041b75747d16cc66798', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7047912c5acd4adeba5d87c3bf895e39', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80377) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:34:19.288829 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45bf25a8-aac0-4c23-b6b4-a714157abc19 admin admin] REQUEST_METHOD: `GET` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:34:19.289112 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45bf25a8-aac0-4c23-b6b4-a714157abc19 admin admin] SCRIPT_NAME: `/identity` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:34:19.289389 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45bf25a8-aac0-4c23-b6b4-a714157abc19 admin admin] PATH_INFO: `/v3/roles` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:34:19.290172 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45bf25a8-aac0-4c23-b6b4-a714157abc19 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:34:19.291774 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45bf25a8-aac0-4c23-b6b4-a714157abc19 admin admin] RBAC: Authorization granted {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:34:19.303438 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: [pid: 80377|app: 0|req: 1445/2876] 10.222.0.82 () {58 vars in 1122 bytes} [Tue Aug 2 10:34:19 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 02 10:34:19.307530 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e41da6d-0e51-499d-8f55-6b78c9229aec admin admin] Authenticating user token {{(pid=80378) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:34:19.318522 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e41da6d-0e51-499d-8f55-6b78c9229aec admin admin] Validating token access rules against request {{(pid=80378) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:34:19.323573 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0bb218d-82d5-47af-9d4a-3d4dd2bf3ade admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '891005fa5ff74041b75747d16cc66798', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7047912c5acd4adeba5d87c3bf895e39', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80378) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:34:19.324259 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0bb218d-82d5-47af-9d4a-3d4dd2bf3ade admin admin] REQUEST_METHOD: `PUT` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:34:19.324560 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0bb218d-82d5-47af-9d4a-3d4dd2bf3ade admin admin] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:34:19.324816 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0bb218d-82d5-47af-9d4a-3d4dd2bf3ade admin admin] PATH_INFO: `/v3/projects/06530e678fda4a44875b57ba126ee314/users/8813b7467a82484f9eec49e1817e4498/roles/237966442f05411191105bf336bc14f3` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:34:19.340173 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0bb218d-82d5-47af-9d4a-3d4dd2bf3ade admin admin] RBAC: Authorizing `identity:create_grant(project_id=06530e678fda4a44875b57ba126ee314, user_id=8813b7467a82484f9eec49e1817e4498, role_id=237966442f05411191105bf336bc14f3)` {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:34:19.342895 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0bb218d-82d5-47af-9d4a-3d4dd2bf3ade admin admin] RBAC: Authorization granted {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:34:19.355514 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 1432/2877] 10.222.0.82 () {60 vars in 1483 bytes} [Tue Aug 2 10:34:19 2022] PUT /identity/v3/projects/06530e678fda4a44875b57ba126ee314/users/8813b7467a82484f9eec49e1817e4498/roles/237966442f05411191105bf336bc14f3 => generated 0 bytes in 48 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 02 10:34:19.359649 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45bf25a8-aac0-4c23-b6b4-a714157abc19 admin admin] Authenticating user token {{(pid=80377) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:34:19.370126 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45bf25a8-aac0-4c23-b6b4-a714157abc19 admin admin] Validating token access rules against request {{(pid=80377) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:34:19.424395 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-835415ce-2c68-49bc-9a0f-a5ca4dc46867 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '891005fa5ff74041b75747d16cc66798', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7047912c5acd4adeba5d87c3bf895e39', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80377) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:34:19.425541 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-835415ce-2c68-49bc-9a0f-a5ca4dc46867 admin admin] REQUEST_METHOD: `GET` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:34:19.425851 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-835415ce-2c68-49bc-9a0f-a5ca4dc46867 admin admin] SCRIPT_NAME: `/identity` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:34:19.426131 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-835415ce-2c68-49bc-9a0f-a5ca4dc46867 admin admin] PATH_INFO: `/v3/roles` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:34:19.426830 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-835415ce-2c68-49bc-9a0f-a5ca4dc46867 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:34:19.428422 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-835415ce-2c68-49bc-9a0f-a5ca4dc46867 admin admin] RBAC: Authorization granted {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:34:19.433493 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG dbcounter [-] [80377] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80377) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 10:34:19.439175 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: [pid: 80377|app: 0|req: 1446/2878] 10.222.0.82 () {58 vars in 1122 bytes} [Tue Aug 2 10:34:19 2022] GET /identity/v3/roles => generated 1420 bytes in 80 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 02 10:34:19.444268 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0bb218d-82d5-47af-9d4a-3d4dd2bf3ade admin admin] Authenticating user token {{(pid=80378) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:34:19.454416 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0bb218d-82d5-47af-9d4a-3d4dd2bf3ade admin admin] Validating token access rules against request {{(pid=80378) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:34:19.458958 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37654b86-7187-4290-99fc-d73af29fe82b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '891005fa5ff74041b75747d16cc66798', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7047912c5acd4adeba5d87c3bf895e39', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80378) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:34:19.459608 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37654b86-7187-4290-99fc-d73af29fe82b admin admin] REQUEST_METHOD: `PUT` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:34:19.459890 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37654b86-7187-4290-99fc-d73af29fe82b admin admin] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:34:19.460160 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37654b86-7187-4290-99fc-d73af29fe82b admin admin] PATH_INFO: `/v3/projects/06530e678fda4a44875b57ba126ee314/users/8813b7467a82484f9eec49e1817e4498/roles/237966442f05411191105bf336bc14f3` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:34:19.462120 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37654b86-7187-4290-99fc-d73af29fe82b admin admin] RBAC: Authorizing `identity:create_grant(project_id=06530e678fda4a44875b57ba126ee314, user_id=8813b7467a82484f9eec49e1817e4498, role_id=237966442f05411191105bf336bc14f3)` {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:34:19.464718 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37654b86-7187-4290-99fc-d73af29fe82b admin admin] RBAC: Authorization granted {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:34:19.475246 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 1433/2879] 10.222.0.82 () {60 vars in 1483 bytes} [Tue Aug 2 10:34:19 2022] PUT /identity/v3/projects/06530e678fda4a44875b57ba126ee314/users/8813b7467a82484f9eec49e1817e4498/roles/237966442f05411191105bf336bc14f3 => generated 0 bytes in 31 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 02 10:34:20.101497 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8728fdee-06e2-43be-b134-a6378151f9b8 None None] REQUEST_METHOD: `POST` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:34:20.101811 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8728fdee-06e2-43be-b134-a6378151f9b8 None None] SCRIPT_NAME: `/identity` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:34:20.102179 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8728fdee-06e2-43be-b134-a6378151f9b8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:34:20.129882 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.auth.core [None req-8728fdee-06e2-43be-b134-a6378151f9b8 None None] MFA Rules not processed for user `8813b7467a82484f9eec49e1817e4498`. Rule list: `[]` (Enabled: `True`). {{(pid=80377) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 02 10:34:20.131104 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.fernet_utils [None req-8728fdee-06e2-43be-b134-a6378151f9b8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80377) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 10:34:20.211567 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: [pid: 80377|app: 0|req: 1447/2880] 10.222.0.82 () {58 vars in 958 bytes} [Tue Aug 2 10:34:20 2022] POST /identity/v3/auth/tokens => generated 3317 bytes in 111 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 02 10:34:20.223205 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37654b86-7187-4290-99fc-d73af29fe82b admin admin] Authenticating user token {{(pid=80378) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:34:20.224952 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.fernet_utils [None req-37654b86-7187-4290-99fc-d73af29fe82b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80378) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 10:34:20.282518 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37654b86-7187-4290-99fc-d73af29fe82b admin admin] Validating token access rules against request {{(pid=80378) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:34:20.286755 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47feea06-cab6-427e-90bd-3bdd1f8f4bb9 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1023de281ae74e9d8a6d6c312c1c97a7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f1b38ebb804c41c69772c998aae72c41', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80378) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:34:20.287384 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47feea06-cab6-427e-90bd-3bdd1f8f4bb9 service cinder] REQUEST_METHOD: `GET` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:34:20.287668 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47feea06-cab6-427e-90bd-3bdd1f8f4bb9 service cinder] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:34:20.288017 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47feea06-cab6-427e-90bd-3bdd1f8f4bb9 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:34:20.320812 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG dbcounter [-] [80378] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80378) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 10:34:20.344375 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47feea06-cab6-427e-90bd-3bdd1f8f4bb9 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:34:20.345843 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47feea06-cab6-427e-90bd-3bdd1f8f4bb9 service cinder] RBAC: Authorization granted {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:34:20.355301 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 1434/2881] 10.222.0.82 () {60 vars in 1365 bytes} [Tue Aug 2 10:34:20 2022] GET /identity/v3/auth/tokens => generated 3248 bytes in 133 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 02 10:34:21.703938 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8728fdee-06e2-43be-b134-a6378151f9b8 None None] Authenticating user token {{(pid=80377) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:34:21.705321 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.fernet_utils [None req-8728fdee-06e2-43be-b134-a6378151f9b8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80377) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 10:34:21.718280 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8728fdee-06e2-43be-b134-a6378151f9b8 None None] Validating token access rules against request {{(pid=80377) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:34:21.722082 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2f4c2db-5ff3-4bca-8276-a0b43e13318d service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1023de281ae74e9d8a6d6c312c1c97a7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f1b38ebb804c41c69772c998aae72c41', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80377) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:34:21.722695 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2f4c2db-5ff3-4bca-8276-a0b43e13318d service cinder] REQUEST_METHOD: `GET` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:34:21.722981 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2f4c2db-5ff3-4bca-8276-a0b43e13318d service cinder] SCRIPT_NAME: `/identity` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:34:21.723261 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2f4c2db-5ff3-4bca-8276-a0b43e13318d service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:34:21.732288 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2f4c2db-5ff3-4bca-8276-a0b43e13318d service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:34:21.733718 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2f4c2db-5ff3-4bca-8276-a0b43e13318d service cinder] RBAC: Authorization granted {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:34:21.743810 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: [pid: 80377|app: 0|req: 1448/2882] 10.222.0.82 () {60 vars in 1365 bytes} [Tue Aug 2 10:34:21 2022] GET /identity/v3/auth/tokens => generated 3317 bytes in 40 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 02 10:34:30.351323 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG dbcounter [-] [80378] Writing DB stats keystone:SELECT=11 {{(pid=80378) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 10:34:31.739636 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG dbcounter [-] [80377] Writing DB stats keystone:SELECT=40 {{(pid=80377) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 10:34:34.636092 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47feea06-cab6-427e-90bd-3bdd1f8f4bb9 service cinder] Authenticating user token {{(pid=80378) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:34:34.698739 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47feea06-cab6-427e-90bd-3bdd1f8f4bb9 service cinder] Validating token access rules against request {{(pid=80378) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:34:34.703239 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06442128-0c8b-4893-8811-7cc91d20969c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '891005fa5ff74041b75747d16cc66798', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7047912c5acd4adeba5d87c3bf895e39', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80378) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:34:34.703884 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06442128-0c8b-4893-8811-7cc91d20969c admin admin] REQUEST_METHOD: `DELETE` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:34:34.704168 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06442128-0c8b-4893-8811-7cc91d20969c admin admin] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:34:34.704448 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06442128-0c8b-4893-8811-7cc91d20969c admin admin] PATH_INFO: `/v3/users/51278249ade046eb8f80d0f5c7743a62` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:34:34.705614 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06442128-0c8b-4893-8811-7cc91d20969c admin admin] RBAC: Authorizing `identity:delete_user(user_id=51278249ade046eb8f80d0f5c7743a62)` {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:34:34.707588 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06442128-0c8b-4893-8811-7cc91d20969c admin admin] RBAC: Authorization granted {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:34:34.753261 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.notifications [None req-06442128-0c8b-4893-8811-7cc91d20969c admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '51278249ade046eb8f80d0f5c7743a62'} {{(pid=80378) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 10:34:34.761552 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.notifications [None req-06442128-0c8b-4893-8811-7cc91d20969c admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '51278249ade046eb8f80d0f5c7743a62'} {{(pid=80378) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 10:34:34.768436 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.notifications [None req-06442128-0c8b-4893-8811-7cc91d20969c admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '51278249ade046eb8f80d0f5c7743a62'} {{(pid=80378) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 10:34:34.769071 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.notifications [None req-06442128-0c8b-4893-8811-7cc91d20969c admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '51278249ade046eb8f80d0f5c7743a62'} {{(pid=80378) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 10:34:34.781552 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.notifications [None req-06442128-0c8b-4893-8811-7cc91d20969c admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '51278249ade046eb8f80d0f5c7743a62'} {{(pid=80378) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 10:34:34.787571 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 1435/2883] 10.222.0.82 () {58 vars in 1224 bytes} [Tue Aug 2 10:34:34 2022] DELETE /identity/v3/users/51278249ade046eb8f80d0f5c7743a62 => generated 0 bytes in 152 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 02 10:34:34.963218 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2f4c2db-5ff3-4bca-8276-a0b43e13318d service cinder] Authenticating user token {{(pid=80377) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:34:34.974501 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2f4c2db-5ff3-4bca-8276-a0b43e13318d service cinder] Validating token access rules against request {{(pid=80377) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:34:35.029708 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3150488c-e16c-4bc2-9abd-b737b2674efa admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '891005fa5ff74041b75747d16cc66798', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7047912c5acd4adeba5d87c3bf895e39', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80377) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:34:35.030376 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3150488c-e16c-4bc2-9abd-b737b2674efa admin admin] REQUEST_METHOD: `DELETE` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:34:35.030656 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3150488c-e16c-4bc2-9abd-b737b2674efa admin admin] SCRIPT_NAME: `/identity` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:34:35.030932 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3150488c-e16c-4bc2-9abd-b737b2674efa admin admin] PATH_INFO: `/v3/projects/984f14e49b2e4d04a235fdb94d34462b` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:34:35.032081 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3150488c-e16c-4bc2-9abd-b737b2674efa admin admin] RBAC: Authorizing `identity:delete_project(project_id=984f14e49b2e4d04a235fdb94d34462b)` {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:34:35.034020 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3150488c-e16c-4bc2-9abd-b737b2674efa admin admin] RBAC: Authorization granted {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:34:35.080180 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.notifications [None req-3150488c-e16c-4bc2-9abd-b737b2674efa admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '984f14e49b2e4d04a235fdb94d34462b', 'request_id': 'req-3150488c-e16c-4bc2-9abd-b737b2674efa'} {{(pid=80377) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 10:34:35.085657 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.notifications [None req-3150488c-e16c-4bc2-9abd-b737b2674efa admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '984f14e49b2e4d04a235fdb94d34462b', 'request_id': 'req-3150488c-e16c-4bc2-9abd-b737b2674efa'} {{(pid=80377) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 10:34:35.091585 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.notifications [None req-3150488c-e16c-4bc2-9abd-b737b2674efa admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '984f14e49b2e4d04a235fdb94d34462b', 'request_id': 'req-3150488c-e16c-4bc2-9abd-b737b2674efa'} {{(pid=80377) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 10:34:35.103244 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.notifications [None req-3150488c-e16c-4bc2-9abd-b737b2674efa admin admin] The token cache is being invalidate because project 984f14e49b2e4d04a235fdb94d34462b was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80377) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 02 10:34:35.103532 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.notifications [None req-3150488c-e16c-4bc2-9abd-b737b2674efa admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80377) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 10:34:35.104157 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.notifications [None req-3150488c-e16c-4bc2-9abd-b737b2674efa admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80377) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 10:34:35.105644 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: [pid: 80377|app: 0|req: 1449/2884] 10.222.0.82 () {58 vars in 1233 bytes} [Tue Aug 2 10:34:34 2022] DELETE /identity/v3/projects/984f14e49b2e4d04a235fdb94d34462b => generated 0 bytes in 143 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 02 10:34:35.109658 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06442128-0c8b-4893-8811-7cc91d20969c admin admin] Authenticating user token {{(pid=80378) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:34:35.111136 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.fernet_utils [None req-06442128-0c8b-4893-8811-7cc91d20969c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80378) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 10:34:35.124744 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06442128-0c8b-4893-8811-7cc91d20969c admin admin] Validating token access rules against request {{(pid=80378) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:34:35.181444 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4aad45ec-3298-4a49-8009-194dde1c2c21 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '891005fa5ff74041b75747d16cc66798', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7047912c5acd4adeba5d87c3bf895e39', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80378) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:34:35.182379 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4aad45ec-3298-4a49-8009-194dde1c2c21 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:34:35.182780 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4aad45ec-3298-4a49-8009-194dde1c2c21 admin admin] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:34:35.183128 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4aad45ec-3298-4a49-8009-194dde1c2c21 admin admin] PATH_INFO: `/v3/users/8813b7467a82484f9eec49e1817e4498` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:34:35.184432 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4aad45ec-3298-4a49-8009-194dde1c2c21 admin admin] RBAC: Authorizing `identity:delete_user(user_id=8813b7467a82484f9eec49e1817e4498)` {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:34:35.186644 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4aad45ec-3298-4a49-8009-194dde1c2c21 admin admin] RBAC: Authorization granted {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:34:35.209532 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG dbcounter [-] [80378] Writing DB stats keystone:SELECT=86,keystone:DELETE=13,keystone:INSERT=1 {{(pid=80378) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 10:34:35.233811 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.notifications [None req-4aad45ec-3298-4a49-8009-194dde1c2c21 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '8813b7467a82484f9eec49e1817e4498'} {{(pid=80378) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 10:34:35.241400 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.notifications [None req-4aad45ec-3298-4a49-8009-194dde1c2c21 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '8813b7467a82484f9eec49e1817e4498'} {{(pid=80378) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 10:34:35.248289 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.notifications [None req-4aad45ec-3298-4a49-8009-194dde1c2c21 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '8813b7467a82484f9eec49e1817e4498'} {{(pid=80378) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 10:34:35.248926 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.notifications [None req-4aad45ec-3298-4a49-8009-194dde1c2c21 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '8813b7467a82484f9eec49e1817e4498'} {{(pid=80378) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 10:34:35.260566 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.notifications [None req-4aad45ec-3298-4a49-8009-194dde1c2c21 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '8813b7467a82484f9eec49e1817e4498'} {{(pid=80378) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 10:34:35.266334 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 1436/2885] 10.222.0.82 () {58 vars in 1224 bytes} [Tue Aug 2 10:34:35 2022] DELETE /identity/v3/users/8813b7467a82484f9eec49e1817e4498 => generated 0 bytes in 157 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 02 10:34:35.445519 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3150488c-e16c-4bc2-9abd-b737b2674efa admin admin] Authenticating user token {{(pid=80377) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:34:35.447111 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.fernet_utils [None req-3150488c-e16c-4bc2-9abd-b737b2674efa admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80377) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 10:34:35.460880 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3150488c-e16c-4bc2-9abd-b737b2674efa admin admin] Validating token access rules against request {{(pid=80377) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:34:35.520579 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55f96263-c695-4db1-969c-cb4cf5a9adb4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '891005fa5ff74041b75747d16cc66798', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7047912c5acd4adeba5d87c3bf895e39', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80377) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:34:35.522302 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55f96263-c695-4db1-969c-cb4cf5a9adb4 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:34:35.522589 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55f96263-c695-4db1-969c-cb4cf5a9adb4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:34:35.522898 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55f96263-c695-4db1-969c-cb4cf5a9adb4 admin admin] PATH_INFO: `/v3/projects/06530e678fda4a44875b57ba126ee314` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:34:35.524078 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55f96263-c695-4db1-969c-cb4cf5a9adb4 admin admin] RBAC: Authorizing `identity:delete_project(project_id=06530e678fda4a44875b57ba126ee314)` {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:34:35.526027 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55f96263-c695-4db1-969c-cb4cf5a9adb4 admin admin] RBAC: Authorization granted {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:34:35.560065 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG dbcounter [-] [80377] Writing DB stats keystone:SELECT=90,keystone:DELETE=9,keystone:INSERT=1 {{(pid=80377) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 10:34:35.574751 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.notifications [None req-55f96263-c695-4db1-969c-cb4cf5a9adb4 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '06530e678fda4a44875b57ba126ee314', 'request_id': 'req-55f96263-c695-4db1-969c-cb4cf5a9adb4'} {{(pid=80377) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 10:34:35.580204 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.notifications [None req-55f96263-c695-4db1-969c-cb4cf5a9adb4 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '06530e678fda4a44875b57ba126ee314', 'request_id': 'req-55f96263-c695-4db1-969c-cb4cf5a9adb4'} {{(pid=80377) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 10:34:35.586821 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.notifications [None req-55f96263-c695-4db1-969c-cb4cf5a9adb4 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '06530e678fda4a44875b57ba126ee314', 'request_id': 'req-55f96263-c695-4db1-969c-cb4cf5a9adb4'} {{(pid=80377) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 10:34:35.599071 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.notifications [None req-55f96263-c695-4db1-969c-cb4cf5a9adb4 admin admin] The token cache is being invalidate because project 06530e678fda4a44875b57ba126ee314 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80377) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 02 10:34:35.599363 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.notifications [None req-55f96263-c695-4db1-969c-cb4cf5a9adb4 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80377) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 10:34:35.599993 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.notifications [None req-55f96263-c695-4db1-969c-cb4cf5a9adb4 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80377) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 10:34:35.601491 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: [pid: 80377|app: 0|req: 1450/2886] 10.222.0.82 () {58 vars in 1233 bytes} [Tue Aug 2 10:34:35 2022] DELETE /identity/v3/projects/06530e678fda4a44875b57ba126ee314 => generated 0 bytes in 157 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 02 10:34:35.607236 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad498f12-2866-4877-9288-36f44ca46d16 None None] REQUEST_METHOD: `POST` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:34:35.607523 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad498f12-2866-4877-9288-36f44ca46d16 None None] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:34:35.607800 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad498f12-2866-4877-9288-36f44ca46d16 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:34:35.627074 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.auth.core [None req-ad498f12-2866-4877-9288-36f44ca46d16 None None] MFA Rules not processed for user `891005fa5ff74041b75747d16cc66798`. Rule list: `[]` (Enabled: `True`). {{(pid=80378) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 02 10:34:35.628199 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.fernet_utils [None req-ad498f12-2866-4877-9288-36f44ca46d16 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80378) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 10:34:35.685913 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 1437/2887] 10.222.0.82 () {58 vars in 958 bytes} [Tue Aug 2 10:34:35 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 80 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 02 10:34:35.694564 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6ecf881-21e2-46c6-9f9d-6478f5150f86 None None] REQUEST_METHOD: `POST` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:34:35.694848 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6ecf881-21e2-46c6-9f9d-6478f5150f86 None None] SCRIPT_NAME: `/identity` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:34:35.695135 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6ecf881-21e2-46c6-9f9d-6478f5150f86 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:34:35.714194 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.auth.core [None req-d6ecf881-21e2-46c6-9f9d-6478f5150f86 None None] MFA Rules not processed for user `891005fa5ff74041b75747d16cc66798`. Rule list: `[]` (Enabled: `True`). {{(pid=80377) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 02 10:34:35.715513 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.fernet_utils [None req-d6ecf881-21e2-46c6-9f9d-6478f5150f86 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80377) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 10:34:35.724014 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: [pid: 80377|app: 0|req: 1451/2888] 10.222.0.82 () {58 vars in 958 bytes} [Tue Aug 2 10:34:35 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 02 10:34:35.728208 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad498f12-2866-4877-9288-36f44ca46d16 None None] Authenticating user token {{(pid=80378) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:34:35.738019 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad498f12-2866-4877-9288-36f44ca46d16 None None] Validating token access rules against request {{(pid=80378) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:34:35.743105 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1785072b-63db-4c5c-874c-223214a321fe admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '891005fa5ff74041b75747d16cc66798', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7047912c5acd4adeba5d87c3bf895e39', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80378) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:34:35.743713 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1785072b-63db-4c5c-874c-223214a321fe admin admin] REQUEST_METHOD: `GET` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:34:35.743997 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1785072b-63db-4c5c-874c-223214a321fe admin admin] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:34:35.744282 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1785072b-63db-4c5c-874c-223214a321fe admin admin] PATH_INFO: `/v3/domains` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:34:35.744996 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1785072b-63db-4c5c-874c-223214a321fe admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80378) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 02 10:34:35.745312 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1785072b-63db-4c5c-874c-223214a321fe admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:34:35.746937 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1785072b-63db-4c5c-874c-223214a321fe admin admin] RBAC: Authorization granted {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:34:35.758844 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 1438/2889] 10.222.0.82 () {58 vars in 1153 bytes} [Tue Aug 2 10:34:35 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 02 10:34:35.762903 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6ecf881-21e2-46c6-9f9d-6478f5150f86 None None] Authenticating user token {{(pid=80377) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:34:35.771981 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6ecf881-21e2-46c6-9f9d-6478f5150f86 None None] Validating token access rules against request {{(pid=80377) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:34:35.776577 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a68968a-155b-48fc-8f85-62c159e456d6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '891005fa5ff74041b75747d16cc66798', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7047912c5acd4adeba5d87c3bf895e39', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80377) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:34:35.777200 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a68968a-155b-48fc-8f85-62c159e456d6 admin admin] REQUEST_METHOD: `POST` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:34:35.777485 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a68968a-155b-48fc-8f85-62c159e456d6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:34:35.777741 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a68968a-155b-48fc-8f85-62c159e456d6 admin admin] PATH_INFO: `/v3/projects` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:34:35.778434 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a68968a-155b-48fc-8f85-62c159e456d6 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:34:35.780301 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a68968a-155b-48fc-8f85-62c159e456d6 admin admin] RBAC: Authorization granted {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:34:35.807707 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: [pid: 80377|app: 0|req: 1452/2890] 10.222.0.82 () {60 vars in 1153 bytes} [Tue Aug 2 10:34:35 2022] POST /identity/v3/projects => generated 371 bytes in 45 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 02 10:34:35.811780 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1785072b-63db-4c5c-874c-223214a321fe admin admin] Authenticating user token {{(pid=80378) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:34:35.822629 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1785072b-63db-4c5c-874c-223214a321fe admin admin] Validating token access rules against request {{(pid=80378) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:34:35.878001 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a6276bd-355a-4992-8bff-e2d8355afe1b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '891005fa5ff74041b75747d16cc66798', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7047912c5acd4adeba5d87c3bf895e39', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80378) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:34:35.878556 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a6276bd-355a-4992-8bff-e2d8355afe1b admin admin] REQUEST_METHOD: `POST` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:34:35.878840 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a6276bd-355a-4992-8bff-e2d8355afe1b admin admin] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:34:35.879112 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a6276bd-355a-4992-8bff-e2d8355afe1b admin admin] PATH_INFO: `/v3/users` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:34:35.879785 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a6276bd-355a-4992-8bff-e2d8355afe1b admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:34:35.882351 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a6276bd-355a-4992-8bff-e2d8355afe1b admin admin] RBAC: Authorization granted {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:34:35.903094 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 1439/2891] 10.222.0.82 () {60 vars in 1144 bytes} [Tue Aug 2 10:34:35 2022] POST /identity/v3/users => generated 286 bytes in 92 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 02 10:34:35.907254 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a68968a-155b-48fc-8f85-62c159e456d6 admin admin] Authenticating user token {{(pid=80377) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:34:35.917436 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a68968a-155b-48fc-8f85-62c159e456d6 admin admin] Validating token access rules against request {{(pid=80377) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:34:35.922027 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d705a64c-d5fd-475f-8301-db10a7857b57 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '891005fa5ff74041b75747d16cc66798', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7047912c5acd4adeba5d87c3bf895e39', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80377) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:34:35.922632 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d705a64c-d5fd-475f-8301-db10a7857b57 admin admin] REQUEST_METHOD: `GET` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:34:35.922916 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d705a64c-d5fd-475f-8301-db10a7857b57 admin admin] SCRIPT_NAME: `/identity` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:34:35.923192 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d705a64c-d5fd-475f-8301-db10a7857b57 admin admin] PATH_INFO: `/v3/roles` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:34:35.923890 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d705a64c-d5fd-475f-8301-db10a7857b57 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:34:35.925506 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d705a64c-d5fd-475f-8301-db10a7857b57 admin admin] RBAC: Authorization granted {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:34:35.935724 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: [pid: 80377|app: 0|req: 1453/2892] 10.222.0.82 () {58 vars in 1122 bytes} [Tue Aug 2 10:34:35 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 02 10:34:35.939565 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a6276bd-355a-4992-8bff-e2d8355afe1b admin admin] Authenticating user token {{(pid=80378) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:34:35.950186 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a6276bd-355a-4992-8bff-e2d8355afe1b admin admin] Validating token access rules against request {{(pid=80378) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:34:35.954728 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9e28fc8-4e1e-44e1-8e1b-c27b6925eb44 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '891005fa5ff74041b75747d16cc66798', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7047912c5acd4adeba5d87c3bf895e39', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80378) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:34:35.955323 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9e28fc8-4e1e-44e1-8e1b-c27b6925eb44 admin admin] REQUEST_METHOD: `GET` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:34:35.955607 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9e28fc8-4e1e-44e1-8e1b-c27b6925eb44 admin admin] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:34:35.955882 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9e28fc8-4e1e-44e1-8e1b-c27b6925eb44 admin admin] PATH_INFO: `/v3/roles` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:34:35.956567 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9e28fc8-4e1e-44e1-8e1b-c27b6925eb44 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:34:35.958218 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9e28fc8-4e1e-44e1-8e1b-c27b6925eb44 admin admin] RBAC: Authorization granted {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:34:35.968919 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 1440/2893] 10.222.0.82 () {58 vars in 1122 bytes} [Tue Aug 2 10:34:35 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 02 10:34:35.973093 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d705a64c-d5fd-475f-8301-db10a7857b57 admin admin] Authenticating user token {{(pid=80377) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:34:35.983312 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d705a64c-d5fd-475f-8301-db10a7857b57 admin admin] Validating token access rules against request {{(pid=80377) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:34:35.988290 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56ed7abc-3c21-4004-b2a7-7e359042e107 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '891005fa5ff74041b75747d16cc66798', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7047912c5acd4adeba5d87c3bf895e39', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80377) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:34:35.988956 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56ed7abc-3c21-4004-b2a7-7e359042e107 admin admin] REQUEST_METHOD: `PUT` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:34:35.989238 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56ed7abc-3c21-4004-b2a7-7e359042e107 admin admin] SCRIPT_NAME: `/identity` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:34:35.989515 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56ed7abc-3c21-4004-b2a7-7e359042e107 admin admin] PATH_INFO: `/v3/projects/bb96b0e610b34e3d8e7c94b1842e0d45/users/1e06212915c04a30b0965022b843855c/roles/e1c04b19fc4543b8b54f58bbcffbc598` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:34:36.007165 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56ed7abc-3c21-4004-b2a7-7e359042e107 admin admin] RBAC: Authorizing `identity:create_grant(project_id=bb96b0e610b34e3d8e7c94b1842e0d45, user_id=1e06212915c04a30b0965022b843855c, role_id=e1c04b19fc4543b8b54f58bbcffbc598)` {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:34:36.009845 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56ed7abc-3c21-4004-b2a7-7e359042e107 admin admin] RBAC: Authorization granted {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:34:36.023290 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: [pid: 80377|app: 0|req: 1454/2894] 10.222.0.82 () {60 vars in 1483 bytes} [Tue Aug 2 10:34:35 2022] PUT /identity/v3/projects/bb96b0e610b34e3d8e7c94b1842e0d45/users/1e06212915c04a30b0965022b843855c/roles/e1c04b19fc4543b8b54f58bbcffbc598 => generated 0 bytes in 51 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 02 10:34:36.033723 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75c50ee3-2ddb-4241-8aa0-63db4b793434 None None] REQUEST_METHOD: `POST` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:34:36.034129 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75c50ee3-2ddb-4241-8aa0-63db4b793434 None None] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:34:36.034508 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75c50ee3-2ddb-4241-8aa0-63db4b793434 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:34:36.042718 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG dbcounter [-] [80378] Writing DB stats keystone:DELETE=7,keystone:SELECT=89,keystone:INSERT=4 {{(pid=80378) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 10:34:36.062827 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.auth.core [None req-75c50ee3-2ddb-4241-8aa0-63db4b793434 None None] MFA Rules not processed for user `1e06212915c04a30b0965022b843855c`. Rule list: `[]` (Enabled: `True`). {{(pid=80378) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 02 10:34:36.064036 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.fernet_utils [None req-75c50ee3-2ddb-4241-8aa0-63db4b793434 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80378) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 10:34:36.126820 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 1441/2895] 10.222.0.82 () {58 vars in 958 bytes} [Tue Aug 2 10:34:36 2022] POST /identity/v3/auth/tokens => generated 3248 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 02 10:34:36.138629 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56ed7abc-3c21-4004-b2a7-7e359042e107 admin admin] Authenticating user token {{(pid=80377) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:34:36.140297 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.fernet_utils [None req-56ed7abc-3c21-4004-b2a7-7e359042e107 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80377) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 10:34:36.220473 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56ed7abc-3c21-4004-b2a7-7e359042e107 admin admin] Validating token access rules against request {{(pid=80377) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:34:36.224728 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-778123ef-0d50-4cb4-be96-46a79e62a168 service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa964af53a2b4fb9ace6afb544d54d40', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f1b38ebb804c41c69772c998aae72c41', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80377) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:34:36.225346 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-778123ef-0d50-4cb4-be96-46a79e62a168 service glance] REQUEST_METHOD: `GET` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:34:36.225638 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-778123ef-0d50-4cb4-be96-46a79e62a168 service glance] SCRIPT_NAME: `/identity` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:34:36.225949 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-778123ef-0d50-4cb4-be96-46a79e62a168 service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:34:36.234076 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-778123ef-0d50-4cb4-be96-46a79e62a168 service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:34:36.235506 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-778123ef-0d50-4cb4-be96-46a79e62a168 service glance] RBAC: Authorization granted {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:34:36.244844 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: [pid: 80377|app: 0|req: 1455/2896] 10.222.0.82 () {60 vars in 1365 bytes} [Tue Aug 2 10:34:36 2022] GET /identity/v3/auth/tokens => generated 3248 bytes in 107 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 02 10:34:46.123918 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG dbcounter [-] [80378] Writing DB stats keystone:SELECT=30 {{(pid=80378) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 10:34:46.240221 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG dbcounter [-] [80377] Writing DB stats keystone:SELECT=74,keystone:DELETE=5,keystone:INSERT=3 {{(pid=80377) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 10:35:10.259744 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75c50ee3-2ddb-4241-8aa0-63db4b793434 None None] Authenticating user token {{(pid=80378) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:35:10.363544 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75c50ee3-2ddb-4241-8aa0-63db4b793434 None None] Validating token access rules against request {{(pid=80378) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:35:10.368182 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9051e84b-9d1b-4d93-a382-928da5d2a065 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '891005fa5ff74041b75747d16cc66798', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7047912c5acd4adeba5d87c3bf895e39', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80378) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:35:10.368821 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9051e84b-9d1b-4d93-a382-928da5d2a065 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:35:10.369107 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9051e84b-9d1b-4d93-a382-928da5d2a065 admin admin] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:35:10.369386 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9051e84b-9d1b-4d93-a382-928da5d2a065 admin admin] PATH_INFO: `/v3/users/1e06212915c04a30b0965022b843855c` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:35:10.370598 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9051e84b-9d1b-4d93-a382-928da5d2a065 admin admin] RBAC: Authorizing `identity:delete_user(user_id=1e06212915c04a30b0965022b843855c)` {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:35:10.372543 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9051e84b-9d1b-4d93-a382-928da5d2a065 admin admin] RBAC: Authorization granted {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:35:10.418845 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.notifications [None req-9051e84b-9d1b-4d93-a382-928da5d2a065 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '1e06212915c04a30b0965022b843855c'} {{(pid=80378) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 10:35:10.426036 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.notifications [None req-9051e84b-9d1b-4d93-a382-928da5d2a065 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '1e06212915c04a30b0965022b843855c'} {{(pid=80378) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 10:35:10.432871 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.notifications [None req-9051e84b-9d1b-4d93-a382-928da5d2a065 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '1e06212915c04a30b0965022b843855c'} {{(pid=80378) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 10:35:10.433517 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.notifications [None req-9051e84b-9d1b-4d93-a382-928da5d2a065 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '1e06212915c04a30b0965022b843855c'} {{(pid=80378) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 10:35:10.445950 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.notifications [None req-9051e84b-9d1b-4d93-a382-928da5d2a065 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '1e06212915c04a30b0965022b843855c'} {{(pid=80378) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 10:35:10.451701 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 1442/2897] 10.222.0.82 () {58 vars in 1224 bytes} [Tue Aug 2 10:35:10 2022] DELETE /identity/v3/users/1e06212915c04a30b0965022b843855c => generated 0 bytes in 192 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 02 10:35:10.456130 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c22d66ee-0c29-4586-ada9-09d5349c00c1 None None] REQUEST_METHOD: `POST` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:35:10.456655 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c22d66ee-0c29-4586-ada9-09d5349c00c1 None None] SCRIPT_NAME: `/identity` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:35:10.456770 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c22d66ee-0c29-4586-ada9-09d5349c00c1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:35:10.496431 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.auth.core [None req-c22d66ee-0c29-4586-ada9-09d5349c00c1 None None] MFA Rules not processed for user `891005fa5ff74041b75747d16cc66798`. Rule list: `[]` (Enabled: `True`). {{(pid=80377) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 02 10:35:10.497603 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.fernet_utils [None req-c22d66ee-0c29-4586-ada9-09d5349c00c1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80377) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 10:35:10.554641 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: [pid: 80377|app: 0|req: 1456/2898] 10.222.0.82 () {58 vars in 958 bytes} [Tue Aug 2 10:35:10 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 02 10:35:10.564854 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9051e84b-9d1b-4d93-a382-928da5d2a065 admin admin] Authenticating user token {{(pid=80378) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:35:10.566296 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.fernet_utils [None req-9051e84b-9d1b-4d93-a382-928da5d2a065 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80378) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 10:35:10.606140 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG dbcounter [-] [80378] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80378) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 10:35:10.650235 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9051e84b-9d1b-4d93-a382-928da5d2a065 admin admin] Validating token access rules against request {{(pid=80378) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:35:10.653947 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c10c9b1-aaa5-4564-9abb-ed314c18f1d7 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '33b87170f0634b67bce60750847b5cbe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f1b38ebb804c41c69772c998aae72c41', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80378) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:35:10.654569 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c10c9b1-aaa5-4564-9abb-ed314c18f1d7 service neutron] REQUEST_METHOD: `GET` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:35:10.654854 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c10c9b1-aaa5-4564-9abb-ed314c18f1d7 service neutron] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:35:10.655131 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c10c9b1-aaa5-4564-9abb-ed314c18f1d7 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:35:10.664608 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c10c9b1-aaa5-4564-9abb-ed314c18f1d7 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:35:10.666076 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c10c9b1-aaa5-4564-9abb-ed314c18f1d7 service neutron] RBAC: Authorization granted {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:35:10.675694 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 1443/2899] 10.222.0.82 () {60 vars in 1365 bytes} [Tue Aug 2 10:35:10 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 111 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 02 10:35:10.949689 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c22d66ee-0c29-4586-ada9-09d5349c00c1 None None] Authenticating user token {{(pid=80377) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:35:10.951262 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.fernet_utils [None req-c22d66ee-0c29-4586-ada9-09d5349c00c1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80377) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 10:35:10.966721 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c22d66ee-0c29-4586-ada9-09d5349c00c1 None None] Validating token access rules against request {{(pid=80377) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:35:10.972216 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42e66bad-34e7-4026-8023-1bbf501f33a8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '891005fa5ff74041b75747d16cc66798', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7047912c5acd4adeba5d87c3bf895e39', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80377) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:35:10.972952 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42e66bad-34e7-4026-8023-1bbf501f33a8 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:35:10.973234 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42e66bad-34e7-4026-8023-1bbf501f33a8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:35:10.973502 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42e66bad-34e7-4026-8023-1bbf501f33a8 admin admin] PATH_INFO: `/v3/projects/bb96b0e610b34e3d8e7c94b1842e0d45` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:35:10.974690 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42e66bad-34e7-4026-8023-1bbf501f33a8 admin admin] RBAC: Authorizing `identity:delete_project(project_id=bb96b0e610b34e3d8e7c94b1842e0d45)` {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:35:10.976598 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42e66bad-34e7-4026-8023-1bbf501f33a8 admin admin] RBAC: Authorization granted {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:35:11.022553 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.notifications [None req-42e66bad-34e7-4026-8023-1bbf501f33a8 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'bb96b0e610b34e3d8e7c94b1842e0d45', 'request_id': 'req-42e66bad-34e7-4026-8023-1bbf501f33a8'} {{(pid=80377) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 10:35:11.028194 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.notifications [None req-42e66bad-34e7-4026-8023-1bbf501f33a8 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'bb96b0e610b34e3d8e7c94b1842e0d45', 'request_id': 'req-42e66bad-34e7-4026-8023-1bbf501f33a8'} {{(pid=80377) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 10:35:11.034725 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.notifications [None req-42e66bad-34e7-4026-8023-1bbf501f33a8 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'bb96b0e610b34e3d8e7c94b1842e0d45', 'request_id': 'req-42e66bad-34e7-4026-8023-1bbf501f33a8'} {{(pid=80377) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 10:35:11.046667 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.notifications [None req-42e66bad-34e7-4026-8023-1bbf501f33a8 admin admin] The token cache is being invalidate because project bb96b0e610b34e3d8e7c94b1842e0d45 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80377) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 02 10:35:11.046952 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.notifications [None req-42e66bad-34e7-4026-8023-1bbf501f33a8 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80377) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 10:35:11.048168 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.notifications [None req-42e66bad-34e7-4026-8023-1bbf501f33a8 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80377) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 10:35:11.049666 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: [pid: 80377|app: 0|req: 1457/2900] 10.222.0.82 () {58 vars in 1233 bytes} [Tue Aug 2 10:35:10 2022] DELETE /identity/v3/projects/bb96b0e610b34e3d8e7c94b1842e0d45 => generated 0 bytes in 101 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 02 10:35:12.160352 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c10c9b1-aaa5-4564-9abb-ed314c18f1d7 service neutron] Authenticating user token {{(pid=80378) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:35:12.161762 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.fernet_utils [None req-2c10c9b1-aaa5-4564-9abb-ed314c18f1d7 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80378) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 10:35:12.251795 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c10c9b1-aaa5-4564-9abb-ed314c18f1d7 service neutron] Validating token access rules against request {{(pid=80378) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:35:12.309180 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-233ab18b-d11e-4b3a-82b8-a966264bf587 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ae6e910e50d74f29bfb89bf18d95a908', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f1b38ebb804c41c69772c998aae72c41', 'project_domain_id': 'default', 'roles': ['admin', 'service', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80378) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:35:12.309822 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-233ab18b-d11e-4b3a-82b8-a966264bf587 service placement] REQUEST_METHOD: `GET` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:35:12.310170 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-233ab18b-d11e-4b3a-82b8-a966264bf587 service placement] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:35:12.310443 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-233ab18b-d11e-4b3a-82b8-a966264bf587 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:35:12.312303 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.fernet_utils [None req-233ab18b-d11e-4b3a-82b8-a966264bf587 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80378) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 10:35:12.324192 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-233ab18b-d11e-4b3a-82b8-a966264bf587 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:35:12.325773 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-233ab18b-d11e-4b3a-82b8-a966264bf587 service placement] RBAC: Authorization granted {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:35:12.335922 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 1444/2901] 10.222.0.82 () {60 vars in 1365 bytes} [Tue Aug 2 10:35:12 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 176 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 02 10:35:21.043795 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG dbcounter [-] [80377] Writing DB stats keystone:SELECT=63,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80377) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 10:35:22.331444 dv-cinder-847132-4-iscsi devstack@keystone.service[80378]: DEBUG dbcounter [-] [80378] Writing DB stats keystone:SELECT=87 {{(pid=80378) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 10:40:16.133315 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42e66bad-34e7-4026-8023-1bbf501f33a8 admin admin] Authenticating user token {{(pid=80377) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:40:16.135351 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.fernet_utils [None req-42e66bad-34e7-4026-8023-1bbf501f33a8 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80377) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 10:40:16.150434 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42e66bad-34e7-4026-8023-1bbf501f33a8 admin admin] Validating token access rules against request {{(pid=80377) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:40:16.156518 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-182c1aa5-0601-430e-9b5d-eb56840558ee service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ae6e910e50d74f29bfb89bf18d95a908', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f1b38ebb804c41c69772c998aae72c41', 'project_domain_id': 'default', 'roles': ['admin', 'service', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80377) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:40:16.157139 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-182c1aa5-0601-430e-9b5d-eb56840558ee service placement] REQUEST_METHOD: `GET` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:40:16.157424 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-182c1aa5-0601-430e-9b5d-eb56840558ee service placement] SCRIPT_NAME: `/identity` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:40:16.157699 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-182c1aa5-0601-430e-9b5d-eb56840558ee service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:40:16.166460 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-182c1aa5-0601-430e-9b5d-eb56840558ee service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:40:16.168031 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-182c1aa5-0601-430e-9b5d-eb56840558ee service placement] RBAC: Authorization granted {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:40:16.177847 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: [pid: 80377|app: 0|req: 1458/2902] 10.222.0.82 () {60 vars in 1365 bytes} [Tue Aug 2 10:40:16 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 45 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 02 10:40:26.173692 dv-cinder-847132-4-iscsi devstack@keystone.service[80377]: DEBUG dbcounter [-] [80377] Writing DB stats keystone:SELECT=6 {{(pid=80377) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}}